.NET news » Search results

Many problems you will face are actually the building data access layer, sometimes thinly disguised, sometimes in your face; it’s one of the broad patterns that you see in computer science – as the cliché says: it keeps rearing its ugly head.

Despite this, the same sorts of mistakes tend to be made in the design of such systems so I’d like to offer a bit of hard-won advice on how to approach a data access problem. Mostly this is going to be in the form of patterns/anti-patterns but nonetheless I hope it will be useful.

As always, in the interest of not writing a book, this advice is only approximately correct.

The main thing that you should remember is that access to the data will take two general shapes.  In database parlance you might say some of the work will be OLTP-ish (online transaction processing) and some of the work will be OLAP-ish (online analytical processing).   But simply, there’s how you update pieces of your data and how your read chunks of it.  And they have different needs.

At present it seems to me that people feel a strong temptation to put an OO interface around the data and expose that to customers.  This can be ok as part of the solution if you avoid some pitfalls, so I suggest you follow this advice:

1. Consider the unit of work carefully

There are likely to be several typical types of updates.  Make sure that you fetch enough data so that the typical cases do one batch of reads for necessary data, modify the data locally, and then write back that data in a batch.  If you read too much data you incur needless transfer costs, if you read too little data then you make too many round trips to do the whole job.

You may have noticed that I began with a model where you fetch some data, change it locally, and write it back.  This is a fairly obvious thing to do given that you are going to want to do the write-back in probably a single transaction but it’s important to do this even if you aren’t working in a transacted system.  Consider an alternative:  if you were to provide some kind of proxy to the data to each client and then RPC each property change back to the server you are in a world of hurt.  Now the number of round trips is very high and furthermore it’s impossible to write correct code because two people could be changing the very same object at the same time in partial/uncoordinated ways. 

This may seem like a silly thing to do but if the authoritative store isn’t a database it’s all too common for people to forget that the database rules exist for a reason and they probably apply to any kind of store at all.  Even if you’re using (e.g.) the registry or some other repository you still want to think about unit-of-work and make it so the each normal kind of update is a single operation.

Whatever you do don’t create an API where each field read/write is remoted to get the value.  Besides the performance disaster this creates it’s impossible to understand what will happen if you several people are doing something like Provider.SomeValue += 1;

2. Consider your locking strategy

Implicit in the discussion above is some notion of accepting or rejecting writes because the data has changed out from under you.  This is a normal situation and making it clear that it can and does happen and should be handled makes everyone’s life simpler.  This is another reason why an API like Provider.SomeValue = 1 to do the writes is a disaster.  How does it report failure?  And if it failed, how much failed?

You can choose an optimistic locking strategy or something else but you’ll need one.  A sure sign that you have it right is that the failure mode is obvious, and the recovery is equally obvious. 

I once had a conversation with Jim Gray where I told him how ironic it was to me that the only reason transactions could ever succeed at all in a hot system was that they had the option of failing.  Delicious irony that.

Remember, even data from a proxy isn’t really live.  It’s an illusion.  The moment you say “var x = provider.X;”  your ‘x’ is already potentially stale by the time it’s assigned.  Potentially stale data is the norm, it’s just a question of how stale and how do you recover. That means some kind of isolation and locking choice is mandatory.

3. Don’t forget the queries

Even if you did everything else completely correctly you’ve still only built half the system if all you can do is read and modify entities.  The OLAP part of your system will want to do bulk reads like “find me all the photos for this week”.  When doing these types of accesses it is vital to take advantage of their read aspect.  Do not create transactable objects just bring back the necessary data in a raw form.  Simple arrays of primitives are the best choice; they have the smallest overheads.  Do not require multiple round-trips to get commonly required data or the fixed cost of the round trip will end up dwarfing the actual work you're trying to do.

These queries are supposed to represent a snapshot in time according to whatever isolation model your data has (which comes back to the requirements created by your use cases and your unit of work).  If you force people to use your object interface to read raw data you will suffer horrible performance and you will likely have logically inconsistent views of your data.  Don’t do that.

One of the reasons that systems like Linq to SQL were as successful as they were (from various perspectives I think) is that they obeyed these general guidelines:

you can get a small amount of data or a large amount of data you can get objects or just data you can write back data chunks in units of your choice the failure mode for read/writes is clear, easy to deal with, and in-your-face (yes, reads can fail, too)

Other data layers, while less general no doubt, would do well to follow the same set of rules.

 

 

The Microsoft Chart controls are a series of classes in the System.Web.UI.DataVisualization.Charting namespace that allow web developers to ability to add charts to their ASP.NET applications. The most pertinent charting-related class is the Chart class, which contains information about the chart's appearance, series, charting areas, and so forth. In most of the demos and code samples we've explored thus far, we've used the Chart class as a Web control, adding the <asp:Chart> declarative markup to our ASP.NET page, setting a few properties and, occasionally, writing a few lines of code. When used as a Web control, the Chart class both creates the chart (as an image) and then renders an <img> element that points to the generated chart image.

Using the Chart Web control is a standard practice in a WebForms application, but it is not suggested when building an ASP.NET MVC application. (While it is possible to add Web controls - including the Chat Web control - to the views of an ASP.NET MVC application, it is generally frowned upon.) So, if we can't use the Chart Web control in an ASP.NET MVC application, how do we display a chart? In addition to being used as a Web control, the Chart class can also be used programmatically. It is quite possible to create a new Chart object, set some properties, plot the data points, and then generate the chart image. In fact, we looked at using this technique in an earlier installment, Programmatically Generating Chart Images, in which we saw (among other things) how to generate chart images programmatically and add them as attachments in an email message.

This article explores how to display charts in an ASP.NET MVC application.

With the Windows Consumer Preview out the door, I thought it would be interesting to write something about creating great performing Windows applications.  I hope to have a lot more to say about this in the future but I think really the most important things I have to say are more inspirational rather than informational at this point. The key point is that things are quite a bit different than they were, and how you think about creating an application with great performance is perhaps different than it ever has been.

OK that’s a mouthful, what am I talking about?

Well one thing is the operating system itself is a lot more frugal. Improvements in Win7 and continuing improvements in Win8 mean you get more of your system resources available for your applications. Great news!  And what are those applications going to do with those resources?  Why use them of course!  Hopefully wisely, to create great experiences when they are active and economically fade to nothing when they are not.  As we’ll see, resource usage is more temporal in Win8 and so management is essential.

Back when I worked on the Whidbey CLR we thought a modest forms-based application ought to be able to start up in something like 50ms and use something roughly like 5MB of working set.

In many ways we considered the working set number to be too big. We addressed that at least in part by making as much of that memory shared as possible – good for amortizing the cost over many processes.  Big fractions of the private memory came from things like the GC heap’s initial allocation, and pre-committed stack pages plus other fixed costs but much of it had to do the raw cost of getting a CLR application going.  Still that’s where we were:  50ms and maybe 5MB, fighting to go down from there.

Fast forward to 2012 and things look a lot different.  A metro style application might spend say 9 frames @60 Hz fading in a splash screen (call it 150ms) and then another 9 fading it out.  And then maybe as many doing some cool slide-in type effect as the title comes in from one side and the data from the other.  These kinds of effects are part of the Metro look and feel.  Now at that point we haven’t actually done anything per se and we’ve spent maybe 450ms just on animations.

Memory wise, to do those transitions we probably have spent private memory to the tune of 3 full screen surfaces.  Easily 4MB each (at 1366x768) with probably 2 live at any given moment and that’s just the memory for the video.  Clearly the world is a different place.  And already the temporal nature of resource usage is showing clearly, those costs are short term.

So what does it mean?

This metro style application experience, despite the fact that it takes longer to start up, is actually much more pleasing to use.  It responded instantly to our command and fit nicely into the expected design.  I think that’s the most important observation: in a metro style application we do get some more latitude to use video memory especially for effects but the price we pay is that we have even less latitude for stalls.  People expect a smooth responsive system at all stages.  No video glitches, no jerkiness, no synchronous pauses to get resources, no long wait-cursors.  The expectation has been set from the first frame that your experience will be a lovely one and applications that don’t deliver will look like old klunkers.

Let’s also consider where this extra memory came from: video memory is in many ways the cheapest in terms of time-to-get-ready.  In previous frameworks we were dealing with mostly code that had to be brought in and working set was a good proxy to startup time because working set tends to translate directly to disk IO.  Clean video surfaces don’t have to be paged in so that (nearly) 1:1 relationship between working set and startup time is ancient history.  Which isn’t to say you can now load all the code you like: you can’t, especially not all at once because page faults turn into glitches.  Like all your other resources you’ll want to get them loaded asynchronously and have a good UI experience while they are coming in.  That means don’t load your whole program at startup, don’t initialize everything you might ever need, and create suitable transitions and incremental experiences for data and program parts arriving.  In a great application all of that is part of the overall experience.

This is why I think it’s even harder on the programmer to get all these things right and why it’s even more important than ever for the frameworks to give you a Pit of Success experience when you use the normal templates and the normal services to get your job done.   If you aren’t careful then you’ll destroy the intended experience and even “a little bit broken” will look terrible in contrast to the expectation set by the design.

Compounding all of this is our desire to run metro style applications on lower end hardware.  Some of these systems are equipped with decent GPUs (though not the greatest)  but their CPUs offer a raw compute speed that might be say 1/6th (on a per core basis) of a high end desktop.  If you want your application to run well in that environment you’ll need to think carefully about how much work you do in one transition and how you use asynchronous patterns to keep you application feeling crisp even with CPU resources that are less abundant.   Careful testing and measuring on these lower end systems will be essential because your normal development machine could delude you with speed you don’t have.

And as if that’s not enough, almost universally power consumption is a huge concern.  In unplugged scenarios, whether it’s laptop or tablet every joule consumed matters.  In some cases a core feature of the tablet is the excellent battery life and hence the lower power CPU and so forth.  As an application or framework author you must consider these things very carefully.  The system will want to suspend your application when it isn’t running – how will this affect the way you store your state?  You will want to have lovely animations while your application is active because that’s what a great application does but you’ll want to be able to turn those off when you aren’t visible.  Background activity generally should be looked at with great suspicion because it will drain power.  What background work must happen?  What could be reduced or suspended entirely?    Great choices will mean superior battery life which could be a huge aspect of your success.

Despite these challenges I’m optimistic.  I think the Metro design is a great one because has a lovely look while at the same time it is not so terribly hard to create that you must spend all resources to make it.  The Metro look and feel is largely composed of things that GPUs do comparatively well.  That’s good news for everyone and it’s a good basis to create your applications.  Asynchronous design patterns are possible in all the languages WinRT supports and especially in Javascript, which is likely to be the most popular choice by volume. Asynchronous patterns are already normal for most javascript libraries.

Bottom line: what constitutes a great performing windows application has changed in the metro world,  but great performance is still important, perhaps more important than ever.

As I said above, I hope to write more about these concerns in the coming weeks but for now I'd like to refer you to some articles that have already been written on MSDN.  It's not everything by a longshot but its something.  While this particular work discusses the problems in the context of javascript, most of the advice is actually broadly applicable no matter what framework you use.   Check it out the guidance here.

When creating ASP.NET 2.0 applications, developers commonly store sensitive configuration information in the Web.config file. The cannonical example is database connection strings, but other sensitive information included in the Web.config file can include SMTP server connection information and user credentials, among others. While ASP.NET is configured, by default, to reject all HTTP requests to resources with the .config extension, the sensitive information in Web.config can be compromised if a hacker obtains access to your web server's file system. For example, perhaps you forgot to disallow anonymous FTP access to your website, thereby allowing a hacker to simply FTP in and download your Web.config file. Eep.

Fortunately ASP.NET 2.0 helps mitigate this problem by allowing selective portions of the Web.config file to be encrypted, such as the section, or some custom config section used by your application. Configuration sections can be easily encrypted using code or aspnet_regiis.exe, a command-line program. Once encrypted, the Web.config settings are safe from prying eyes. Furthermore, when retrieving encrypted congifuration settings programmatically in your ASP.NET pages, ASP.NET will automatically decrypt the encrypted sections its reading. In short, once the configuration information in encrypted, you don't need to write any further code or take any further action to use that encrypted data in your application.

In this article we'll see how to programmatically encrypt and decrypt portions of the configuration settings and look at using the aspnet_regiis.exe command-line program. We'll then evaluate the encryption options ASP.NET 2.0 offers. There's also a short discussion on how to encrypt configuration information in ASP.NET version 1.x.

A ZIP file is a popular, decades-old file format used for file compression and archiving. Commonly, such files have a .zip extension and are used to reduce the size of one or more files and/or to archive multiple files and folders into a single file. Additionally, the contents of a ZIP file can optionally be encrypted and viewable only by those who know the password. Both Microsoft's Windows and Apple's Mac OS provide built-in operating system support for opening, reading, and creating ZIP files.

In a recent project I needed to create ZIP files from an ASP.NET application on the fly. Specifically, there was a web page that listed of series of data files that were created by an external process. Users visiting this page select a file to download, which then displays a dialog box in their browser, allowing them to open the file or save it to their hard drive. This user interface worked for most of our users, as they were only interested in downloading one or two files at most. However, some of our users needed to download upwards of 20 files. For them, clicking a download link, saving it to their hard drive, and repeating, 20 times, was frustrating and time consuming. To improve this user interface we created the notion of "download profiles," which allow users to associate a name - like "Accounting Files" - with a collection of file types that are available for download. After creating a "download profile," a user could then choose to download all available files that belong to that profile. This would create a ZIP file with the appropriate files and display a dialog box in the user's browser, allowing them to open or save the ZIP. With this enhancement, our power users can now download their 20 files with one mouse click.

This article starts with a look at different ways to create ZIP files in an ASP.NET application, but then focuses on using the free and open-source DotNetZip library.

One of the most sure-fire ways to improve a web application's performance is to employ caching. Caching takes some expensive operation and stores its results in a quickly accessible location. ASP.NET version 1.0 introduced two flavors of caching:

• Output Caching - caches the entire rendered markup of an ASP.NET web page or User Control for a specified duration.
• Data Caching - a programmatically-accessible, in-memory data cache for storing objects in the web server's memory.

For a more in-depth discussion on ASP.NET 1.x's caching capabilities, refer to Scott McFarland's Caching with ASP.NET and Steve Smith's ASP.NET Caching: Techniques and Best Practices articles.

In ASP.NET 2.0, the caching system has been extended to include SQL cache dependencies, cache profiles, and post-cache substitution for output cached pages. The Caching for Performance section of the ASP.NET 2.0 QuickStarts provides a good overview of ASP.NET 2.0's caching options. This article explores output caching in ASP.NET 2.0, starting with an overview of output caching and followed by a detailed look at creating pages that include both cached and non-cached markup using fragment caching and post-cache substitution techniques.

In .NET applications, an illegal operation - an invalid cast, attempting to reference a null value, trying to connect to a database that's been taken offline, and so on - raises an exception. Exceptions can be caught and handled directly in code through the use of Try / Catch blocks. For ASP.NET applications, if the exception is not handled in code, it bubbles up to the ASP.NET runtime, which raises an HttpUnhandledException. By default, unhandled exceptions result in a page that displays the text, "Runtime Error" with instructions for developers on how to display exception details (see the screen shot to the right). This "Runtime Error" error page is what is seen by external visitors; if you visit your site through localhost and an unhandled exception occurs, the default error page includes the type and details of the exception thrown.

End users will no doubt find the "Runtime Error" page to be intimidating and confusing - do you think the average computer user knows what "Runtime" means? All the user knows is that something went horribly wrong. They might fear that their data or progress has been lost and that they are responsible for the error. Ironically, the person who does care that an unhandled exception has occurred - the developer - is left out of the loop unless the end user takes the time to email the developer the details of the error (what page it happened on, the steps the user had performed that caused the error, and so on)..