Archive for the “.NET Application Architecture” Category
I recently had the opportunity to look into and make use of the Microsoft System.Security.SecureString class. This class is one of those dark corners of the .NET Framework that you don’t think about on a day-to-day basis but are really glad that it’s there when your security auditor starts asking questions about how PII data such as social security numbers are protected while resident in memory. The SecureString class takes care of this problem, helping you avoid a situation where unencrypted sensitive String data is left lingering around on the .NET heap. However, since this class does reference unmanaged memory buffers, its use is not entirely intuitive. I’ve attempted to demystify things with the explanation, drawing and code snippets in this post.
The diagram below shows that, in the case of System.String, what you get is an unencrypted string located in managed memory. Due to the immutability of String objects and the nondeterministic nature of the .NET Garbage Collector, the need for one string may result in multiple string objects scattered across managed memory, waiting to be compromised.
In the case of a SecureString, you don’t have an unsecure String in managed memory. Instead, you get a DPAPI encrypted array of characters in unmanaged memory. And, since SecureString implements the IDisposable interface, it’s easy to deterministically destroy the string’s secure contents. There are some limited .NET 4.0 Framework classes that accept SecureStrings as parameters, including the cryptographic service provider (CSP), X.509 certificate classes and several other security related classes. But what if you want to create your own classes that accept and deal with secure strings? How do you deal with the SecureString from managed .NET code and how do you ensure that you don’t defeat the purpose of the SecureString by leaving intermediate strings unsecure in memory buffers?
The simple console application below exhibits how a SecureString can be properly used and disposed; with the SecureString contents being made available to managed code and the intermediate memory zeroed out when no longer needed.
static void Main(string args)
// Wrapping the SecureString with using causes it to be properly
// disposed, leaving no sensitive data in memory
using (SecureString SecString = new SecureString())
Console.Write("Please enter your password: ");
ConsoleKeyInfo CKI = Console.ReadKey(true);
if (CKI.Key == ConsoleKey.Enter) break;
// Use the AppendChar() method to add characters
// to the SecureString
// Make the SecureString read only
// Display password by marshalling it from unmanaged memory
// Example demonstrating what needs to be done to get SecureString value to
// managed code. This method uses unsafe code; project must be compiled
// with /unsafe flag in the C# compiler
private unsafe static void DisplaySecureString(SecureString SecString)
IntPtr StringPointer = Marshal.SecureStringToBSTR(SecString);
// Read the decrypted string from the unmanaged memory buffer
String NonSecureString = Marshal.PtrToStringBSTR(StringPointer);
// Zero and free the unmanaged memory buffer containing the
// decrypted SecureString
This example should be useful to you in working SecureString into your own application. Like any other security measure, there’s a cost to the additional security. In the case of the SecureString, there’s overhead to adding characters to the SecureString as well as marshaling data out of unmanaged memory. The final reference example I’ll provide is from Microsoft’s SecureString implementation, specifically the code to initialize the secure string. From this code, you can clearly see the check for platform dependencies, buffer allocation, pointer creation and the ProtectMemory() call which invokes the Win32 native RtlEncryptMemory function.
private unsafe void InitializeSecureString(char* value, int length)
this.m_length = length;
byte* pointer = null;
Buffer.memcpyimpl((byte*) value, pointer, length * 2);
if (pointer != null)
No Comments »
One of the things I was really eager to do was help one of our clients manage the archival and history of projects within their TFS repository. Historically, VSS volumes sizes have gotten out of control over time, resulting in commensurately poor performance. Obviously, a SQL Server backing database offers lots of advantages over the Jet database engine but even SQL Server performance will degrade over time as the history volume in long-running projects backs up.
I was hoping that TFS 2008 had built in functionality to manage project archiving and history management. Not only does the TFS 2008 not have such a function but the co-mingling of data (all the projects on a server write to the same database) means that it’s nearly impossible to break out what data belongs to what project and apply different types of information lifecycle management rules such as modifying the type of storage used, applying specific backup criteria to different projects, or taking a project completely offline so that it no longer impacts the performance of the TFS database but can still be retained for historical purposes.
The good news is that, if you’re willing to make the move, TFS 2010 has functionality to explicitly address the requirement for TFS archiving and history management. TFS 2010 Team Project Collections allow you to organize similar projects into collections and, most importantly for our needs, allocate a different set of hardware resources for each team project collection. The benefit of this setup and applicability to the intent of this blog post should be immediately obvious. The downside of this approach is that you can’t work (link work items, branch & merge, etc.) across project collections. An annotated version of a diagram from the MSDN Team Project Collections documentation can be found below.
No Comments »
Everybody loves lists of tools. Scott Hanselman’s annual list of Windows tools has been immensely popular over the years and has opened my eyes to a bunch of new tools. The topic of tools has also been the subject of some very popular books, such as Windows Developer Power Tools and Java Power Tools.
These tool discussions are also recurring themes on all of the major discussion forums. It seems that every so often one of these questions hits StackOverflow and everyone chimes in with their favorite current tools. Invariably, for the .NET tool lists, there are some tools that always show up and; enjoying near universal advocacy in the .NET developer community. This includes tools like Reflector and Fiddler on the free side and Ants Profiler and Resharper on the commercial side.
For this blog post, I’ve decided to go with 5 tools you’re not likely to find on any/many of these lists. While some of these tools are .NET-specific, other tools are just solid development tools that are likely to be great additions to any .NET team’s toolbox with the added benefit that they work across multiple technologies.
- Badboy. Likely the biggest sleeper on my list. Badboy is an extremely easy-to-learn web application testing tool. Check out the online documentation to understand features and then use it to guide your learning. Chances are that you’ll have most of the basic and intermediate level scripting tasks mastered within the first 30 minutes of using the tool. Compare the cost of a Badboy license ($45 / individual or $30 / each for a 10-pack) with the cost of your existing web application testing tool. Chances are you’d be saving hundreds, if not thousands of dollars per license. If you need to scale beyond simple Badboy threading / load testing capabilities, Badboy scripts can be exported in a format consumable by Apache JMeter for more heavy duty controller/generator type load testing. Also, the Wave Test Manager server, from the makers of Badboy, allows you to upload and share badboy scripts across a project, schedule execution of the scripts, and access the reports from the tests on a central server.
- Lightspeed ORM. When the discussion of Object Relational Mappers (ORMs) comes up, NHibernate and the Entity Framework are almost always at the forefront of the conversation. LLBLGen gets added to the list as well if commercial ORM’s are on the table. Rarely, if ever, is the Lightspeed ORM from the Mindscape team down under ever brought up. It should be. If an awesome Visual Studio modeling experience and second generation LINQ provider don’t convince you, maybe the Rails’esque data migration facilities will. Still not convinced? Check out the custom LinqPad provider and LINQ-to-SQL to Lightspeed drag and drop conversion. If there are new features you’d like to see or if you need bug fixes, Ivan and the team at Mindscape are all ears and provide a near legendary turn around time.
- Silverlight Spy. Let’s recap just in case you missed the news – Silverlight is hot!!! It’s a pretty significant change from either the MVC or WebForms approach most .NET web developers are used to and takes a while to wrap your mind around. Silverlight Spy does for Silverlight what Reflector did for the .NET Framework, pulls back the covers so that you can inspect and understand. Silverlight Spy provides insight into the XAP package, isolated storage information, performance data, an accessibility view and so much more. The message from Microsoft over the last 6 months has been – learn Silverlight. That task is made so much easier with Silverlight Spy at your side.
- DTM Data Generator. Microsoft recently finally got around to including a data generator in some versions of Visual Studio. If you restrict yourself to SQL Server and are willing to deal with slow data generation, it might even be a good fit for you. RedGate’s SQL Data Generator, which I’ve written about before is much more efficient at loading data, as long as you stick with SQL Server. If you’re looking for data generation tool to meet your needs, irrespective of the underlying database you use, DTM’s Data Generator is the tool for you. DTM’s data generator supports SQL Server, Oracle, MySQL, DB2, Sybase, and any database you can access through OLE DB or a DSN. It supports inserts of most major datatypes, including BLOB generation and supports a variety of rules comparable to RedGate’s product, including the use of custom rules. The enterprise version can be executed from the command line in silent mode, making it perfect for generation of data in preparation for the execution of an automated test suite.
- Performance Analysis of Logs (PAL). This tool just doesn’t get enough love from the .NET development community. Oft maligned as the “poor man’s SCOM”, PAL can be a real timesaver and/or lifesaver. It’s so simple: capture the PAL specified counters for the platform being monitored (most major MS products such as Windows Server, IIS, MOSS, SQL Server, BizTalk, Exchange, and AD are supported), import the counters and let PAL do its thing. It’s “thing” is producing a detailed report for the counters showing how they looked across the duration of the capture and when the counters exceeded thresholds. PAL also provides explanations for each of the counters and details the implications are of exceeding the thresholds. More useful information for a better price you will not find.
7 Comments »
Performance counters for WCF have been available ever since the first release of WCF with the .NET 3.0 Framework. As long as these counters have been available, Microsoft has been cautioning about the memory requirements and potential performance degradation associated with insufficient shared memory allocation. I thought that I had heard at the PDC that WCF 4 would fix some of this but going back to the WCF session video, it looks as if these counters won’t really be addressed by WCF 4 but instead superseded by the ETW instrumentation present in AppFabric. So, until everyone moves to AppFabric, I see a need for a bit more guidance than the “allocate enough memory” that Microsoft offers us.
Enabling WCF Performance Counters
Enabling WCF performance counters is a breeze and is covered pretty well elsewhere. The configuration change below will turn on all three types of WCF counters: Endpoint, Operation, and Service.
<diagnostics performanceCounters="All" />
Your options for enabling the counters are: All, ServiceOnly, and Off. WCF performance counters are included for a reason so I wouldn’t recommend disabling them entirely. Instead, as a rule of thumb you should enable “All” if you’re performing specific service debugging activities that require all the counters and should leave on “ServiceOnly” for normal operations, including in a production environment.
Calculating Performance Counter Memory Size
Before diving into sizing, it’s best to provide a bit of background on performance counter memory allocation. Managed performance counters consume memory that is shared across all the .NET processes running on a machine; essentially a memory-mapped file. Although the .NET Framework 1.0 and 1.1 used global shared memory, .NET 2.0 and above use separate shared memory per performance counter category, with each category having a default size of approximately 128KB (that is ¼ the default global shared memory).
You also need to know about services, endpoints, operations – the WCF counter groups:
- Services. Services are at the root of the WCF hierarchy. Services can have multiple endpoints and expose multiple operations. WCF has 33 performance counters for each service
- Endpoints. WCF endpoints provide the client access to a service through address, binding, and contract. You can provide multiple endpoints per service. WCF has 19 performance counters for each instance of an endpoint across a service.
- Operations. A WCF service operation is a discrete function performed by a WCF service. WCF provides 15 performance counters: per endpoint, per service.
What you’re ultimately looking to come up with is a sizing for each one of the WCF performance counter categories. Without providing a mathematical formula, I’ll walk through a brief hypothetical example to calculate the sizes. In this example, I’ll assume that we have 20 services on a machine, each of these services has 3 endpoints, and each service has 10 operations exposed across each of the three endpoints:
- We’ll assume an average size per performance counter of 350 bytes, which is a fairly conservative yet accurate estimate.
- For the service counters, we have 20 services * 33 performance counters * 350 bytes = 231,000 bytes (231 KB)
- For the endpoint counters, we’ll need 20 services * 3 endpoints * 19 performance counters * 350 bytes = 399,000 bytes (399 KB)
- Operations counters come in the largest, due to the multiplicative effect, at 20 services * 3 endpoints * 10 operations * 15 counters * 350 bytes = 3,150,000 bytes (3.15 MB).
From the above numbers, you’ll hopefully notice two things. First, I hope you now understand why I recommend the “ServiceOnly” setting unless you’re in an environment where you absolutely need the other counters. Second, even with a medium size service load, we’ve exceeded the default performance counter category maximum memory and are quickly heading for the dreaded “System.InvalidOperationException: Custom counters file view is out of memory” exception.
Setting Performance Counter Memory Size
Aside from the mechanics of setting the performance counter category memory size, there is only so much guidance I can provide. What you will set these values to will depend upon a couple of factors:
- Whether you’ve set WCF counters to “ServiceOnly” or to “All”. If you’ve used the former, you’ll only need to tweak the service-specific private memory. If you go with “All”, you’ll want to set each category’s memory space individually.
- The math you do for your counter categories based upon the example I provided in the previous section.
For the size of separate shared memory, the DWORD FileMappingSize value in the registry key HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServices<category name>Performance is referenced first, followed by the value specified for the global shared memory in the configuration file. If the FileMappingSize value does not exist, then the separate shared memory size is set to ¼ the global shared memory setting in the configuration file, which is 528KB by default.
To specify the WCF category-specific sizes, simply set the registry value for each of the three registry keys associated with the three WCF service categories and then reboot the machine. Keep in mind that, like other application sizing activities, sizing of the WCF counter memory will need to be repeated as the number of services, endpoints, and operations change on a particular machine.
2 Comments »
As many of you likely know, Lutz Roeder turned over control of one of the “must have” .NET developer tools, .NET Reflector to Red Gate software. True to their promise, RedGate has continued to support the free version of Reflector and make continued improvements, including the addition of a Visual Studio plugin to jump into Reflector and the support of .NET 4 assemblies with their most recent release of the tool.
In addition to their support of the free tool, RedGate has extended Reflector’s core disassembly capabilities and is now offering a commercial version of the tool, Reflector Pro. RedGate has rolled the tool into their .NET Developer Bundle. So, if you hold existing .NET Developer Bundle licenses, you can pick up Reflector Pro and start using it. So I did.
Reflector Pro integrates right into Visual Studio (including the VS 2010 RC), which is really important given its core competency. What it allows, simply stated, is for you to step through any assembly in Visual Studio as if it were your own. This is a killer feature that you almost never need… until you really need it. Reflector Pro does this by disassembling assemblies of your choosing and then generating the corresponding debug symbols you need to perform a variety of functions:
- Stepping into third party libraries
- Setting breakpoints in third party libraries
- Watch and modify values in third party libraries
Like the free version of Reflector we’ve all grown to love, the Pro version of the software isn’t needed all the time and it just works the way you want when you need it. RedGate’s site (http://www.red-gate.com/products/reflector/index.htm) provides a simple video demo and walkthrough. More your should not need to get going with this tool.
No Comments »