16 August 2008
Visual Studio's unit testing framework isn't widely covered in books or in other places, and this makes it tougher for teams to adopt strict test guidelines for their developers. At the end of the day, if you aren't unit testing from day one of your project, you're in for a world of hurt down the line.
If you've new to Visual Studio unit testing, or just wanting to learn about Silverlight unit testing, you may feel a little lost. Here's some information that just scratches the surface of the Visual Studio unit testing world.
So here are some of the questions people may have, let me know if there are others that you have! If you're already familiar with unit testing this will all be redundant.
Most unit tests are simple class libraries written in your language of choice. They reference your production application's projects or built assemblies, and also the unit test framework.
Metadata decorates the tests, and the tests themselves typically include calls to the test framework's assertion classes.
Patterns and examples:
For testing desktop applications, class libraries, and other code built for the full .NET framework, you're in for a treat: Visual Studio 2008 Professional and other SKUs put great built in unit test support right inside your development environment. You'll always be a click away from running a test or seeing where you stand.
There are integration tests, end-to-end tests, performance and stress tests, and many, many more. But at Microsoft I like to think of tests in 3 sets. In some cases it is possible for all the tests in a product to build upon a 'unit test framework'.
Unit: "True" unit tests. Quick, small, efficient, very useful to always run. Independent.
Developer: Unit tests plus tests that exercise simple scenarios and conditions, such as the control framework in WPF, or performing a super simple request. Often these reach outside of the typical "unit test boundaries," but are still quick to run and useful.
Developers are typically required to run all of the developer tests on their machine when preparing a check-in to validate the state of the code.
All tests: The set of all developer tests and everything that the test development team has created. Sometimes reaches into the tens of thousands of scenarios and cases at the end of a product cycle once a matrix of platforms and configurations is taken into consideration.
Developers and testers work together to make sure that all tests are run regularly, and especially when working on approaching a major product milestone, release, or servicing activity.
When building out the Silverlight unit test system that runs tests inside of your web browser, we made the call to reuse Visual Studio's unit test framework metadata. This is a great win, for a few reasons:
Make sure that your unit tests are actually small, independent, simple, and not dependent on the web browser, Silverlight itself, or the more advanced "beyond unit testing" capabilities that are also present in Microsoft.Silverlight.Testing. If you inherit from the "SilverlightTest" base class, you probably have a test that won't port to the desktop .NET framework.
We've worked hard to provide a subset of the full Visual Studio unit test metadata, but here's a quick guide for you:
OK to use in both worlds:
Desktop features that will not work with the Silverlight test framework today (or are no-ops):
Silverlight test features that will not port to the desktop:
Hope this helps get some of you started!
Jeff Wilcox is a Principal Software Engineer at Microsoft in the Open Source Programs Office, helping Microsoft scale to 15,000+ engineers using, contributing to and releasing open source.