B. Tools and frameworks

B.1.1 Moq

Moq is an open source newcomer to the world of Mocking, and has adopted an API that tries to be both very simple to learn, and very easy to use. The API also follows the AAA (Arrange Act Assert) style (vs.the Record-Replay style) and relied heavily on .NET 3.5 features such as lambdas and extension methods. If you’re planning working with .NET 3.5 exclusively this is a relatively pain free learning curve (but you need to feel really comfortable with using lambdas).

Feature wise is has less features than most other mock frameworks, which also means its simpler to learn.

You can get mock at http://code.google.com/p/moq/

B.1.2 Rhino Mocks

Rhino Mocks is a widely used open source framework for mocks and stubs. It’s also the framework used throughout this book for examples. You can find out more about it in chapter 5. It is loaded with features and has recently moved to using the new AAA syntax (used to have record-replay syntax).

You can get Rhino Mocks at http://ayende.com/projects/rhino-mocks.aspx

B.1.3 Typemock Isolator

Typemock Isolator is a powerful commercial Isolation framework which tries to remove the terms “mocks” and “stubs” from its vocabulary in favor of a more simple and terse API. It differs from the other frameworks by allowing to “isolate” components from their dependencies regardless of how the system is designed (although it supports all the other features the other frameworks have). This makes it ideal for people who are getting into unit testing and want a more incremental approach to learning. Because it does not force design for testability, you can learn to write tests correctly and then move on to learning better design, without having the two being mixed together as a necessity.(disclosure: while writing this book I’ve also been working at Typemock)

You can get Typemock Isolator at www.Typemock.com

B.1.4 NMock

NMock is an open source mocking framework which started out as a direct port of JMock from Java. It used to be the de facto mocking framework until Rhino took its place in the open source world. The main reason it was de-throwned is because it did not offer strong-typing of method names (you had to use strings to define expectations on methods). Lately it has been getting back into development after sitting quietly for a while, and its new released (2.0) marks a nice change in the API. It remains to be seen how well it will take vs. the current competition.

You can get NMock at http://www.NMock.org/

B.1.5 NUnit.Mocks

NUnit.Mocks is an ultra small open source mocking framework that comes built in with Nunit as a separate dll. It initially started its life as an aid for testing NUnit internally and NUnit’s authors still discourage using it in your own projects since it may “disappear” in the coming future. It is still one of the simplest frameworks to learn with, but I don’t recommend using it for anything other than a learning exercise.

You can get NUnit.Mocks as part of the installation of NUnit at www.Nunit.org

B.2.1 Microsoft Unit Testing Framework

The Microsoft Unit Testing Framework (also known as “MS Test”) comes bundled in with any version of visual studio .NET 2008 professional or above. It allows basic features similar to NUnit but runs a little slower. The upcoming versions of Visual Studio (2010) will add a lot of power to this framework, but you can use it today just as easily as you can with NUnit.

One big con in this framework is that it’s not easily extensible as the other test frameworks. To see how cumbersome it is to add a simple attribute to it, see http://code.msdn.microsoft.com/yunit .

One big plus about it is that it integrated well as part of the Visual Studio Team System tool suite, and provide good reporting, coverage and build automation out of the box. If your company uses team system, I highly suggest using MS Test as your Test Framework of choice for the good integration possibilities.

You can get MS Test built in with Visual Studio.

B.2.2 NUnit

NUnit is currently the “De-facto” test framework for unit test developers in .NET. It’s open source and in almost ubiquitous use among those who unit test. I cover Nunit deeply in chapter 2. NUnit is easily extensible and contains a large user base and forums. As such I’d recommend it to anyone starting out with unit testing in .NET. I still personally use it today.

You can get Nunit at www.Nunit.org

B.2.3 MbUnit

The “mb” in MbUnit stands for “Model-based” testing. It is fully open source. It started out as a competitor to Nunit and soon zoomed passed NUnit in terms of features and abilities. It has its own UI and console runners which also support running tests written in NUnit. mbUnit is easily extensible and supports lots of interesting test attributes such as “Repeat” or “Timeout”. If you are looking for something more in your current test framework, MbUnit is a good way up from NUnit. Note, that I personally almost never have to use such features, but if you are doing part-integration testing and part unit testing with the same framework, mbUnit is a good fit.

You can get MbUnit at www.mbunit.com

B.2.4 Gallio

The Gallio Automation Platform is an open, extensible, and neutral system for .NET that provides a common object model, runtime services and tools (such as test runners) that may be leveraged by any number of test frameworks.

Gallio seeks to facilitate the creation of a rich ecosystem of interoperable testing tools produced by and for the community at large so as to address a far wider range of testing needs than any previous independent testing framework has done before. Unit tests, integration tests, performance tests, and even semi-automated test harnesses should all be able to leverage common facilities where appropriate. Moreover, they should present consistent interfaces to the world so that they can easily be integrated into the systems and processes of the enterprise.

At present Gallio can run tests from MbUnit versions 2 and 3, MSTest, NBehave, NUnit, and xUnit.Net.Gallio provides tool support and integration with CCNet, MSBuild, NAnt, NCover, Pex, Powershell, Resharper, TestDriven.Net, TypeMock, and Visual Studio Team System.

Gallio also includes its own command-line runner, Echo, and a Windows GUI, Icarus. Additional runners are planned or under development. Third parties are also encouraged to leverage the Gallio platform as part of their own applications.

You can get gallio at www.gallio.org

B.2.5 XUnit

XUnit is an open source test framework, developed in cooperation with one of the original authors of Nunit, Jim Newkirk. It is a minimalistic and elegant test framework that tries to “get back to basics” by having less features, not more, than the other frameworks, and by supporting different names on its attributes.

Because it reads so radically different than the other frameworks, it takes a while to get used to if you have experience with say, NUnit or MbUnit. If you’ve never used any test framework before, xunit is very easy to grasp and use, and is robust enough to be used in a real project.

What is so radically different about it? It has no Setup or TearDown methods, for example (you have to use the constructor and a “dispose” method on the test class. Another big difference is in how easy it is to extend it.

For more information and download see http://www.codeplex.com/xunit

B.2.6 Pex

Pex (Program EXploration) is an intelligent assistant to the programmer. From a parameterized unit test, it automatically produces a traditional unit test suite with high code coverage. In addition, it suggests to the programmer how to fix the bugs.  

With Pex you can created special tests that have parameters in them, and put special attributes on those tests. The Pex engine will generate new tests that you can later run as part of your test suite. It is great for finding corner cases and edge conditions that are not handled properly in your code.

You should use pex in addition to a regular test framework, such as NUnit or MbUnit.

You can get it at http://research.microsoft.com/projects/pex/

B.3.1 StructureMap

StuctureMap is an open source DI framework that has one clear differentiator from the others. It’s API is very fluent and tries to mimic natural language and generic constructs as much as possible. It has a relatively small user base and the current documentation on it is lacking at best, but it contains some very powerful features such as a built in auto-mocking container (a container that can created stubs automatically when requesting an object), powerful lifetime management, XML-less configuration, integration with ASP.NET and more.

You can get it at http://structuremap.sourceforge.net

B.3.2 Microsoft Unity (Recommended)

Unity is a latecomer to the DI Container field, but provides a very simple and minimalistic approach that can be easily learned and used by beginners. Advanced users may find it lacking but it certainly answers my 80-20 rule. It provides 80% of the features you look for most of the time.

It’s also open source by Microsoft, and has good documentation. I’d recommend it to start with a DI container.

You can get it at http://www.codeplex.com/unity

B.3.3 Castle Windsor

The Castle project is a large open source project that covers a lot of areas. One of those areas is called “Windsor” and it provides a mature and powerful implementation of a DI container. It contains most of the features you’d ever want in a container and more, but has a relatively high learning curve due to its high amount of features.

You can learn about Castle Windsor at http://www.castleproject.org/container/ and download the castle project at http://www.castleproject.org  

B.3.4 Autofac (Auto Factory)

Autofac is a fresh approach toIoCin .NET that fits well with C# 3.0 syntax. It was designed with modern .NET features andobsessiveobject-orientation in mind. The API is radically different than the other frameworks, and required a bit of getting used to. It also required .NET 3.5 to work and a good knowledge of lambda syntax. It’s hard to explain, so you’ll have to go to the site and see how different it is. I recommend using it for people who already have experience with other DI frameworks.  

You can get it at http://code.google.com/p/autofac/

B.3.5 Common Service Locator Library

The Common Service Locator library contains a shared interface for service location which application and framework developers can reference. The library provides an abstraction over IoC containers and service locators. Using the library allows an application to indirectly access the capabilities without relying on hard references. The hope is that using this library, third-party applications and frameworks can begin to leverage IoC/Service Location without tying themselves down to a specific implementation.

The project contains a full test suite which validates a particular implementation of the locator meets the functional requirements. Additionally, several adapters for working with popular IoC containers will be included within the project at a future date.

You can get the Common Service Locator library at http://www.codeplex.com/CommonServiceLocator

B.3.6 Spring.NET

Spring.NET is an open sourceDI framework. The design of Spring.NET is based on the Java version of the Spring Framework, which has shown real-world benefits and is used in thousands of enterprise applications world wide.  As such, it is one of the oldest DI frameworks for .NET, with the most “baggage” but with loads of features. Spring .NET is not a quick port from the Java version, but rather a 'spiritual port' based on following proven  architectural and design patterns in that are not tied to a particular platform.   

You can get Spring.NET at http://www.springframework.net

B.3.7 Microsoft Managed Extensibility Framework

The Managed Extensibility Framework (MEF) enables greater reuse of applications and components. Using MEF, .NET applications can make the shift from being statically compiled to dynamically composed. If you are building extensible applications, extensible frameworks and application extensions, then MEF is for you.  

MEF includes a lightweight DI framework so you can easily inject dependencies into various places in your code by using special attributes.  

It does require a bit of a learning curve, and I wouldn’t recommend using it solely as a DI framework. However, if you do use it for extensibility features in your application, it can also be used as a DI container.  

You can get MEF at http://www.codeplex.com/MEF

B.3.8 Ninject

Ninject is a latecomer to the DI field but has a very simple syntax to learn, with very good usability. There isn’t much else to day about it except that I’d highly recommend taking a look at it at http://ninject.org/

B.4.1 Use integration tests for your data layer

Lots of people have asked me how I think they should test their data layer. Should they abstract away the database interfaces? Should they use the real database?

My answer is that I usually write integration style test for my data layer (they later in my app that talks directly to the database). The reason for this is that data logic is almost always divided between the application logic and the database itself (triggers, security rules, referential integrity..). unless we can test the database logic in complete isolation(I’ve found no really good framework for this purpose, only some really bad ones), the only way to make sure it works in our tests is to couple testing the data layer logic to the real database.

By testing the data layer and the database together, we leave little surprises for later on in the project in those places.

Testing against the database, however, has it’s problems. It’s main problem is that you are testing against shared state by many tests. Insert a line to the DB in one test, the other test can see that line as well.

We need a way to roll back the changes we made to the database, and thankfully, there is good support for this in the current test tools and the .NET framework.

B.4.2 Using Rollback attributes

The three major frameworks, MbUnit, NUnit and XUnit support the special [Rollback] attribute that you can put on top of your test method. When used, the attribute creates a special database transaction that the test code runs in. when the test is finished, the DB transaction is rolled back automatically and the changes to the DB vanish.

To learn more about how this works visit an MSDN article I wrote a while back called “Simplify Data Layer Unit Testing using Enterprise Services” at http://msdn.microsoft.com/en-us/magazine/cc163772.aspx

If you are not interested in using the Rollback attributes the frameworks provide, or are testing in other means, you can use the very simple class introduced in .NET 2.0 called TransactionScope.

B.4.3 Using TransactionScope to Rollback

For examples on how to use this class in your setup and teardown code see this blog article: http://www.bbits.co.uk/blog/archive/2006/07/31/12741.aspx called “Even simpler Database Unit Testing with TransactionScope”

B.5.1 Ivonna

Ivonna is a commercial Asp.Net testing tool that allows writing unit tests for your WebForms applications. What makes it different is that Ivonna runs in the same process as the test runner.

Unlike other testing tools, Ivonna is not a "client-side" tool. It doesn't just test the HTML output (although such a possibility exists), it creates and runs an Asp.Net request and allows you to examine the intrinsic objects, such as the Page object. This opens many new possibilities, such as examining properties of page controls and other objects (such as the HttpContext object), Faking external dependencies in order to isolate the page behavior, Injecting setup code and assertions into the page's lifecycle event handlers and more.

Ivonna is being developed in partnership with TypeMock and runs as an add-on to the TypeMock Isolator framework.

B.5.2 Team System Web Test

Visual Studio Team Test includes the powerful ability to record and replay web requests for pages, and verify various things during these runs. This is strictly integration testing, but it is really powerful. The latest versions also support recording ajax actions on the page, and make things much easier to test.

B.5.3 NUnitASP

NUnitAsp is no longer being supported but still has a wide user base. It is a class library for use within your NUnit tests. It provides NUnit with the ability to download, parse, and manipulate ASP.NET web pages.

With NUnitAsp, your tests don't need to know how ASP.NET renders controls into HTML. Instead, you can rely on the NUnitASP library to do this for you, keeping your test code simple and clean. For example, your tests don't need to know that a DataGrid control renders as an HTML table. You can rely on NUnitAsp to handle the details. This gives you the freedom to focus on functionality questions, like whether the DataGrid holds the expected values.

Most people end up using it for acceptance testing. In those cases, it would be better to use frameworks such as Watin and Watir, described later. You can get it at http://nunitasp.sourceforge.net/

B.5.4 Watir

Watir(pronounced “water”) stands for “Web application testing in Ruby”. It is a simple open-source library for automating web browsers written in Ruby. It allows you to write tests that are easy to read and easy to maintain. It is optimized for simplicity and flexibility.

Watir drives browsers the same way people do. It clicks links, fills in forms, presses buttons. Watir also checks results, such as whether expected text appears on the page.

Watir works with Internet Explorer on Window and is currently being ported to support Firefox and Safari.

The big problem is that it has a learning curve: you need to learn Ruby to use it. Other than that it is quite a lovely framework for integration testing the browser.

You can get it at http://wtr.rubyforge.org/

B.5.5 Watin

Inspired by Watir, development of WatiN(pronounced “what-in”) started in December 2005 to make a similar kind of Web Application Testing possible for the .Net languages. Since then WatiN has grown into an easy to use, feature rich and stable framework. WatiN is developed in C# and aims to bring you an easy way to automate tests with Internet Explorer.

If you know .NET you can learn to use this framework, and it is quite powerful.

You can get it at http://watin.sourceforge.net/

B.5.6 Selenium

Selenium is a suite of tools to automate web app testing across many platforms. It has existed longer than all the other frameworks in this list, and also has an API wrapper for .NET. It is an integration testing framework in very wide use, and is a good place to start. Beware – it has many features and the learning curve is high.

You can get it at http://selenium.openqa.org/

B.6.1 NUnitForms

NUnitForms is an NUnit extension for unit and acceptance testing of Windows Forms applications.It aimes to make its easy to write automated tests for your Windows Forms (UI) classes. NUnitForms tests can open a window and interact with the controls. The tests will automatically manipulate and verify the properties of the gui.

NUnitForms takes care of cleaning up your forms between tests, detecting and handling modal dialog boxes, and verifying that your expectations for the test are fulfilled.

You can get it at http://nunitforms.sourceforge.net/

B.6.2 Project White

Project White is, in a way, a successor to NUnitForms, in that it supports a richer set of application types (WPF, WinForm, Win32 and Java JWT), and sports a newer API with better usability. Unline NUnitForms, it is more of an integration test framework, since it allows spawning separate processes that it tests against. It uses the UI Automation API (part of windows) to do its bidding, which gives it much more power. This of it as Selenium or Watin for Winforms.

You can get it at http://www.codeplex.com/white

B.6.3 Team System UI Tests

The upcoming version of Team System will support a new kind of test – UI Test. You will be able to record actions on UI windows and then play them back and verify assertions during test runs. As will all team system tools, its main advantage will be its integration with other team system tools, reports, source control and servers.

B.7.1 Typemock Racer

Typemock Racer is a managed framework for multi threaded code testing, that helps visualize, detect and resolve deadlocks and race conditions in managed code. You use it by simply putting an attribute on top of an existing test method and the engine does all the work. It also allows full thread visualization during debugging of a threaded test.

It is a commercial product that works with all flavors of Visual Studio (including Express) and all test frameworks. You can get it at www.Typemock.com

(Disclosure: during the writing of this book I’ve joined the developer team at Typemock)

B.7.2 Microsoft Chess

CHESS is an automated tool for finding errors in multithreaded software by systematic exploration of thread schedules. It finds errors, such as data-races, deadlocks, hangs, and data-corruption induced access violations, that are extremely hard to find with current testing tools. Once CHESS locates an error, it provides a fully repeatable execution of the program leading to the error, thus greatly aiding the debugging process. In addition, CHESS provides a valuable and novel notion of test coverage suitable for multithreaded programs. CHESS can use existing concurrent test cases and is therefore easy to deploy. Both developers and testers should find CHESS useful.

It currently supports native code, but a managed .NET version should be coming out soon as part of Visual Studio Team System offering (commercial product).

You can get it at http://research.microsoft.com/projects/CHESS/

B.7.3 Osherove.ThreadTester

This is a little open source framework I developed a while back. It allows running multiple threads during one test to see if anything weird happens to your code (deadlocks, for example). It is not feature complete, but it is a good attempt at a multithreaded test (rather than a test for multi threaded code).

You can get it at http://weblogs.asp.net/rosherove/archive/2007/06/22/multi-threaded-unit-tests-with-osherove-threadtester.aspx

B.8.1 Fitnesse

Fitnesse is a lightweight, open-source framework that makes it easy for software teams to define Acceptance Tests -- web pages containing simple tables of inputs and expected outputs, and then run those tests and see the results.

Fitnesse is quite buggy but has been in use in many places with varying degrees of success.

You can learn more about Fitnesse at www.fitnesse.org

B.8.2 StoryTeller

StoryTeller is a new tool for efficient creation and management of automated testing of .Net code with the NFit/FitNesse engine.StoryTeller is specifically created to support an Acceptance Test Driven Development strategy.All existing .Net FitNesse tests will run under StoryTeller.Features will include editing, tagging, and integration with source control, CruiseControl.Net, NAnt and/or MSBuild, and support for application versioning.

It is still under development. The project page is at http://storyteller.tigris.org/ but most of the real details are found on the author’s blog: http://codebetter.com/blogs/jeremy.miller/archive/tags/StoryTeller/default.aspx