Thank you for joining the easy unit testing family!

Register your software now to get a FREE trial!

Full Name* :
Company E-Mail* :
Phone Number :

Isolation Framework evaluation checklist

This checklist was created to be used by developers and technical team leaders who may be evaluating different isolation frameworks for use in their commercial software project(s).
1.

How easy is it for your team to get started with the framework?

It may be worthwhile to time a new developer within your team with the framework to get an accurate idea of how long it takes to get started. For example – how long will it take them to write the first three tests for some class in your system? Is the API clear and simple? Is there a single point of entry in the API? Is there clear guidance on what to do at each step of the way? How often, if at all, do you need to check the docs and tutorials? How easy is it to look for the next step when you're not sure what to do? Some tools offer guidance within the IDE, while some provide extensive help. Some don't do either.

2.

How resistant to change do you need your tests to be?

Different frameworks provide varying levels of change resistance (when production code changes). Change resistance can be measured according to the number of tests needed to modify for a single piece (method) of changed production code. Does the tool support these production changes without affecting the tests?

Do the frameworks support recursive fakes? Is it non-strict by default or does it throw exceptions on unexpected interactions by default? How does it handle method overloads (if your production code changes to call a different overload?). How are arguments verified? Are all arguments ignored? Are actual values used for expectations?

All these things affect the fragility of your test. The more fragile it is, the less change resistant it is. If your production code changes a lot you will need to take this into account.

3.

How effective will the tool be for your specific project?

Your project might be a greenfield project (fresh new code) or have a lot of legacy code. not all isolation frameworks were designed, or have the same support for all (or any) legacy code scenarios. If your unit tests need to be written against legacy code (exisiting code without tests):

See if your production code contains static constructors, internal or private classes that might need to be faked. Does the framework support faking them? Does your code instantiate objects directly all over the place, or does it use a dependency injection framework or factory of some kind that would need to be faked? Make sure the framework you choose supports these scenarios, or that you have the time needed to refactor your code for testing such cases.

4.

How do your tests look when you use the framework?

Test code is still code. It needs to be readable, and when you jump into it to debug, you need to be able to see at a glance what you’re testing.

Are your tests messy? Does the framework cause your tests to be longer than a page? Can you understand the expected behavior of the code under test if the framework is involved? Can your team members understand without a great deal of explanation what you are using the framework for in each test? Can you still write test code according to accepted industry best practices such as AAA (arrange-act-assert) when using this framework? Not all frameworks support the easier-to-understand AAA tests, for example.

5.

How well is the framework integrated with your coding environment and ecosystem?

Can you use the framework from your various test runners (TestDriven.NET, Resharper, MS Test etc.) or in combination with profiling and code coverage tools?

Some frameworks incorporate profiling, and it is important to see that they work effectively with other profilers and runners. Can you run your tests with code coverage? Or with other profiling technologies?

How well does the framework integrate into various versions of Visual Studio? Do you need it to support VS 2005? VS 2008? What .NET versions should it support? Some of the frameworks require .NET 3.5 and up.

6.

Do you need the ability to verify interactions between objects in your tests?

Some frameworks may not have the built in ability to verify interactions, requiring you to provide your own manual flagging mechanisms (hand rolled mocks). For complicated interfaces this can grow cumbersome and result in an un-maintainable piece of code.

Can you verify method calls between the code under test and its 3rd party dependencies? For example, that a SharePoint method gets called at the end of a test.

7.

What kind of technical support do you require?

How fast do you need to get a response to a question if you are not sure how to accomplish a specific task with the tool?

How many people are on staff at the vendor who makes the framework? And at what hours of the day?

Is support provided by a company or an independent individual? A company is often more invested in supporting you than an individual – or they stand to lose money.

What is the company's main expertise? Are unit testing and agility at its core business? If so, the support team is more likely to well versed be on these subjects.

8.

What is your total cost of ownership for the tool?

The total cost should consider price, time and effort required to work and implement the tool in your project or organization. Compare the following:
a. Price of tool
b. Time to get started and reach RTM test (readable, trustworthy, maintainable)
c. Time to write tests
d. Time to fix test because of production change (change resistance)

Writing and running your first unit test in C#
 You can find the code used in this tutorial in Test0_Tutorial.cs, in the Isolator installation folder (usually C:\Program Files\Typemock\Isolator\VERSION), under the Typemock.Example.CSharp solution.

Step 1 - Create a new Test Project


This step shows you how to create a new test project in Visual Studio, and add the Typemock references you need.
  • Open Visual Studio, click ‘File’ and create a new Test Project.

    new Test Project
  • Right click ‘Project Properties’, click ‘Add References’ and add ‘Typemock Isolator core DLL’ and ‘Typemock Isolator C# API’

    For this example test, you’ll need a reference to System.Windows.Forms as well.

Step 2 - Write unit tests


This section will walk you through the three stages in unit tests that use the C# API to isolate.
    1. Arrange – Set up your faked object’s behaviour
    2. Act - Call your test code
    3. Assert - Make sure that the tests were called

    4. In the new test project we opened in Visual Studio, go to UnitTest1.cs in the ‘Solutions’ column and add the following test code to the beginning of the file:

using System.Windows.Forms;
using TypeMock.ArrangeActAssert;

Add the Isolated attribute to the test class:
[TestClass, Isolated]
public class Test0_Tutorial

Example Test 1 - Simple test using MessageBox

This example shows how to ignore a method call, and verify it was called during our test. We’ll use the following APIs in this test:
  • Isolate.WhenCalled - The method inside the WhenCalled clause will be ignored, and will return a fake value
  • Isolate.Verify.WasCalledWithExactArguments - Verifies that the method inside the WhenCalled clause was called during the test with the exact arguments

The following takes place in the code:


[TestMethod]
public void SimpleTestUsingMessageBox()
{

 // Arrange

1  Isolate.WhenCalled(()=> MessageBox.Show(String .Empty)).WillReturn(DialogResult.OK);

 // Act

2 MessageBox.Show

("This is a message");

 // Assert

3 Isolate

.Verify.WasCalledWithExactArguments(()=>MessageBox.Show("This is a message"));
}

  • Set all calls to MessageBox.Show to be ignored, and return a fake value.
  • Call MessageBox.Show - notice that when you run the test, the MessageBox doesn't actually appear.
  • Check that the static function ‘Show’ was called during the test with the parameter "This is a message".

Example Test 2 - Complex Test

This example shows how to test the interaction between two classes:
  • SomeClass - Has a shared sub called MyMethod
  • UserOfSomeClass - Has a sub called DoSomthing

 


public class  SomeClass
{
 public static void  MyMethod()
 {
 // do work
 }
}
public class  UserOfSomeClass
{
 public void  DoSomeThing()
 {
  try
  {
   SomeClass.MyMethod();
  }
  catch(Exception exc)
  {
   MessageBox.Show("Exception caught: " & exc.Message);
  }
 }
}


    This test checks that the user sees every exception thrown from SomeClass using a MessageBox.

    The following APIs are used in the test:

    We’ll use the following APIs in this test:

  • Isolate.WhenCalled(..).WillThrow - The method inside the WhenCalled clause will throw a specified exception when called
  • Isolate.WhenCalled(..).WillReturn - The method inside the WhenCalled will be ignored, and return a fake value
  • Isolate.Verify.WasCalledWithExactArguments - Verifies that the method inside the WhenCalled clause was called with the exact arguments during the text execution

  [TestMethod]
 public void  ComplexTest()
 {
 // Arrange
 1aIsolate.WhenCalled(()=> SomeClass.MyMethod()).WillThrow( new Exception ("foo"));

 2aIsolateWhenCalled(()=> MessageBox.Show(String.Empty)).WillReturn(DialogResult.Cancel);

 // Act


 2UserOfSomeClass user = new UserOfSomeClass();user.DoSomthing();

 3Isolate .Verify.WasCalledWithExactArguments(()=> MessageBox .Show("Exception caught: foo"));

 }


    The following takes place in the code:

  • The actual test code - Create a new instance of UserOfSomeClass and call the DoSomething method.

  • Check that MessageBox.Show was called during our test with ‘Exception caught: foo’.

  • Here is the full source code we used in our examples:

using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.Windows.Forms;
using TypeMock.ArrangeActAssert;


[TestClass, Isolated]
public class Test0_Tutorial

{

 [TestMethod]
 public void SimpleTestUsingMessageBox()
 {
  // Arrange
  Isolate.WhenCalled(()=> MessageBox.Show(String.Empty)).WillReturn(DialogResult.OK);

  // Act
  MessageBox.Show("This is a message");

  // Assert
  Isolate.Verify.WasCalledWithExactArguments(()=> MessageBox.Show("This is a message"));
 }

 [TestMethod]
 public void ComplexTest()
 {
  // Arrange
  Isolate.WhenCalled(()=> SomeClass.MyMethod()).WillThrow( new Exception ("foo"));
  Isolate.WhenCalled(()=> MessageBox.Show(String.Empty)).WillReturn( DialogResult.Cancel);

  // Act
  UserOfSomeClass user = new UserOfSomeClass ();
  user.DoSomthing();

  // Assert
  Isolate.Verify.WasCalledWithExactArguments(()=> MessageBox.Show("Exception caught:
  foo"));
 }


}


public class SomeClass
{
 public static void MyMethod()
 {
  // do work
 }
}

public class UserOfSomeClass
{
 public void DoSomething()
 {
  try
  {
   SomeClass.MyMethod();
  }
  catch ( Exception exc)
  {
   MessageBox .Show("Exception caught: " & exc.Message);
  }
 }
}