About stubs, fakes and mocks

September 2, 2009

Test driven development is not an easy subject. And to be honest, I’ve not been convinced of it’s usefullness when first reading about it, witnessing this post. What I do believe firmelly is the usefulness of unit testing. But I have found that overall resistance to unit testing is rather big mostly because if not done well it turns out to be rather timeconsuming.

Now, let me start by saying I’m not an expert on the subject. I’ve tried applying it to some projects with some success and wanted to use it extensively in the next major release of my Offline Maps project. So I did some research to find where I could improve and my first subject was mock objects.

It turned that mocks are only a part of the complete picture.

Ok mocks, but what about stubs, dummies, fakes, etc…

There is a lot of lingo surrounding unit testing. Before we get any deeper on mock objects lets clearly define what they are and more importandly, what they are not and what differentiates them from stubs, fakes, etc…

Stubs

According to Jeremy Miller, the difference between mocks and stubs is mostly in the sort of testing you’re performing:

The real difference between a mock and a stub is in the style of unit testing, i.e. state-based testing versus interaction testing. A stub is a class that is hard-coded to return data from its methods and properties. You use stubs inside unit tests when you’re testing that a class or method derives the expected output for a known input.

Gerard Meszaros provides a bit more detail:

In many circumstances, the environment or context in which the system under test (SUT) operates very much influences the behavior of the SUT. To get good enough control over the indirect inputs of the SUT, we may have to replace some of the context with something we can control, a Test Stub.

So, you use stubs if you want to provide data to your system under test and the outcome of your system under test is dependend on the content of that data. A good example would be an object which has as input the current date and for example calculates yesterday. If you where to use the effective current date, then you could never write a test because each different day you execute the test would use a different current day, thus calculate a different yesterday and thus the value against which you want to check would change every day. If you however create a stub which provides a current day that you can set fixed, then you can reproduce the result whenever you want:

An example:

The code we want to test is the following:

public ITodayProvider TodayProvider { get; set; }
public DateTime Yesterday 
{
	get { return TodayProvider.Today.AddDays(-1); } 
}

The implementation of ITodayProvider used in the application uses the system date:

public DateTime Today
{
	get { return DateTime.Today; }
}

The implementation of ITodayProvider used in the test however allows to set the date that will be returned:

public DateTime Today
{
	get;
	set;
}

The test with the ITodayProvider using the system date does not produce repeatable results:

[Test]
public void WithoutStubTest()
{
	CalculateYesterday yesterdayCalculator = new CalculateYesterday();
	yesterdayCalculator.TodayProvider = new TodayProvider();

	// Unless you execute this test exactly on 1st september 2009, this test will fail
	Assert.AreEqual(new DateTime(2009, 8, 31), yesterdayCalculator.Yesterday);
}

The test with the ITodayProvider which can be set does allow to produce repeatable results:

[Test]
public void WithStubTest()
{
	CalculateYesterday yesterdayCalculator = new CalculateYesterday();
	TodayProviderStub stub = new TodayProviderStub();
	stub.Today = new DateTime(2009, 9, 1);
	yesterdayCalculator.TodayProvider = stub;

	// This test will always succeed, that is if you implementerd the yesterday calculation correctly.
	// It is however completely independent of the day the test is executed
	Assert.AreEqual(new DateTime(2009, 8, 31), yesterdayCalculator.Yesterday);
}

Fakes

Where a stub actually influences the system undertest, a fake only “fakes” an object in the system under test. Again according to Gerard Meszaros:

The SUT often depend on other components or systems. The interactions with these other components may be necessary but the side-effects of these interactions as implemented by the real depended-on component (DOC), may be unnecessary or even detrimental. A Fake Object is a much simpler and lighter weight implementation of the functionality provided by the DOC without the side effects we choose to do without.

Matin Fowler in his post on different kinds of test doubles states the following:

Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production (an InMemoryTestDatabase is a good example) So whereas stubs don’t have any working functionality, fakes have. They just take some shortcuts. Stubs are typically hardcoded to return some value.

As an example, suppose your object uses a webservice for retrieving some values, you may not be interested in setting up a connection to the webservice, provide a connection string, etc… because the actual webservice used is unimportant to your system under test. You also don’t want everyone executing your tests to be dependend on the availability of that webservice or detrimental behaviour like unavailable connections, timeouts and all that sort of stuff specific to webservices to mess up your test. In this case you could use a fake in webservice which provides the same functionality but is much easier in setup and which doesn’t exhibit the unwanted behaviour of unavailable connections and timeouts.

The code we want to test is the following:

public ITodayProvider TodayProvider { get; set; }
public IDayRangeCalculatorService DayRangeService { get; set; }
public DateTime YesterdayUsingDayRangeService
{
	get 
	{
		IDayRangeCalculator dayRangeCalculator = DayRangeService.ConnectToService();
		int daysBetween = dayRangeCalculator.DaysBetweenTodayAndYesterday();

		return TodayProvider.Today.AddDays(daysBetween); 
	}
}

A fake implementation of the service allows to get away of all problems relating to connecting to and using of webservices:

class DayRangeCalculatorServiceFake : IDayRangeCalculatorService
{
	#region IDayRangeCalculatorService Members

	public IDayRangeCalculator ConnectToService()
	{
		return new DayRangeCalculatorFake();
	}

	#endregion
}

class DayRangeCalculatorFake : IDayRangeCalculator
{
	#region IDayRangeCalculator Members

	public int DaysBetweenTodayAndYesterday()
	{
		return -1;
	}

	#endregion
}

By using the fake in the test we know our test will not fail because of problems related to the webservice. So if our test fails, then we know it’s because our algoritme failed:

[Test]
public void WithDaysBetweenServiceFakeTest()
{
	CalculateYesterday yesterdayCalculator = new CalculateYesterday();
	TodayProviderStub stub = new TodayProviderStub();
	stub.Today = new DateTime(2009, 9, 1);
	yesterdayCalculator.TodayProvider = stub;

	yesterdayCalculator.DayRangeService = new DayRangeCalculatorServiceFake();

	// In this test we are only interested if the method returns the correct yesterday 
	//  and not if the logging was done correctly
	Assert.AreEqual(new DateTime(2009, 8, 31), yesterdayCalculator.YesterdayWithLogging(new LoggingDummy()));
}

Dummies

Dummies have no influence at all on the tests. They are just used to provide values for method parameters, etc…

Fowler:

Dummy objects are passed around but never actually used. Usually they are just used to fill parameter lists.

Thus, you use a dummy if you test requires an object which thus not have any influence on the outcome of your test. An example could be a method called in your test which has some specific outcome you want to check but for example also take a logging object as a parameter. Remember we are unit testing and each test checks on some aspect of the object. So we want to check the specific outcome of calling the method but in this test we are not at all interested in the logging of the method. Then we provide a dummy logging object which actually thus nothing but allows us to call the method and check the expected outcome.

The code we want to test is the following:

public ITodayProvider TodayProvider { get; set; }

public DateTime YesterdayWithLogging(ILogging logging)
{
	logging.LogMessage("Executing YesterdayWithLogging");
	return TodayProvider.Today.AddDays(-1);
}

In our test we don’t care about logging. So we provide a dummy logging class which does nothing:

class LoggingDummy:  ILogging
{
	#region ILogging Members

	public void LogMessage(string message)
	{
	}

	#endregion
}

In our test we don’t have to worry about the logging

[Test]
public void WithLoggingDummyTest()
{
	CalculateYesterday yesterdayCalculator = new CalculateYesterday();
	TodayProviderStub stub = new TodayProviderStub();
	stub.Today = new DateTime(2009, 9, 1);
	yesterdayCalculator.TodayProvider = stub;

	// In this test we are only interested if the method returns the correct yesterday 
	//  and not if the logging was done correctly
	Assert.AreEqual(new DateTime(2009, 8, 31), yesterdayCalculator.YesterdayWithLogging(new LoggingDummy()));
}

Now, if all this is a bit overwhelming, don’t be concerned. Your test doubles will mostly be a mix of the above, or their focus or purpose will shift as shown by Daniel Cazzulino in his postMocks, Stubs and Fakes: it’s a continuum .

Mock objects, what problems to they solve ?

In many posts about the type of test doubles there is also the difference between test spies and mocks. From what I could find on them the difference is not that clear to me. So I will treat them as being the same. Of course, if you can explain the difference in a clear way, you’re welcome, or if you know a reference let me know.

About behaviour

We all know it but lets state it once more explicitely:
Objects are instances of classes which exhibit both state and behaviour.

Anyone studying UML will agree that both the class diagram and the sequence diagram are needed to document the interactions of systems. Where the class diagram shows what properties the classes in the system have, the sequence diagram shows what the important method calls made by the objects are. So you could say that the class diagram documents the state of a system (not entirely thru) and the sequence diagram documents the behaviour of the system. And as you need both the class diagram and the sequence diagram you need both state testing and behaviour testing.

When to use mock objects

The main difference with traditional testing is that where traditional testing allows a more state-based approach of testing, mock objects provide a behaviour style of testing. We all know the traditional style of unit testing using frameworks like NUnit, MbUnit and there assert-style testing:

  1. Setup your test
  2. Execute some action
  3. Assert that the properties of the object under test are what you expect

We assert that the state the object under test is in, is what we expect. To check this state, we check the values of the object’s properties.

Behaviour style testing on the other hand verifies the dynamic behaviour of the object under test. Remember that we are using mock objects to represent companion objects “on the edges of the object under test”. That is, objects used by our object under test. Now, sometimes we will want to know if our object under test calls a certain method with certain arguments of the companion object. The post Using Mock Objects Effectively provides a great example:

On closer inspection, it becomes clear that the application is not actually logging anything. Log4J is responsible for doing that. What the application is truly doing is sending a signal (a request) to log4j to log a message. That distinction is *very* important. The appropriate test then is simply to verify that the application is sending the correct signal to the service provider under the right conditions.

We now get following test sequence:

  1. Setup your test
  2. Execute some action
  3. Assert that the required methods have been executed with the correct parameters and in the correct sequence

We assert that the object under test executed the behaviour we expect. To check this behaviour, we check the methods it called and the arguments it provided to those methods.

So, using mock objects for behavioural testing can co-exist with the more traditional style of sate based testing.

Martin Fowler, him again, warns for some caveats when using mock style testing in his article Mocks Aren’t Stubs:

When you write a mockist test, you are testing the outbound calls of the SUT to ensure it talks properly to its suppliers…
Coupling to the implementation also interferes with refactoring, since implementation changes are much more likely to break tests than with classic testing.

An example

The code we want to test is the following:

public ITodayProvider TodayProvider { get; set; }

public DateTime YesterdayWithLogging(ILogging logging)
{
	logging.LogMessage("Executing YesterdayWithLogging");
	return TodayProvider.Today.AddDays(-1);
}

In our test we don’t care about the exact output of the logging but we do care if the logging is called and if it is called with the correct arguments. So we want to record the methodcall and the arguments with which it got called:

class LoggingMock:  ILogging
{
	public int NumberOfMethodCalls { get; private set; }
	public List<String> MethodCallArguments { get; private set; }

	#region ILogging Members

	public void LogMessage(string message)
	{
		NumberOfMethodCalls++;
		if (MethodCallArguments == null)
		{
			MethodCallArguments = new List<string>();
		}

		MethodCallArguments.Add(message);
	}

	#endregion
}

By recording the calls we can check in our test if the method got called and if the provided arguments where correct:

[Test]
public void WithLoggingMockTest()
{
	CalculateYesterday yesterdayCalculator = new CalculateYesterday();
	TodayProviderStub stub = new TodayProviderStub();
	stub.Today = new DateTime(2009, 9, 1);
	yesterdayCalculator.TodayProvider = stub;
	LoggingMock loggingMock = new LoggingMock();

	// In this test we want to see if the logging is called correctly
	DateTime yesterday = yesterdayCalculator.YesterdayWithLogging(loggingMock);

	Assert.AreEqual(1, loggingMock.NumberOfMethodCalls);
	Assert.AreEqual("Executing YesterdayWithLogging", loggingMock.MethodCallArguments[0]);
}

Sample Code

The Code

Links

Mock Objects and Stubs: The Bottle Brush of TDD
Why and When to Use Mock Objects
Best and Worst Practices for Mock Objects
xUnit Patterns
Using Mock Objects Effectively
MF Bliki: TestDouble
Mocks Aren’t Stubs
Mock object

I’ve just started a new project on Codeplex: DxFluentUI.

The goal of the project is to provide a fluent API for the creation of DevExpress controls. It currently provides an incomplete implementation for the XtraGrid and XtraVerticalGrid controls.

Look back for further details….

links for this week:

unit testing windows forms applications and more perticular: databinding
While working on some code which encapsulates windows forms controls, I had to check if my abstraction works. And especially, if the databinding works:
Unit Testing WinForms 2.0 Databinding

Adding features to Visual Studio 2008
It had happened to me before but this time I had to succeed: I needed to install the crystal report features which are part of visual studio. Unfortunately, starting the setup form the dvd failed with the error: “A problem has been encountered while loading the setup components. Canceling setup.” After some research following post helped me (make sure you read it until the end):
Adding features to Visual Studio 2008 may fail to load setup components

Links for this week:

Global namespaces
I wanted to name a namespace “CompanyName.System” which turned out to be a bad idea: it gave me the unfortunate compiler error “The type or namespace name ‘SomeClassName’ could not be found”. The solution is easy according to the MSDN article How to: Use the Namespace Alias Qualifier (C# Programming Guide) but unfortunately the error happened in the designer file of a Windows.Form and there is no point in changing any code there. I suppose Microsoft should use the global:: identifier in generated code but they don’t. Or is there a solution I’m overlooking?

WCF Service hosting and Client access in a Windows Forms application
Actually, I came across this one too late. Last week I made a test application which was I simple Windows Forms application whome hosted a WCF service and connected to that same WCF service. A big NO GO was the result, as this guy also experienced…

WCF Services with constructor parameters
I eventually didn’t use it, but I had a use case this week for having a WCF service in which the constructor would have some parameters. An alternative could be to provide the object instance to use for implementing the service yourself instead of letting WCF instantiate it for you as the standard behaviour is. Again, I’m not the first one willing to do this as these links prove:
Unity & friends: The WCF service host side
WCF Dependency Injection Behavior
WCF Service Dependency Injection

Generics in .Net
I’m still getting used to this. Comming from a C++ background I’m still suprised by compiler errors trying to use a template parameter as a base class, or as an attribute parameter. In C# this is not allowed. Why? Read these posts:
Generic argument as a base class
Generic argument as a base class
Using Generic Parameter as an Attribute constructor parameter

Porting my Offline Maps application to the .NET Compact Framework, I was in for a big supprise:

I thought it would be simple: I intentionally seperated most of the GUI code from the Business code thinking that I would simply need to port the GUI and everything would be OK.

No sir, that didn’t work…

Problems encountered

The first supprise was that in Visual Studio 2005 you can not set a reference in a Device Application project to a normal .NET library. You must specify a special Class Library project specific to Windows Mobile (I’ ve been using the Windows Mobile 6 Professional SDK).

Secondly, not being used to programming for the .NET Compact Framework, I thought that having explicitly used mainstream classes, a simple recompile for the Compact Framework would suffise. However that didn’t work: apparently you can have equally named classes, but they don’t necessarily have the same methods. The Compact Framework mostly just supports a subset of the full Framework class methods.

Solutions

Looking for solutions I came across this article.
It offers a multitude of solutions from which I mostly used the “one folder, two projects” and “use partial classes”.

Another point to keep in mind is that if you have overloaded methods in a class with versions taking objects and versions taking the bare properties of those objects, sometimes the Compact Framework only supports those taking bare properties.

An example of this last case is a method which takes a rectangle as an argument opposed to a method taking an X and Y coordinate for the position and a Width and Height for the size of the rectangle.

Also, if some classes or methods aren’t supported that can lead to a whole area of functionality not being supported. An example of this are the StartCap, EndCap and LineJoin properties of a pen which are not supported in the Compact Framework. They can require you to implement a complete abstraction layer for your functionality. Fortunately I got a way with a partial class with two different implementations of a “CreateRoundedPen” method.

Something which surprised me the most is that some Environment static properties which after all are meant to enable cross platform development aren’t supported neither.

As an example the Compact Framework does not support the Environment.NewLine property. Weel, at least not in the version I used. I noticed in the Compact Framework 3.5 documentation that it is now supported.

Conclusion

So, cross device development from a single code base does take some preparation. It requires you to think about how to setup your projects folder structure and to read the documentation on classes and methods so to choose the method supported in both platforms.

But what experience, and derived from it tips, do you have?

Links

[1] Share Code: Write Code Once For Both Mobile And Desktop Apps
[2] MSDN documentation on the Environment.NewLine property

Book data

  • Title: Essential .NET, Volume 1: The Common Language Runtime
  • Author: Don Box
  • ISBN: 0-201-73411-7

Review

The book starts of with discussing the evolution of component technologies from Microsoft and how .NET was conceived to solve the problems with COM. After this short introductory chapter, the author continues with discussing .NET.

Read the rest of this entry »

TheServerside.net has an interesting post about software factories.

Apparently someone is afraid that automation will lead to “dumb-ification”:

While that’s a great theory, what ends up happening is developers get plugged into an assembly line and are treated like battery chickens expected to squeeze our code. In our increasingly diversified platform and architecture world the software factory is as outdated as Henry Ford’s Model T assembly line, where you could have any color as long as it was black.

And then there is this other guy who believes it is an opportunity:

It explains how, by providing a simple but effective architectural framework for organizing a project, called a schema, Software Factories help their developers define better metrics, and how, by providing a framework for harvesting and reusing experience, Software Factories help their developers use metrics more effectively to improve future projects.

I’m with the last one. Software factories are an opportunity.

I can’t believe anyone still thinks we want to treat people as mindless tools. This is 2006, not 1906. And even if anyone would like to, people will not let him.

The Practical Traveller

July 28, 2006

The holidays are back again, and everyone is preparing for their holiday trip. Our European roads are stuffed with cars filled with people and their luggage. They all share a common goal:

“Get there co-travellers as fast and comfortable possible to their destination”

There are of course different ways for drivers to prepare for the trip.
Read the rest of this entry »

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: