Test driven development is not an easy subject. And to be honest, I’ve not been convinced of it’s usefullness when first reading about it, witnessing this post. What I do believe firmelly is the usefulness of unit testing. But I have found that overall resistance to unit testing is rather big mostly because if not done well it turns out to be rather timeconsuming.
Now, let me start by saying I’m not an expert on the subject. I’ve tried applying it to some projects with some success and wanted to use it extensively in the next major release of my Offline Maps project. So I did some research to find where I could improve and my first subject was mock objects.
It turned that mocks are only a part of the complete picture.
Ok mocks, but what about stubs, dummies, fakes, etc…
There is a lot of lingo surrounding unit testing. Before we get any deeper on mock objects lets clearly define what they are and more importandly, what they are not and what differentiates them from stubs, fakes, etc…
Stubs
According to Jeremy Miller, the difference between mocks and stubs is mostly in the sort of testing you’re performing:
The real difference between a mock and a stub is in the style of unit testing, i.e. state-based testing versus interaction testing. A stub is a class that is hard-coded to return data from its methods and properties. You use stubs inside unit tests when you’re testing that a class or method derives the expected output for a known input.
Gerard Meszaros provides a bit more detail:
In many circumstances, the environment or context in which the system under test (SUT) operates very much influences the behavior of the SUT. To get good enough control over the indirect inputs of the SUT, we may have to replace some of the context with something we can control, a Test Stub.
So, you use stubs if you want to provide data to your system under test and the outcome of your system under test is dependend on the content of that data. A good example would be an object which has as input the current date and for example calculates yesterday. If you where to use the effective current date, then you could never write a test because each different day you execute the test would use a different current day, thus calculate a different yesterday and thus the value against which you want to check would change every day. If you however create a stub which provides a current day that you can set fixed, then you can reproduce the result whenever you want:
An example:
The code we want to test is the following:
public ITodayProvider TodayProvider { get; set; }
public DateTime Yesterday
{
get { return TodayProvider.Today.AddDays(-1); }
}
The implementation of ITodayProvider used in the application uses the system date:
public DateTime Today
{
get { return DateTime.Today; }
}
The implementation of ITodayProvider used in the test however allows to set the date that will be returned:
public DateTime Today
{
get;
set;
}
The test with the ITodayProvider using the system date does not produce repeatable results:
[Test]
public void WithoutStubTest()
{
CalculateYesterday yesterdayCalculator = new CalculateYesterday();
yesterdayCalculator.TodayProvider = new TodayProvider();
// Unless you execute this test exactly on 1st september 2009, this test will fail
Assert.AreEqual(new DateTime(2009, 8, 31), yesterdayCalculator.Yesterday);
}
The test with the ITodayProvider which can be set does allow to produce repeatable results:
[Test]
public void WithStubTest()
{
CalculateYesterday yesterdayCalculator = new CalculateYesterday();
TodayProviderStub stub = new TodayProviderStub();
stub.Today = new DateTime(2009, 9, 1);
yesterdayCalculator.TodayProvider = stub;
// This test will always succeed, that is if you implementerd the yesterday calculation correctly.
// It is however completely independent of the day the test is executed
Assert.AreEqual(new DateTime(2009, 8, 31), yesterdayCalculator.Yesterday);
}
Fakes
Where a stub actually influences the system undertest, a fake only “fakes” an object in the system under test. Again according to Gerard Meszaros:
The SUT often depend on other components or systems. The interactions with these other components may be necessary but the side-effects of these interactions as implemented by the real depended-on component (DOC), may be unnecessary or even detrimental. A Fake Object is a much simpler and lighter weight implementation of the functionality provided by the DOC without the side effects we choose to do without.
Matin Fowler in his post on different kinds of test doubles states the following:
Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production (an InMemoryTestDatabase is a good example) So whereas stubs don’t have any working functionality, fakes have. They just take some shortcuts. Stubs are typically hardcoded to return some value.
As an example, suppose your object uses a webservice for retrieving some values, you may not be interested in setting up a connection to the webservice, provide a connection string, etc… because the actual webservice used is unimportant to your system under test. You also don’t want everyone executing your tests to be dependend on the availability of that webservice or detrimental behaviour like unavailable connections, timeouts and all that sort of stuff specific to webservices to mess up your test. In this case you could use a fake in webservice which provides the same functionality but is much easier in setup and which doesn’t exhibit the unwanted behaviour of unavailable connections and timeouts.
The code we want to test is the following:
public ITodayProvider TodayProvider { get; set; }
public IDayRangeCalculatorService DayRangeService { get; set; }
public DateTime YesterdayUsingDayRangeService
{
get
{
IDayRangeCalculator dayRangeCalculator = DayRangeService.ConnectToService();
int daysBetween = dayRangeCalculator.DaysBetweenTodayAndYesterday();
return TodayProvider.Today.AddDays(daysBetween);
}
}
A fake implementation of the service allows to get away of all problems relating to connecting to and using of webservices:
class DayRangeCalculatorServiceFake : IDayRangeCalculatorService
{
#region IDayRangeCalculatorService Members
public IDayRangeCalculator ConnectToService()
{
return new DayRangeCalculatorFake();
}
#endregion
}
class DayRangeCalculatorFake : IDayRangeCalculator
{
#region IDayRangeCalculator Members
public int DaysBetweenTodayAndYesterday()
{
return -1;
}
#endregion
}
By using the fake in the test we know our test will not fail because of problems related to the webservice. So if our test fails, then we know it’s because our algoritme failed:
[Test]
public void WithDaysBetweenServiceFakeTest()
{
CalculateYesterday yesterdayCalculator = new CalculateYesterday();
TodayProviderStub stub = new TodayProviderStub();
stub.Today = new DateTime(2009, 9, 1);
yesterdayCalculator.TodayProvider = stub;
yesterdayCalculator.DayRangeService = new DayRangeCalculatorServiceFake();
// In this test we are only interested if the method returns the correct yesterday
// and not if the logging was done correctly
Assert.AreEqual(new DateTime(2009, 8, 31), yesterdayCalculator.YesterdayWithLogging(new LoggingDummy()));
}
Dummies
Dummies have no influence at all on the tests. They are just used to provide values for method parameters, etc…
Dummy objects are passed around but never actually used. Usually they are just used to fill parameter lists.
Thus, you use a dummy if you test requires an object which thus not have any influence on the outcome of your test. An example could be a method called in your test which has some specific outcome you want to check but for example also take a logging object as a parameter. Remember we are unit testing and each test checks on some aspect of the object. So we want to check the specific outcome of calling the method but in this test we are not at all interested in the logging of the method. Then we provide a dummy logging object which actually thus nothing but allows us to call the method and check the expected outcome.
The code we want to test is the following:
public ITodayProvider TodayProvider { get; set; }
public DateTime YesterdayWithLogging(ILogging logging)
{
logging.LogMessage("Executing YesterdayWithLogging");
return TodayProvider.Today.AddDays(-1);
}
In our test we don’t care about logging. So we provide a dummy logging class which does nothing:
class LoggingDummy: ILogging
{
#region ILogging Members
public void LogMessage(string message)
{
}
#endregion
}
In our test we don’t have to worry about the logging
[Test]
public void WithLoggingDummyTest()
{
CalculateYesterday yesterdayCalculator = new CalculateYesterday();
TodayProviderStub stub = new TodayProviderStub();
stub.Today = new DateTime(2009, 9, 1);
yesterdayCalculator.TodayProvider = stub;
// In this test we are only interested if the method returns the correct yesterday
// and not if the logging was done correctly
Assert.AreEqual(new DateTime(2009, 8, 31), yesterdayCalculator.YesterdayWithLogging(new LoggingDummy()));
}
Now, if all this is a bit overwhelming, don’t be concerned. Your test doubles will mostly be a mix of the above, or their focus or purpose will shift as shown by Daniel Cazzulino in his postMocks, Stubs and Fakes: it’s a continuum .
Mock objects, what problems to they solve ?
In many posts about the type of test doubles there is also the difference between test spies and mocks. From what I could find on them the difference is not that clear to me. So I will treat them as being the same. Of course, if you can explain the difference in a clear way, you’re welcome, or if you know a reference let me know.
About behaviour
We all know it but lets state it once more explicitely:
Objects are instances of classes which exhibit both state and behaviour.
Anyone studying UML will agree that both the class diagram and the sequence diagram are needed to document the interactions of systems. Where the class diagram shows what properties the classes in the system have, the sequence diagram shows what the important method calls made by the objects are. So you could say that the class diagram documents the state of a system (not entirely thru) and the sequence diagram documents the behaviour of the system. And as you need both the class diagram and the sequence diagram you need both state testing and behaviour testing.
When to use mock objects
The main difference with traditional testing is that where traditional testing allows a more state-based approach of testing, mock objects provide a behaviour style of testing. We all know the traditional style of unit testing using frameworks like NUnit, MbUnit and there assert-style testing:
- Setup your test
- Execute some action
- Assert that the properties of the object under test are what you expect
We assert that the state the object under test is in, is what we expect. To check this state, we check the values of the object’s properties.
Behaviour style testing on the other hand verifies the dynamic behaviour of the object under test. Remember that we are using mock objects to represent companion objects “on the edges of the object under test”. That is, objects used by our object under test. Now, sometimes we will want to know if our object under test calls a certain method with certain arguments of the companion object. The post Using Mock Objects Effectively provides a great example:
On closer inspection, it becomes clear that the application is not actually logging anything. Log4J is responsible for doing that. What the application is truly doing is sending a signal (a request) to log4j to log a message. That distinction is *very* important. The appropriate test then is simply to verify that the application is sending the correct signal to the service provider under the right conditions.
We now get following test sequence:
- Setup your test
- Execute some action
- Assert that the required methods have been executed with the correct parameters and in the correct sequence
We assert that the object under test executed the behaviour we expect. To check this behaviour, we check the methods it called and the arguments it provided to those methods.
So, using mock objects for behavioural testing can co-exist with the more traditional style of sate based testing.
Martin Fowler, him again, warns for some caveats when using mock style testing in his article Mocks Aren’t Stubs:
When you write a mockist test, you are testing the outbound calls of the SUT to ensure it talks properly to its suppliers…
Coupling to the implementation also interferes with refactoring, since implementation changes are much more likely to break tests than with classic testing.
An example
The code we want to test is the following:
public ITodayProvider TodayProvider { get; set; }
public DateTime YesterdayWithLogging(ILogging logging)
{
logging.LogMessage("Executing YesterdayWithLogging");
return TodayProvider.Today.AddDays(-1);
}
In our test we don’t care about the exact output of the logging but we do care if the logging is called and if it is called with the correct arguments. So we want to record the methodcall and the arguments with which it got called:
class LoggingMock: ILogging
{
public int NumberOfMethodCalls { get; private set; }
public List<String> MethodCallArguments { get; private set; }
#region ILogging Members
public void LogMessage(string message)
{
NumberOfMethodCalls++;
if (MethodCallArguments == null)
{
MethodCallArguments = new List<string>();
}
MethodCallArguments.Add(message);
}
#endregion
}
By recording the calls we can check in our test if the method got called and if the provided arguments where correct:
[Test]
public void WithLoggingMockTest()
{
CalculateYesterday yesterdayCalculator = new CalculateYesterday();
TodayProviderStub stub = new TodayProviderStub();
stub.Today = new DateTime(2009, 9, 1);
yesterdayCalculator.TodayProvider = stub;
LoggingMock loggingMock = new LoggingMock();
// In this test we want to see if the logging is called correctly
DateTime yesterday = yesterdayCalculator.YesterdayWithLogging(loggingMock);
Assert.AreEqual(1, loggingMock.NumberOfMethodCalls);
Assert.AreEqual("Executing YesterdayWithLogging", loggingMock.MethodCallArguments[0]);
}
Sample Code
Links
Mock Objects and Stubs: The Bottle Brush of TDD
Why and When to Use Mock Objects
Best and Worst Practices for Mock Objects
xUnit Patterns
Using Mock Objects Effectively
MF Bliki: TestDouble
Mocks Aren’t Stubs
Mock object
Serge Desmedt 