Jul
17
2010

Unit Test using test doubles, aka Mock, Stub, Fake, Dummy

Following my post about Application Acceptance Testing, we went, beginning of that week, in some very interesting discussions during a meeting at Innoveo in which I presented the differences between Mock and Stub in Unit Test. After the meeting as I often do I gather up from the web some posts which expressed in more details what I was talking about and made some extract of the posts.

This time I found some interesting content follow-up on my explanations on the site of Rhino.Mock framework website and a post from Martin Fowler; Mock’s aren’t Stubs

Understanding the difference between a Stub and a Mock is an important distinction to do, because they have each one purpose which isn’t the same.

Some framework like the excellent mockito decided not to make the distinction in the way you create the two, some other framework like the also excellent Rhino.Mock does this distinction. At the end what is important is that the developer understand the difference.

Extract from Rhino.Mock website

“In short term, mock are for behavior verification, and stub are for state verification.

In longer term, a mock is an object that we can set expectations on, and which will verify that the expected actions have indeed occurred. A stub is an object that you use in order to pass to the code under test. You can setup expectations on it, so it would act in certain ways, but those expectations will never be verified.

In general, it is recommended to follow the principle of "Test only one thing per test". So each unit test should validate no more than one significant interaction with another object. This generally implies that a given test should have no more than one mock object, but it may have several stubs, as needed. (An exception to this would be if the "one thing" tested inherently requires expectations on two dependent objects, for example, verifying that a method on one object is only called after a method on another object has been called.)

If you want to verify the behavior of the code under test, you will use a mock with the appropriate expectation, and verify that. If you want just to pass a value that may need to act in a certain way, but isn't the focus of this test, you will use a stub.

IMPORTANT: A stub will never cause a test to fail.”

Extract from Martin Fowler post

“This difference is actually two separate differences. On the one hand there is a difference in how test results are verified: a distinction between state verification and behavior verification. On the other hand is a whole different philosophy to the way testing and design play together, which I term here as the classical and mockist styles of Test Driven Development.”

Fowler in his post use an older way (post from 2007) of expressing Arrange-Act-Assert, mixed with Fixture terms: setup, exercise, verify, teardown.

Today’s the expressions Arrange-Act-Assert or Record-Replay (which he is talking later on in the post) are common.

Stub

“This style of testing uses state verification: which means that we determine whether the exercised method worked correctly by examining the state of the SUT and its collaborators after the method was exercised. As we'll see, mock objects enable a different approach to verification.”

Mock

“The SUT is the same - an order. However the collaborator isn't a warehouse object, instead it's a mock warehouse - technically an instance of the class Mock.

The second part of the setup creates expectations on the mock object.The expectations indicate which methods should be called on the mocks when the SUT is exercised.

Once all the expectations are in place I exercise the SUT. After the exercise I then do verification, which has two aspects. I run asserts against the SUT - much as before. However I also verify the mocks - checking that they were called according to their expectations.

The key difference here is how we verify that the order did the right thing in its interaction with the warehouse. With state verification we do this by asserts against the warehouse's state. Mocks use behavior verification, where we instead check to see if the order made the correct calls on the warehouse. We do this check by telling the mock what to expect during setup and asking the mock to verify itself during verification. Only the order is checked using asserts, and if the method doesn't change the state of the order there's no asserts at all.”

“When you're doing testing like this, you're focusing on one element of the software at a time -hence the common term unit testing. The problem is that to make a single unit work, you often need other units - hence the need for some kind of warehouse in our example.”

Mock/Stub are also called test doubles inspired from stunt double in movies! I think the analogy is quite good.

“Meszaros then defined four particular kinds of double:

  • Dummy objects are passed around but never actually used. Usually they are just used to fill parameter lists.
  • Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production (an in memory database is a good example).
  • Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test. Stubs may also record information about calls, such as an email gateway stub that remembers the messages it 'sent', or maybe only how many messages it 'sent'.
  • Mocks are what we are talking about here: objects pre-programmed with expectations which form a specification of the calls they are expected to receive.

Of these kinds of doubles, only mocks insist upon behavior verification. The other doubles can, and usually do, use state verification. Mocks actually do behave like other doubles during the exercise phase, as they need to make the SUT believe it's talking with its real collaborators - but mocks differ in the setup and the verification phases.”

Mock objects always use behavior verification, a stub can go either way. Meszaros refers to stubs that use behavior verification as a Test Spy. The difference is in how exactly the double runs and verifies and I'll leave that for you to explore on your own.”

Test Driven Development

The classical TDD style is to use real objects if possible and a double if it's awkward to use the real thing. So a classical TDDer would use a real warehouse and a double for the mail service. The kind of double doesn't really matter that much.

A mockist TDD practitioner, however, will always use a mock for any object with interesting behavior. In this case for both the warehouse and the mail service.

”An important offshoot of the mockist style is that of Behavior Driven Development (BDD). BDD was originally developed by my colleague Dan North as a technique to better help people learn Test Driven Development by focusing on how TDD operates as a design technique. This led to renaming tests as behaviors to better explore where TDD helps with thinking about what an object needs to do. BDD takes a mockist approach, but it expands on this, both with its naming styles, and with its desire to integrate analysis within its technique. I won't go into this more here, as the only relevance to this article is that BDD is another variation on TDD that tends to use mockist testing. I'll leave it to you to follow the link for more information.”

“Occasionally you do run into things that are really hard to use state verification on, even if they aren't awkward collaborations. A great example of this is a cache. The whole point of a cache is that you can't tell from its state whether the cache hit or missed - this is a case where behavior verification would be the wise choice for even a hard core classical TDDer. I'm sure there are other exceptions in both directions.”

“It's at this point that I should stress that whichever style of test you use, you must combine it with coarser grained acceptance tests that operate across the system as a whole. I've often come across projects which were late in using acceptance tests and regretted it.”

TDD's origins were a desire to get strong automatic regression testing that supported evolutionary design. Along the way its practitioners discovered that writing tests first made a significant improvement to the design process.

“As I've learned from Test Driven Development itself, it's often hard to judge a technique without trying it seriously.”

Jun
11
2010

From WPF functional Unit Tests to Specifications using MSpec and White

I am in the train back home and wanted to try out quickly to migrate our WPF functional tests written has Unit Tests to BDD Specifications.

Here is the code I started from, pure Unit Test using NUnit and White

[Test]
public void Opening_Valid_VersionZip()
{
    OpenAndWait("Product.zip");

    Assert.That(MainWindow.Title.Equals("Product.zip - Innoveo Skye® Editor"));
    Assert.That(Status.Text.Equals("product"));
    Assert.That(ProductTree.Nodes.Count >= 1);
    Assert.IsFalse(SplashScreen.Visible);
    Assert.IsTrue(SaveButton.Enabled);
    Assert.IsTrue(ActivateButton.Enabled);
}

Now the same functional test written as a BDD specification using MSpec

[Subject("OpenVersionZip")]
public class when_user_open_valid_versionzip : MainWindowViewSpecs
{
    Establish context = () => {};

    Because of = () => OpenAndWait("Product.zip");

    It should_display_mainwindow_title_correctly = () =>
        MainWindow.Title.ShouldEqual("Product.zip - Innoveo Skye® Editor");

    It should_display_status_correctly = () =>
        Status.Text.ShouldEqual("product");

    It should_display_the_product_tree = () =>
        ProductTree.Nodes.ShouldNotBeNull();

    It should_hide_the_splashscreen = () =>
        SplashScreen.Visible.ShouldBeFalse();

    It should_enable_save_button = () =>
        SaveButton.Enabled.ShouldBeTrue();

    It should_enable_activate_button = () =>
        ActivateButton.Enabled.ShouldBeTrue();
}

And the output in ReSharper MSpec plugin

4687909611_49a4e5a71a_o[1]

Which one do you prefer? I personally have made my choice.

Jan
28
2010

White’s tip for your automated WPF functional tests

When you build automated WPF functional test using White in which you need to open a file through a Windows open file dialog, you will be confronted with the following issue. Windows open file dialog remember the last path with which you opened a file.

So you might have some unit tests that are green for a while which starts to be red for no apparent reasons.

The solution I came to is as this.

First I use Visual Studio, Copy to Output Directory, to copy the needed file to the output directoy in which your software will be started by the unit tests, e.g. for notValidVersionZip.zip

4309956698_b62daf51f5_o[1]

So now I am sure that the needed file is in the same path than the application. I then also need to be sure that when the application start the Windows open file dialog it points to this path. In the past implementation I was just using a filename and was lucky enough the path used by the Windows open file dialog was the correct one.

To get to the correct path is easy. We just navigate to the correct path using the Windows open file dialog in an automated way. The correct path is the path in which the application as been started, so you can get it like that:

  1. /// <summary>
  2. /// Gets the current path.
  3. /// </summary>
  4. /// <returns></returns>
  5. private static string GetCurrentPath()
  6. {
  7.     return Path.GetDirectoryName(Assembly.GetExecutingAssembly().CodeBase);
  8. }

We have the correct path and we still need to automate the Windows open file dialog to navigate to that path. We can do this like that:

  1. protected void Open(string filename)
  2. {
  3.     OpenButton.Click();
  4.  
  5.     var openModalWindow =
  6.         MainWindow.ModalWindow("Please choose a Zip file", InitializeOption.NoCache);
  7.     Assert.IsNotNull(openModalWindow);
  8.  
  9.     var splittedPath = GetCurrentPath().Split(new[] { '\\' });
  10.  
  11.     foreach (var pathPart in splittedPath)
  12.     {
  13.         openModalWindow.Enter(pathPart);
  14.         openModalWindow.Keyboard.PressSpecialKey(KeyboardInput.SpecialKeys.RETURN);
  15.         openModalWindow.WaitWhileBusy();
  16.     }
  17.  
  18.     openModalWindow.Enter(filename);
  19.     openModalWindow.Keyboard.PressSpecialKey(KeyboardInput.SpecialKeys.RETURN);
  20. }

Basically we split the path into it different path parts that White will enter into the dialog followed by a enter. Don’t forget to use the method WaitWhileBusy() after each enter, otherwise it will be too fast and sometime your test will not go to the correct path and then will not find the file.

Finally White enter the filename followed by enter and the file is opened.

Nice!

If you are using like me ReSharper to run your unit tests don’t forget to set it up to run tests from Project output folder.4309993844_8d9e828f8c_o[1]

Jan
28
2010

Automated WPF functional tests using White

I’d like to introduce a tool that I have added for a month or two in my toolset. This tool is White from ThoughtWorks. Here is the description of White:

White: Automate windows applications

White supports all rich client applications, which are Win32, WinForm, WPF and SWT (java).
It is .NET based and hence you wouldn't have use proprietary scripting language. You can use your favourite .NET language, IDE and tools for developing tests/automation programs.
White provides consistent object oriented API for all kinds of applications. Also it hides all the complexity of Microsoft's UIAutomation library and windows messages (on which it is based).
(While WHITE is completely ready to be used, the documentation is still work under progress. Please do point out the areas which needs documentation.)

When I found White with it’s version 0.19, I was a bit skeptical about the state of this piece of software. But it is really ready to be used.

By the way the latest version 0.19 of White supports also Silverlight.

I had the need to automate some functional tests of a WPF application that I am working on. When such an application is growing it becomes more and more difficult to do functional tests manually, so it needs automation.

So what I was searching for was something that would let me write unit tests in C# with NUnit which automate a WPF application. It is not that I don’t want to learn another language but I needed to be efficient so I preferred to use something I knew. And what I have found is White.

I have tested it with the RadRibbonBar for WPF of Telerik and it works just great. btw Thanks to Telerik for the offered license, which my company Innoveo Solutions also ordered now.

If you are starting with automated functional testing I recommend you to read the Functional Testing. It explains step by step how to write automated functional tests with White. The basic can be also used with another tool.

It is funny to see you application running without you clicking around.

I still need to have this run by our build server, TeamCity. I have currently not done any investigation on that point. I plan to have those tests running at least nightly, but I have to see how the unit tests could be started on an environment with a desktop.

Oct
8
2009

Unit Tests without leaving the keyboard

I like the Roy Osherove blog: Five Whys, Leadership in software teams.

Follow up on those two posts “How to measure programmer productivity using TDD Katas” or “Be Productive and Go Mouseless”. I would like to share a little keyboard shortcut which save me quite some time on my daily developments.

As you might know I am a fan of ReSharper and when I am doing Test Driven Development I hate having to switch from keyboard to mouse to run the test I just wrote. So searching for ReSharper Keyboard command I found Resharper_UnitTest_ContextRun which I assign to the Ctrl+Shift+<.

Now I can run the unit test I just wrote with just a combination of keys which are really near to each other so really easy!

The bonus of this command is that if I have the cursor in one unit test in a TestFixture class then it will run that particular test, but if I have the cursor out of any unit test method it will run all tests of that TextFixture. Very efficient!

Enjoy it!

May
9
2009

Updated my Live Template for NUnit in ReSharper

I tend to use a lot the Live Template of ReSharper, for example working for some time now with the WPF MVVM pattern I created a ViewModel template of such a class and use it extensively.

Today I updated the File Template I use to write my NUnit tests like this:

using NUnit.Framework;

namespace $NAMESPACE$
{
    // ReSharper disable InconsistentNaming

    [TestFixture]
    public class $CLASSNAME$
    {       
        /*
         * ... hold ...
         *
         * Arrange - Act - Assert
         */

        [Test]
        public void $FIRST_TEST_NAME$()
        {
            $END$
        }
    }

   // ReSharper restore InconsistentNaming
}

where I have the variable names defined as this:

Nov
5
2008
unit test // Euss // ORM // Agile

Follow up on “Reducing ORM Friction” by Rob Conery

In my development process I do use what Rob is describing in his post “Crazy Talk: Reducing ORM Friction” with some slight differences.

For example I developed Tech Head Brothers portal this way, as Innoveo Solutions web site. I use TDD and Domain Driven Development and I keep the mapping as one of the last step for my implementation.

I do have a generic Repository interface as following:

using System;

using System.Collections.Generic;

using System.Linq;

 

namespace TechHeadBrothers.Portal.Infrastructure.Interfaces

{

    /// <summary>

    /// IRepository exposes all methods to access the data repository

    /// </summary>

    public interface IRepository

    {

        void InitializeRepository();

 

        bool Save<T>(T entity) where T : class;

        bool SaveAll<T>(IList<T> entities) where T : class;

 

        bool Delete<T>(string id) where T : class;

 

        T Find<T>(string id) where T : class;

        IQueryable<T> Find<T>();

        IQueryable<T> DetachedFind<T>();

        IQueryable<T> Find<T>(System.Linq.Expressions.Expression<Func<T, bool>> expression);

 

        int Count<T>();

        int Count<T>(System.Linq.Expressions.Expression<Func<T, bool>> expression);

    }

}

 

 

My ORM mapping tool of choice is Euss. And here comes the slight difference, I do have one implementation of my interface leveraging Euss, and that’s it. All different possibilities are handled by Euss. During my work on the definition of the domain I took the habit to use an Euss XML Engine or an Euss Memory Engine. I use those two engine for my unit test and my real application.

Following the lean principle I postpone the choice of the data repository till the last minute, when I know more about the real need. So it really happen that I stay with an XML Engine so that all my data are stored in an XML file. If I need more I go to an Euss SQL Mapper Engine and then define the mapping.

So I moved to the ORM framework the different implementations.

Now I am still free to go to another ORM, or something else, by using the interface IRepository.

I used several time this technique and I am currently happy about it.

Jun
25
2008

ReSharper and Source Analysis Tools for C#

If you do have Microsoft Source Analysis Tools for C# and JetBrains ReSharper 4.0 installed in you development environment than you might get issue when you run your unit tests through ReSharper Unit Test Explorer and Sessions. As you can read on my bug report feedback:

There is known problem in StyleCop, which prevents build system of VS to report any progress. Unit Testing waits for project to be built. You can switch building to "Never" on Session toolbar and build manually, or uninstall StyleCop

Jun
1
2008

Mocking .NET framework SmtpClient class

This Saturday like the last two I planned to work on my wooden terrace, but with the weather we have for this year's spring, it was almost impossible. So I replaced that with some development.

I am using Rhino.Mocks as mock object framework and went to the following solution to mock SmtpClient.

Capabilities of Rhino.Mock are to mock interfaces, delegates and virtual methods of classes!

My goal was to test my MailService class which use SmtpClient and in particular the method SendAsync, which is not a virtual method. SmtpClient inherit from System.Object so no way to use an interface for the unit test.

Next step was then to make an interface, ISmtpClient, out of the SmtpClient of the .NET Framework using Reflector for .Net. Then I modified the dependency of my MailService class from SmtpClient of the .NET framework to my interface ISmtpClient.

using System.Net;

using System.Net.Mail;

using System.Security.Cryptography.X509Certificates;

using System.Security.Permissions;

 

namespace TechHeadBrothers.Portal.Services.Mail

{

    public interface ISmtpClient

    {

        // Events

        event SendCompletedEventHandler SendCompleted;

 

        // Properties

        X509CertificateCollection ClientCertificates { get; }

        ICredentialsByHost Credentials { get; set; }

        SmtpDeliveryMethod DeliveryMethod { get; set; }

        bool EnableSsl { get; set; }

        string Host { get; set; }

        string PickupDirectoryLocation { get; set; }

        int Port { get; set; }

        ServicePoint ServicePoint { get; }

        int Timeout { get; set; }

        bool UseDefaultCredentials { get; set; }

 

        // Methods

        void Send(MailMessage message);

        void Send(string from, string recipients, string subject, string body);

        [HostProtection(SecurityAction.LinkDemand, ExternalThreading = true)]

        void SendAsync(MailMessage message, object userToken);

        [HostProtection(SecurityAction.LinkDemand, ExternalThreading = true)]

        void SendAsync(string from, string recipients, string subject, string body, object userToken);

        void SendAsyncCancel();

    }

}

Then I wrote the class SmtpClientProxy. It is based on the design pattern Proxy. So it basically maintains a reference, and controls access, to the real SmtpClient so it can be used in place of the real SmtpClient.

namespace TechHeadBrothers.Portal.Services.Mail

{

    public class SmtpClientProxy : ISmtpClient

    {

        private readonly SmtpClient smtpClient;

 

        public SmtpClientProxy()

        {

            smtpClient = new SmtpClient();

            smtpClient.SendCompleted += smtpClient_SendCompleted;

        }

 

        #region ISmtpClient Members

 

        public event SendCompletedEventHandler SendCompleted;

 

        public X509CertificateCollection ClientCertificates

        {

            get { return smtpClient.ClientCertificates; }

        }

 

        public ICredentialsByHost Credentials

        {

            get { return smtpClient.Credentials; }

            set { smtpClient.Credentials = value; }

        }

Finally my MailService class with the dependency injection of ISmtpClient interface and a default constructor using my SmtpClientProxy :

namespace TechHeadBrothers.Portal.Services.Mail

{

    /// <summary>

    /// Service to deliver Emails

    /// </summary>

    public class MailService : IMailService

    {

        private readonly ISmtpClient smtpClient;

 

        public MailService() : this(new SmtpClientProxy())

        {

        }

 

        public MailService(ISmtpClient smtpClient)

        {

            this.smtpClient = smtpClient;

        }

In my unit test I will use the constructor in which I can specify the mock of ISmtpClient, otherwise I will use the default constructor.

So that was for the first issue; having the possibility to mock SmtpClient. Now you certainly have realized the second issue that popped up. In my ISmtpClient I have one event:

    public interface ISmtpClient

    {

        // Events

        event SendCompletedEventHandler SendCompleted;

This event is for sure also in my proxy class, SmtpClientProxy as it inherit form ISmtpClient. In the constructor of SmtpClientProxy I add an event handler on the SendCompleted event of the SmtpClient. This event handler just fires the event exposed by the proxy class, so that I can have an event handler in MailService class to handle the SendComplete event.

        private void smtpClient_SendCompleted(object sender, AsyncCompletedEventArgs e)

        {

            if (SendCompleted != null)

                SendCompleted(sender, e);

        }

Nothing really special. But now the question rise! I need to mock ISmtpClient in my unit test. But my MailService SendMailMessage method call the SmtpClient.SendAsync method and also add an event handler to SmtpClient.SendCompleted event.

        /// <summary>

        /// Sends a MailMessage object using the SMTP settings.

        /// </summary>

        /// <param name="mailMessage">Email message to be sent</param>

        public void SendMailMessage(MailMessage mailMessage)

        {

            try

            {

                mailMessage.IsBodyHtml = true;

                mailMessage.BodyEncoding = Encoding.UTF8;

 

                this.message = mailMessage;

 

                smtpClient.SendCompleted += smtpClient_SendCompleted;

                smtpClient.SendAsync(mailMessage, null);

            }

            catch (SmtpException)

            {

                this.OnEmailFailed(mailMessage);

            }

        }

 

        private void smtpClient_SendCompleted(object sender, AsyncCompletedEventArgs e)

        {

            this.OnEmailSent(message);

        }

In my mock then I need to have the same thing happening, even if I mock the interface ISmtpClient.

Here is the solution I came to:

namespace TechHeadBrothers.Portal.Services.Tests.Mail

{

    [TestFixture]

    public class MailServiceTest

    {

        #region Setup/Teardown

 

        [SetUp]

        public void SetUp()

        {

            mocks = new MockRepository();

        }

 

        #endregion

 

        private MockRepository mocks;

 

        [Test]

        public void SendMailMessageRaiseEmailSentEvent()

        {

            bool emailSentRaised = false;

 

            var message = new MailMessage("lk@test.com", "mk@test.com");

 

            var smtpClient = mocks.Stub<ISmtpClient>();

 

            var mailService = new MailService(smtpClient);

            mailService.EmailSent += ((sender, e) => { emailSentRaised = true; });

 

            using (mocks.Record())

            {

                var raiser = Expect.Call(() => smtpClient.SendCompleted += null)

                                   .IgnoreArguments().GetEventRaiser();

 

                Expect.Call(() => smtpClient.SendAsync(message, null))

                      .Do((Action<MailMessage, object>) ((arg1, arg2) => raiser.Raise(message, null)));

            }

 

            using (mocks.Playback())

            {

                mailService.SendMailMessage(message);

 

                Assert.That(emailSentRaised, Is.EqualTo(true));

            }

        }

    }

}

I create a mock of the interface ISmtpClient. Then I inject this mock into my MailService class. I create a event handler for my MailService.EmailSent event using a lambda. This lambda, if called, will change the boolean value emailSentRaised from false to true. This exactly what I want to test in that unit test; that the EmailSent event is raised.

Then I get raiser object, a Rhino Mock IEventRaiser, from smtpClient.SendCompleted event.

Finally I set an expectation on smtpClient.SendAsync method adding a Do() handler that will raise the SendCompleted event.

Last point is using the mailService object, calling the SendMailMessage method and asserting that my boolean emailSentRaised went from false to true.

And here is the result in ReSharper Unit Test Session window. Green!

 

Update: By the way, I forgot to add a link to a post from Phil Haack "Using Rhino Mocks To Unit Test Events on Interfaces" and another link to a post from Jean-Paul S. Boodhoo "Raising events (from a mock) using Rhino Mocks". To great post to read!

Apr
9
2008

Team City 3.1 and NDepend 2.8 integration

I finally managed to integrate those two wonderful tools Team City and NDepend as you can see on the following picture:

I will post soon the way I use to integrate those two, as I did for NCover.

About Laurent

Laurent Kempé

Laurent Kempé is the editor, founder, and primary contributor of Tech Head Brothers, a French portal about Microsoft .NET technologies.

He is currently employed by Innoveo Solutions since 10/2007 as a Senior Solution Architect and certified Scrum Master.

Founder, owner and Managing Partner of Jobping, which provides a unique and efficient platform for connecting Microsoft skilled job seekers with employers using Microsoft technologies.

Laurent was awarded Most Valuable Professional (MVP) by Microsoft from April 2002 to April 2012.

JetBrains Academy Member
Certified ScrumMaster
My status

Twitter

Flickr

www.flickr.com
This is a Flickr badge showing public photos and videos from Laurent Kempé. Make your own badge here.

Month List

Page List