14 April 2007

Test Smell: Everything is mocked


This Test Smell is about a weakness in the design of tests, rather than in the code to be tested.

It may sound obvious, but you don't have to mock every class in your system.

Don't mock value objects

There isn't much point in writing mocks for simple value objects (which should be immutable anyway), just create an instance and use it. For example, in this test:

@Test public void sumsTotalRunningTime() {
  Show show = new Show();
  Video video1 = context.mock(Video.class);
  Video video2 = context.mock(Video.class);
  context.checking(new Expectations(){{
    one(video1).getTime(); will(returnValue(40));
    one(video2).getTime(); will(returnValue(23));
  assertEqual(63, show.getRunningTime())

where Video holds details of a part of a show. It's not worth creating an interface / implementation pair to control which time values are returned, just create instances with the appropriate values and use them. There are a couple of heuristics for when a class is not worth mocking. First, it has only accessors or simple methods that act on values it holds, it doesn't have any interesting behaviour. Second, you can't think of a meaningful name for the class other than VideoImpl or some such vague term.

Don't mock third-party libraries

There are two problems when mocking a third-party library. First, you can't use the tests to drive the design, the API belongs to someone else. Mock-based tests for external libraries often end up contorted and messy to get through to the functionality that you want to exercise. These tests are giving off smells that highlight designs that are inappropriate for your system but, instead of fixing the problem and simplifying the code, you end up carrying the weight of the test complexity. The second issue is that you have to be sure that the behaviour you implement in a mock (or stub) matches the external library. How difficult this is depends on the API, which has to be specified (and implemented) well enough for you to be certain that the tests are meaningful.

To develop against an external API, first TDD your code to define interfaces for the services that your application needs in terms of your domain. Then write a thin layer using the library to implement these interfaces, with focussed integration tests to confirm your assumptions about how it works. How to make integration tests work effectively will depend on your environment, but you will need to do it anyway so you might as well start early and get the benefit. There will be relatively few integration tests, compared to the number of unit tests, so they should not get in the way of the build even if they're not as fast as the in-memory tests.

There are some exceptions when mocking third-party libraries can be helpful. You might use mocks to simulate behaviour that is hard to trigger with the real library, such as throwing exceptions. Similarly, you might use mocks to test a sequence of calls, for example making sure that a transaction is rolled back if there's a failure. There should not be many of these in a test suite.


Anonymous said...

Some advice I give people about when they've mocked too much.

Oli Bye said...

Don't say Don't!

I'm not sure I agree with not mocking third party libraries.

As I remember, the first mock object was HttpServletRequest as we wanted to test Servlets outside a container.

Recently we used JMock to a Mock up JDBC driver, so that we could test how slightly different hibernate mappings behaved.

Yes it was annoying setting all the field values, but we learnt a huge amount about hibernate.

We wouldn't do this every time we did a mapping, but I personally think the JMock is great for understanding the behaviours of third party libraries.

We've be able to tune our mappings for performance, and catch changes in the implementation that would have broken our code.

Never say never!

Steve Freeman said...

Thanks for an interesting response. That's why I added the last paragraph, but you show what an experienced team can do with the technique.

We use Joe Walnes' phrase "Never mock a type you don't own" to get across the idea of mocks as a design technique, since many authors seem to have got stuck on just using mocks to make tests with databases go faster.

We have another posting in the backlog that concerns mocking clients of a framework.

Anonymous said...

I definitely agree with what you say in this post.

When reading the comments though I noticed the link to this article "http://jbrains.ca/permalink/90".

The author seems to have the same views as I do on domain model testing and I wondered how/if you think this fits in with the interaction based style you guys go for?

Meno said...

Isn't a unit test designed to test some code in isolation regardless of whether an external class does something interesting or even if it is just a dumb entity object with getter/setter properties? That is, I shouldn't care or need to know what the external class is doing, which is why it should be stubbed.

Steve Freeman said...

@Meno. It's not that simple. The result of mocking everything tends to be the sort of brittle, incomprehensible tests that people say mocking leads to. Every time someone posts an example just mocks a simple getter, a fairy dies...

This stuff works best when we think in terms of relationships and actions, rather than queries and lumps of data.

For a longer discussion, see http://www.mockobjects.com/book/achieving-object-oriented-design.html