To mock or not to mock? What is the best practice?

[Carl McGraw]

Hello all. I am working on a personal Java project that is large/complex enough to need unit testing to check that it is operating as expected (and for refactoring).

This is a very general question about what is viewed as the best practice when writing tests.

Lets say I have the following interface:

public interface GenericClass {
    public void run();
}

Then the implementation:

public class ImplementationClass implements GenericClass{
    private A componentA;
    private B componentB;
    private C componentC;
 
    public implementationClass() {...}

    // Used for mocking out components, not used in production
    public ImplementationClass(A componentA, B componentB, C componentC) {...}

    public void run {
        componentA.doA();
        componentB.doB();
        componentC.doC();
    }
}

Now when writing unit tests for ImplementationClass. Is it better practice to mock out each of the component classes and just test that they were run?

In doing this I seem to bind the unit tests to the implementation of ImplementationClass. When I change the implementation details of the class in the future I find that almost all of that specific unit test needs to be rewritten (since it may no longer rely on the components being used as expected).

Is it better to instead write the unit test from the prospective of the "user" and just have a simple input -> ImplementationClass -> output type of test, without any regards to what is happening inside of the class?

I find that when doing this any changes to the implementation of ImplementationClass don't really effect the test itself. But the test has dependencies on other modules (which isn't really a unit test anymore is it?)

I worry that the implementation and the unit tests are so bound up together it actually is making my test code more brittle than it needs to be.

What do you think?

This question may be entirely subjective. If that is the case, what is your opinion then on the best practice? If you were just starting on a new project and had to handle some of the original code base, which would be helpful to have?

[Tim Cooke]

Hello Carl, welcome to the Ranch!

As with most non-trivial problems, the answer is "it depends".

It depends on what you define as a unit. If your ImplementationClass, A, B, and C all work together to perform a single 'unit' of functionality in your application then you might want to test them all together using the production versions of each class.

However, if any of the dependencies A, B, or C are difficult to instantiate, are very slow, or integrate with some external system that isn't always available or accessible, then you might want to replace them with a mocked version in your tests.

You are quite right in your observation that by writing too fine grained tests you end up encasing your implementation in stone, which can be very restrictive when you come to do some refactoring. Being pragmatic about your tests will help you with finding what a suitable testable unit is in your application. Your tests should describe a function in terms of your problem domain, not in terms of the implementation.

[Yegor Bugayenko]

Instead of mocking it's better to use fake objects. I've published an article on this subject a few weeks ago: http://www.yegor256.com/2014/09/23/built-in-fake-objects.html

So, in your case, before instantiating ImplementationClass, prepare fake versions of A, B, and C. Then, use them in the ctor. As a rule of thumb, try to stay as far away from mocking frameworks, as you can.

[Matthew Brown]

I generally find the distinctions between the different kinds of test double to be not particularly helpful. Mocking frameworks, for instance, often provide very easy ways of producing a fake object.

[Tim Cooke]

Yegor Bugayenko wrote:Instead of mocking it's better to use fake objects

Not always.

Yegor Bugayenko wrote:As a rule of thumb, try to stay as far away from mocking frameworks, as you can.

Again. Not always.

That's quite a bold statement to throw out there with no context. It is true that there are times when a mocking framework, such as Mockito and EasyMock, are too much and a hand rolled Spy, Fake, or Stub, will make a more readable test. But to state it as a 'rule of thumb' is just plain misleading. It is better to understand about creating your own Fakes, and understand what a mocking framework gives you, so you can make an informed decision on which solution works best in your particular context.

In this example, we don't know what the dependencies A, B, and C are so we cannot presume to know what would be the best way to deal with them in a test. If you need to deal with them at all even.

Side note:
I quite like your article that you've linked to, however I can't help but feel you've glossed over the complexity of writing the fake MkRegion object you use in the example.

[Carl McGraw]

Sorry in advanced. I wasn't sure how techincal of a response I'd get on my first post so I went a little abstract in the question. I feel like you all know a lot, so let me respond and then try to be more specific in my question (with more concrete code at the end).

Time Cooke wrote:It depends on what you define as a unit. If your ImplementationClass, A, B, and C all work together to perform a single 'unit' of functionality in your application then you might want to test them all together using the production versions of each class.

That makes sense. But how do I really determine what is a "unit" of functionality?

I have tried following SRP as best as I can, and as such have many smaller classes which compose other

Tim Cooke wrote:Your tests should describe a function in terms of your problem domain, not in terms of the implementation

As I am still a novice programmer I am clearly having a difficult time drawing that line in the sand. Maybe more experience will help more.

How do you normally do this? Do you utilize your API to decide what is and isn't a "unit"?

Yegor Bugayenko wrote:As a rule of thumb, try to stay as far away from mocking frameworks, as you can.

As a newbie I've been a bit of a Java "purist" with my projects. Only utilizing the standard library (because I have been trying to build comprehensive projects that showcase some knowledge). So I've never really used a mock framework (and have never developed a system so complex that I felt that I couldn't complete any given test with only 1-3 simple mocks I made by hand).

Reading your blog post you echo what I was hearing from Kent Beck and Martin Fowler in a interview I watched a couple months back. In it Beck and Fowler both seem to reel at the idea of mocking frameworks that return multiple levels of mocks. They also claim to not really use mocking at all during their unit testing because it pushes into the implementation. Having stewed on this a while I am trying to practice and write better unit tests.

Yegor Bugayenko wrote:Instead of mocking it's better to use fake objects. I've published an article on this subject a few weeks ago

I am not even understanding what the difference between the mock and fake are (except for a framework being used to generate one). I can't help but feel very confused by the fake object you're using. While it may be clear to you, I find it difficult to understand out of the context of the test.



But maybe we are having the same problem with my simple example as well

I will try to be more specific and show you what I am doing. Then you can tell me if this is a bad idea (since its hard to look at it in the abstract).

Lets use the previous example, but make it more interesting by having object interactions instead of just straight method calls. This is a more concrete example of what I am actually doing in my tests:

Lets say I have the following class I am testing (from before):

public ImplementationClass implements GenericClass {
     ...

     public void run() {
         Result result = ComponentA.doA();
         componentB.doB(result);
         // componentC missing because no one really cares...
     }
}

Now in this case I had simply done the following mocks:

public class AMock implements A {
    public Result result;

    public Result doA() {
        return result;
    }
}

public class BMock implements B {
     public Result result;
      
     public void doB(Result data) {
         result = data;
     }
}

public class ResultStub implements Result{
     // Whatever methods are simply empty
}

Then my test code has a little preparation it does, but I can control what is or isn't returned by the components.

public class ImplementationClassTest {
    private ImplementationClass classUnderTest;
    private AMock componentA;
    private BMock componentB;
    private ResultStub result;

    @Before
    public void setUp() {
        this.componentA = new AMock();
        this.componentB = new BMock();
        this.result = new ResultStub();
        
        this.classUnderTest = new ImplementationClass(componentA, componentB);
    }

    @After
    public void tearDown() {/*tear down*/}

    @Test
     public void testRun_ResultIsPassedToComponentB() {
         // Arrange
         componentA.result = result;
         
         // Apply
         classUnderTest.run();

         // Assert
         assertEquals(result, componentB.result);
     }
}

Generally that is as complex as my structure gets. I try to follow SRP enough such that I've never run into a case that I had to return multiple levels of mocks (normally just a mock that returns a stub, that stub is just confirmed that it is passed around).

Doing this I've been able to test all potential return values from expected to null and with throwing potential exceptions. But I am worried that it isn't good practice to do this.

[Yegor Bugayenko]

These are fake objects, no mocks: AMock and BMock

Mock is something you create like this:

A a = Mockito.mock(A.class);
Mockito.doReturn("hello").when(a).doSomething();

And this approach is error-prone, you should use it as a last resort. Fakes objects will make your tests compact and maintainable. Mocks will pollute your unit tests and even turn them into unmaintainable mess.

[Junilu Lacar]

@Yegor:

Interesting article about fake vs mock objects although I don't think it really got to the root of the problem: designs with big objects and complex dependencies. I don't see mock objects per se as evil. In my experience, the problem is often the context in which mocks are sometimes used or abused, as the case may be.

Developers live in the nuts and bolts of their code and when they're in that mode of thinking, it's easy to miss the forest for the trees. Mocks get a lot of blame dumped on them because they're right there in open; they're an easy target. However, if I dig deeper and look beyond the mocks, I usually find the underlying problem is really in the design of the class being tested. When unit tests and mocking starts to get out of hand, the smells I look for are: 1) too many dependencies, 2) insufficient or leaky abstractions, 3) mixed levels of abstraction (violations of SLAP - single level of abstraction principle). These are design problems, not mocking problems.

For example, I find it curious that in the article cited, the package that's supposed to contain only fake objects is named com.jcabi.dynamo.mock and that the example fake object class is named MkRegion, where "Mk" appears to be shorthand for "Mock"(?). It almost seems like you can't decide if these are mocks or fakes. Wishy-washy names are a code/design smell to me. Fake or mock, it's kind of the same difference though so I really wouldn't linger too long on this.

What I would look at more closely is the fact that the test code that uses the fake Region, MkRegion, does not really look a whole lot better. There's still too much setup noise for my taste. I would ask:

1. Why do we care that an Employee can fetch salary from a DynamoDb specifically? DynamoDb is an implementation detail; shouldn't Employee only be dealing with an abstraction of a source for salary data?
2. Why are details of how the datasource is set up (even if it's just a "fake" data source) showing up in a test for the Employee class? Again, should we care about this level of detail at the Employee class?

After some thought, I would probably try to rewrite the test this way:

  private Employee testSubject;
  ...
  @Test
  public void getSalary_accesses_datasource() {
      when(mockedRegion.getSalary("Jeff").thenReturn(50000);
      testSubject = new Employee("Jeff", mockedRegion);
      assertThat(testSubject.getSalary(), is(50000));
  }

Of course, there's a little bit more duplication to refactor out of the above code but more or less, the test becomes a lot less noisy now.
@OP:
I think mocks have their place in tests and there are many cases where they can help you drive an appropriate level of abstraction into your designs. That said, when mocking and mocks start to degrade the clarity of your code, then check your design to see if implementation details are misplaced and need to be pushed out/down to some other class.

[Yegor Bugayenko]

@Junilu you're absolutely right about Mk/Fk naming. I gave that names to classes some time ago, when I wasn't sure how to call them right - mock or fake. Now I'm sure, they are fake objects.

About your example, I also agree. If we can design our class this way, even better. But even though, compare this test:

region = Mockito.mock(Region.class);
Mockito.when(region).getSalary("Jeff").thenReturn(50000);
employee = new Employee("Jeff", region);  
assertThat(employee.getSalary(), is(50000));

with this one:

region = new MkRegion(50000);
employee = new Employee("Jeff", region);  
assertThat(employee.getSalary(), is(50000));

I find the second one much easier to understand.

[Tim Cooke]

Whether the test case written by Junilu, or Yegor, is better all depends on the complexity of MkRegion. What does it look like?

[Yegor Bugayenko]

What is more important that "how does it look like" is "how does it behave". Its behavior is rather complex, since it returns other objects, that also expose some behavior, etc. etc. The entire hierarchy of objects is hidden behind its simple 2-3 method interface.

[Junilu Lacar]

Carl McGraw wrote:Generally that is as complex as my structure gets. I try to follow SRP enough such that I've never run into a case that I had to return multiple levels of mocks (normally just a mock that returns a stub, that stub is just confirmed that it is passed around).

Doing this I've been able to test all potential return values from expected to null and with throwing potential exceptions. But I am worried that it isn't good practice to do this.

Carl, I think it's great that even as a novice programmer, you're already aware of design principle like SRP. Your concern about tests being too tied to the implementation is valid. However, there are different levels of testing and there happens to be one called "whitebox testing" where the tests are focused on internal details. In contrast, blackbox tests are more like the client-focused tests you mentioned that are not tied to a particular implementation but rather on the externally observable behavior. By nature, whitebox tests are more brittle than blackbox tests.

There's no rule that says all your tests for one particular class have to be in one unit test class. So, my approach would be to separate whitebox tests from blackbox tests. This way, if your implementation changes, only the test class with the whitebox tests needs to be changed.

The other thing to think about regarding your tests is this: are you using tests to drive your design thinking or are you using them to verify behavior after you've implemented some code? Most developers will answer the latter. I find that the former is more helpful in coming up with useful tests. I'll try to explain with an example later.

[Carl McGraw]

Junilu Lacar wrote:There's no rule that says all your tests for one particular class have to be in one unit test class. So, my approach would be to separate whitebox tests from blackbox tests. This way, if your implementation changes, only the test class with the whitebox tests needs to be changed.

I've read a bit about TDD/BDD and what not, but actually getting my design driven by tests is very difficult to actually accomplish. These tests most certainly are "verifying" types, and as such I think they are still "beginner" types of tests.

I haven't been able to really take the leap and have my design be informed by the tests yet. I feel like if I get enough practice writing tests though eventually things will click.

[Junilu Lacar]

Carl McGraw wrote:These tests most certainly are "verifying" types, and as such I think they are still "beginner" types of tests.

I haven't been able to really take the leap and have my design be informed by the tests yet. I feel like if I get enough practice writing tests though eventually things will click.

My son's viola teacher tells him "Practice makes habits. Perfect practice makes perfect." It doesn't take that much effort, even for a novice, to make the mind shift to being test-driven. In fact, I imagine it would be easier for novices to become test-driven because they don't have old habits to break. And you seem to have a leg up on the foundation of being test-driven because you are already thinking about design principles like SRP. Become familiar with the other SOLID principles. DRY and SLAP are also useful.

I was trying to look for an example but I think Yegor's example is a good of an example as any to start.

region = new MkRegion(50000);
employee = new Employee("Jeff", region);  
assertThat(employee.getSalary(), is(50000));

Even though this code looks pretty clean and perhaps even clear, try reading it out loud. Does it make sense when you hear yourself saying what the code does? Why would you construct a Region with 50000 as the constructor parameter? What does that mean? And why would I construct an employee with a Region parameter? Does that make sense? For me, the answer is "No" in both cases. I would then dig deeper into the design to see what kind of object relationships and collaborations I have set up.

Here are another couple of thoughts about tests that will help you make the mind shift to being test-driven:
1. Tests are detailed design specifications. They should reveal various aspects of your software design.
2. Test code should not be treated as a second-class citizen. You should give as much care and attention to test code as you do production code.

Edit - I just remembered the correct quote: "Practice makes permanent. Perfect practice makes perfect."

[Junilu Lacar]

Take a look at your example now:

public class ImplementationClass implements GenericClass{
    private A componentA;
    private B componentB;
    private C componentC;
 
    public implementationClass() {...}

    // Used for mocking out components, not used in production
    public ImplementationClass(A componentA, B componentB, C componentC) {...}

    public void run {
        componentA.doA();
        componentB.doB();
        componentC.doC();
    }
}

First off, I don't think it's very helpful for you to use names like ComponentA, ComponentB, and ComponentC. You want think about objects, their behaviors, and their relationships with each other. Good, meaningful names help you discover inconsistencies in your design as much as they help tell you when the design makes sense. So think of names that are meaningful. Same thing goes for the methods: doA(), doB(), and doC() are meaningless.

Next, ask yourself things like "Does it make sense to write a test for run()?" (Same question could be asked of getSalary() in the previous example, actually). That is, is the behavior of run() meaningful enough or significant enough to warrant writing test(s) against it? Is there an interesting aspect of its design or behavior that I want to document with a test? Does the run() method make sense as a behavior for this object in the first place?

[Junilu Lacar]

My previous two replies may seem like a digression from the original question of mocks or no mocks. So, to round out my comments and bring it back to that point, IMO, mocks are tools that facilitate your thinking about object relationships. If your mocking gets too complicated, then your object relationships are likely too complicated as well. If your test code requires a lot of complicated setup to test one small behavior, then there may be problems with your abstractions.

Again, I don't see mocks themselves as evil, they're just easy to blame for something that the programmer got himself into by focusing on the mechanisms of the testing framework instead of looking more closely at the design. Shifting your point-of-view to a more test-driven and design-centric approach helps you look past the mocks and ask better questions that delve deeper into the design of the software. This usually leads to better tests and better testing practices.

I think that as your designs get better, you'll find that mocks become friends that you can lean on at appropriate times but won't really need to have around all the time.

[Yegor Bugayenko]

Junilu Lacar wrote:If your mocking gets too complicated, then your object relationships are likely too complicated as well. If your test code requires a lot of complicated setup to test one small behavior, then there may be problems with your abstractions.

+1 Touche!

[Marcus Denker]

I would also say that if your mocking gets too complicated, it's the code that is the problem.

Mocking makes testing EASIER and normally DECOUPLES the test from the code. The thing is, if you do not mock,
you will use the entire real object - if anything changes with these objects, you will have to refactor your test.
Also, how can mocking make a test more complicated then creating fake / stub objects? I saw so many tests with 10-100 lines of just fake / stub object creation that totally distracted from the test and had to be touched every time the classes changed.
Using Mockito to mock objects however, using the Mockito.when()...thenReturn / thenThrow notation however, makes it really clear of what is expected to happen.

[Tim Cooke]

Awarding a Cow to Carl for a Journal worthy thread. Moo!