aspose file tools*
The moose likes Testing and the fly likes To Mock or Not to Mock, That is the Question Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Spring in Action this week in the Spring forum!
JavaRanch » Java Forums » Engineering » Testing
Bookmark "To Mock or Not to Mock, That is the Question" Watch "To Mock or Not to Mock, That is the Question" New topic
Author

To Mock or Not to Mock, That is the Question

Junilu Lacar
Bartender

Joined: Feb 26, 2001
Posts: 4757
    
    7

E Armitage wrote:IMO, it's much better to do away with mocks and use tools like arquillian so your tests run against a test/mock database rather than mocking your classes.


Hmm, just took a brief look at Arquillian and it looks like it's concerned with integration tests. Here's one guy who would disagree with needing integration tests and doing away with mocked up DAOs for unit testing: http://www.infoq.com/presentations/integration-tests-scam If the name doesn't ring a bell, he's the author of "JUnit Recipes". I think I'm with J.B. on this.


Junilu - [How to Ask Questions] [How to Answer Questions]
E Armitage
Rancher

Joined: Mar 17, 2012
Posts: 892
    
    9
Junilu Lacar wrote:
E Armitage wrote:IMO, it's much better to do away with mocks and use tools like arquillian so your tests run against a test/mock database rather than mocking your classes.


Hmm, just took a brief look at Arquillian and it looks like it's concerned with integration tests. Here's one guy who would disagree with needing integration tests and doing away with mocked up DAOs for unit testing: http://www.infoq.com/presentations/integration-tests-scam If the name doesn't ring a bell, he's the author of "JUnit Recipes". I think I'm with J.B. on this.

I did say it's my opinion.
Not quite true that Arquiillian is for integration tests. You still write unit tests. You just don't need to write lots of untested mocks anymore.
I expect a lot of people will still write mocks but more and more are realizing the false sense of security they provide.
P.S There are more arguments about this here: http://bill.burkecentral.com/2012/03/13/java-ee-wins-over-spring/
Junilu Lacar
Bartender

Joined: Feb 26, 2001
Posts: 4757
    
    7

E Armitage wrote:You just don't need to write lots of untested mocks anymore.


But that's the whole point of using mocks: so you don't have to worry about untested components. You make the mocks behave exactly the way you expect the components they represent to behave. That is, you are assuming that you are using 100% correct (fully tested) components. You're also making the assumption that when the real components are plugged in, they will work according to contract. If you're worried about the other components' behavior, then the problem is probably that you don't have enough tests for those components. That's not a mock vs. no mock issue. That's a trust issue.

Thanks for the link, I'll check it out.
E Armitage
Rancher

Joined: Mar 17, 2012
Posts: 892
    
    9
Junilu Lacar wrote:
E Armitage wrote:You just don't need to write lots of untested mocks anymore.


But that's the whole point of using mocks: so you don't have to worry about untested components. You make the mocks behave exactly the way you expect the components they represent to behave. That is, you are assuming that you are using 100% correct (fully tested) components. You're also making the assumption that when the real components are plugged in, they will work according to contract. If you're worried about the other components' behavior, then the problem is probably that you don't have enough tests for those components. That's not a mock vs. no mock issue. That's a trust issue.

Thanks for the link, I'll check it out.

You didn't get my point. It's not the other components that are untested. It's the mocks themselves that are not tested (and I have seen and created bugs in mocks myself). The point is that you just don't need to write the mocks at all since it is now easy and cheaper to provide the real component. You still write comprehensive unit tests for those components as well.
The argument is not about which approach provides better test coverage. Both approaches result in the same number of unit tests for the same units. The Arquillian way just doesn't need the mocks to achieve it.
Junilu Lacar
Bartender

Joined: Feb 26, 2001
Posts: 4757
    
    7

E Armitage wrote:You didn't get my point. It's not the other components that are untested. It's the mocks themselves that are not tested (and I have seen and created bugs in mocks myself). The point is that you just don't need to write the mocks at all since it is now easy and cheaper to provide the real component. You still write comprehensive unit tests for those components as well.


And you seem to have missed my point, too. In my last post, untested mocks == untested components. I submit that there should be no such thing as an untested mock. You make the mocks behave exactly the way you want them to behave. Mocking/stubbing should ensure that you are getting the exact behavior you want (equivalent to 100% tested) and Mokito makes it easy to do that. Now as for buggy mocks, well that's a different story. When you hand-craft your mocks, which I guess is what you and the OP do, then yes, there's a likelihood of you hand-crafting some bugs into them, too. Which is why I discourage writing hand-crafted mocks because they often add unnecessary complexity.

E Armitage wrote:The argument is not about which approach provides better test coverage.


Sorry, I missed that. Didn't think I was saying anything about code coverage per se.
Junilu Lacar
Bartender

Joined: Feb 26, 2001
Posts: 4757
    
    7

E Armitage wrote:The point is that you just don't need to write the mocks at all since it is now easy and cheaper to provide the real component...The Arquillian way just doesn't need the mocks to achieve it.


I really don't know much about Arquillian but from what I've seen and what you've said so far, it looks like it makes integration testing easier by taking care of a lot of the plumbing concerns. But that's not relevant to mocks vs. no mocks for unit testing.

One rule of thumb for writing good unit tests is that you don't cross execution boundaries: Going to a file system, a database, a web service, etc. This is because doing so makes the test run a lot slower. You want a unit test to run fast. Really fast. When you are unit testing a Business Service (eg. BalanceTransferService) that interacts with an Infrastructure Service like a DAO or Web Service endpoint, you absolutely should mock out that Infrastructure Service in your unit test for the Business Service. So what would be the Arquillian way of doing this without a mock?
E Armitage
Rancher

Joined: Mar 17, 2012
Posts: 892
    
    9
I really don't see how you can guarantee 100% tested mocks hand-crafted or otherwise. Even the JUnit fathers have pointed out that the tests of the test cases and their mocks is production code.
The real question here is, why have the mocks at all when you have the real service available cheaply?
E Armitage
Rancher

Joined: Mar 17, 2012
Posts: 892
    
    9
Junilu Lacar wrote:..
..This is because doing so makes the test run a lot slower. You want a unit test to run fast. Really fast. When you are unit testing a Business Service (eg. BalanceTransferService) that interacts with an Infrastructure Service like a DAO or Web Service endpoint, you absolutely should mock out that Infrastructure Service in your unit test for the Business Service. So what would be the Arquillian way of doing this without a mock?

Speed is no longer an issue for most ee project tests in ee6. A lot is changing with ee6. Arqullian starts up a minimal deployment archive for the test you want to run (you hand pick the classes you want included). There could be some services that will still be to too heavy to leverage so you may still have to mock. Generally, such services are now the exception rather than the rule. You certainly don't need to mock out a DAO at all, just fire up an in memory DB. WS endpoints may or may not need to be mocked out depending on their location relative to the unit under test. It's rare to find more than five such services in a scalable application (your mileage may vary).
Take away the speed bottleneck as has been done by ee6 and modern day ci server specs and you are left with very few reasons to write mocks. This should be good news because it allows us more time to spend time writing business logic and business logic test cases.
Junilu Lacar
Bartender

Joined: Feb 26, 2001
Posts: 4757
    
    7

E Armitage wrote:I really don't see how you can guarantee 100% tested mocks hand-crafted or otherwise. Even the JUnit fathers have pointed out that the tests of the test cases and their mocks is production code.


I don't think they meant what you think they meant.

When I create a mock to test with my production code, I'm making a representation of an implementation that is 100% correct in the way it behaves. I am not making any guarantees that a real implementation will also work 100% correctly though. So "The tests of the test cases and their mocks is production code." simply means that production code will show you if the assumptions you made in your tests were good or not. If you failed to write a test that production code eventually shows you should have written there's nothing your current tests or mocks could have done to prevent that. There's a quote, and I paraphrase, "I can't guarantee the correctness of this program. I've only tested it. I haven't run it in production." or something to that effect. Can't remember who it was either but I think that's more in line with what you quoted the JUnit fathers as saying.

E Armitage wrote:The real question here is, why have the mocks at all when you have the real service available cheaply?


Loosely coupled code and being able to test in isolation comes to mind. Depends on how you define cheap. If cheap includes fast, reliable, and correct, then maybe a real service can be used in a unit test. I have a nagging feeling that you and I differ in our definition of a unit test though.

Edit: Found the quote I was thinking of: http://en.wikiquote.org/wiki/Donald_Knuth -
Donald Knuth wrote:"Beware of bugs in the above code; I have only proved it correct, not tried it."


Yeah, I "mis-remembered" and the original is different from what I was thinking but I think pretty much of the same essence.
Junilu Lacar
Bartender

Joined: Feb 26, 2001
Posts: 4757
    
    7

This is direct from the Arquillian site:
Arquillian is an innovative and highly extensible testing platform for the JVM that enables developers to easily create automated integration, functional and acceptance tests for Java middleware


Sounds cool and potentially very useful. It seems to me, however, that some of the motivation to do away with mocks stems from anti-patterns that have evolved around their use, not because of mocks or the mocking technique themselves. Their battle cry appears to be "Death to all bugs!", not "Death to all mocks!"

I read Bill Burke's blog entry that was cited and most of the responses to it. I have to say that I'm firmly in the camp that takes an opposing view of Bill's position and assertions about mocking and Spring. I wish no ill-will towards either Spring or JEE and whichever technology facilitates good, well-architected solutions is what I will use. It's also comforting and validating to see that others who also oppose Bill's views have said pretty much some of the same things I have said in this thread.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: To Mock or Not to Mock, That is the Question