Win a copy of High Performance Python for Data Analytics this week in the Python forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Paul Clapham
  • Ron McLeod
  • Bear Bibeault
  • Liutauras Vilda
Sheriffs:
  • Jeanne Boyarsky
  • Tim Cooke
  • Junilu Lacar
Saloon Keepers:
  • Tim Moores
  • Tim Holloway
  • Stephan van Hulst
  • Jj Roberts
  • Carey Brown
Bartenders:
  • salvin francis
  • Frits Walraven
  • Piet Souris

Mock Objects

 
Ranch Hand
Posts: 3404
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I just realised we have been using Mock Objects for a while so it's not a new idea. The term 'Mock Objects' is new!
 
HS Thomas
Ranch Hand
Posts: 3404
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
But definitely not in an OO context. So it would be interesting to see how it's different.
 
Sheriff
Posts: 3341
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I'm moving this to Testing.
 
HS Thomas
Ranch Hand
Posts: 3404
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Oh good. I find it freer here to say we used to call it Test Data.
But now that it's all about objects and not databases I guess Mock Objects is a better phrase.
 
Sheriff
Posts: 6920
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I think you need to be careful with assuming that "mock objects" is the same as "test data" or "dummy objects", or whetever you have used before.
Mock objects used for testing have a few significant features rarely found in simpler dummy-data objects. A good mock object is designed from the ground up to enable behaviour to be tested, not just data values. Often this is done by preloading return values and/or by counting or asserting in method calls.
For example. let's assume I wanted to test whether my code stores the correct value ("ugh") in the attribute named "wibble" in a shared context. Without mock objects I might test it like this:

This looks OK, but there is a problem. This code does not test that my code leaves all the other attributes alone. This code does not test that it does not put the value in the attribute more than once. Does this matter? Maybe not, but wouldn't it be comforting to know that it only puts "ugh" into "wibble"? Wouldn't it be nice for our test system to fail an assertion and throw a stack trace as soon as a problem occurred, rather than waiting until after the dust has settled?
Instead, lets create a simple mock object class:

Now the test code tests that the object under test only puts what we are expecting:

That is the power of mock objects in a nutshell. It can take a while to think through the implications, but it is a really powerful tool for your testing toolbox, and surprisingly different to a more traditional dummy-data approach.
Did that make sense?
[ January 02, 2004: Message edited by: Frank Carver ]
 
Ranch Hand
Posts: 775
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Frank Carver:
I think you need to be careful with assuming that "mock objects" is the same as "test data" or "dummy objects", or whetever you have used before.
Mock objects used for testing have a few significant features rarely found in simpler dummy-data objects. A good mock object is designed from the ground up to enable behaviour to be tested, not just data values. Often this is done by preloading return values and/or by counting or asserting in method calls.


I agree with Frank, in fact I'd go much further - mock objects have almost nothing to do with test data. Data may get stuffed into one at times, and pulled out at times, but mock objects are all about beneficially reducing the complexity of a test. Think about it... changing data won't change code complexity, at most it will just cause different paths to be exercised. Mock objects reduce the amount of test data you need to get good test coverage on a localized section of functional code.
Without mock objects you need to put more effort into creating test data and placing it in all the right locations in your environment just to support the integration complexities of your system. You get rid of a lot of that grief with properly chosen mock objects. That way you are more likely to create enough tests to get good coverage. When it is hard to write and maintain tests, you don't write very many and you don't maintain them. That doesn't sound like "test data" to me. :-)
[ January 02, 2004: Message edited by: Reid M. Pinchback ]
 
HS Thomas
Ranch Hand
Posts: 3404
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Just testing. Thanks Frank and Reid. I've also been used to writing tests and creating Data scripts for Testing Teams before the applications have been written. I can assume creating Mock Objects can be re-used at this level too and filtered down to future development teams - those that come on board later so that they don't have to re-invent the wheel as it were ?
OK I am asking do Test Teams (including User Acceptance) use Mock Objects as a replacement for data scripts ?
[ January 03, 2004: Message edited by: HS Thomas ]
 
author
Posts: 11962
5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by HS Thomas:
I am asking do Test Teams (including User Acceptance) use Mock Objects as a replacement for data scripts ?

I would say no. Mock Objects is a technique for low-level testing -- the kind that unit tests generally are. I would assume that this Test Team would be more involved in functional testing (acceptance testing, integration testing, system testing, or whatever the project vocabulary calls it), right?
 
HS Thomas
Ranch Hand
Posts: 3404
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Lasse Koskela:
I would assume that this Test Team would be more involved in functional testing (acceptance testing, integration testing, system testing, or whatever the project vocabulary calls it), right?


That's right. Previously unless we were running / testing against the same data scripts communication of defects was extremely difficult.The Test teams would use the system to generate behaviour to change that Test data.
So creating Mock Objects is really a Design Tool in development(like the data scripts were/are) but not when designing Tests in testing.....
 
Reid M. Pinchback
Ranch Hand
Posts: 775
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Yup, that sounds right to me. You want repeatable testing, and while Mock Objects help you get that, as Lasse said mocks are focused at a lower level than you probably want.
Having repeatable integration tests is a challenge. Sometimes you can achieve it with a bunch of scripts to help cleanup the environment of the integrated system, but that can be pain when dealing with databases, directory services, queuing systems, and particularly external data service feeds not under your control. Sometimes you can look at creating simulators to help with repeatability, but they take more elbow-grease to create than mocks because they need a lot more in the way of realistic behaviour and memory.
 
HS Thomas
Ranch Hand
Posts: 3404
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Reid. I am sure someone will come up with something innovative soon re: repeated Integration Testing.
 
Ranch Hand
Posts: 42
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If you have "admin" type applications that allow you to put an application into the state desired, you can include doing the admin tasks in your test case.
We do that and we also created a few QA SQL packages in our database that we add functions to as needed. Basically, we call these packages to simulate the state we need. We then automate these package calls and call them in our automation. So far, we have been able to do some very complex tests that we weren't able to do manually before (because it was just too tedious and took too long).
We use Jameleon for our testing tool. Jameleon enables us to to create a simple tag that does complex things and then simply place that tag in the tests we want. The whole idea of Jameleon is to be able to break tasks up into independent actions (making simple, clean and easy to maintain code), and then organizing the functions to create a test case. Jameleon also auto-generates our test case docs based on the test scripts so we able to see at a high level where and why the test failed or even what the tests that passed are really doing. And when we change our script our test case docs change.
 
HS Thomas
Ranch Hand
Posts: 3404
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Jamelon looks . What products are there similar to Jamelon ?
 
author
Posts: 14112
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I am not sure *how* similar, as I haven't taken a closer look at Jameleon, but we use http://fitnesse.org. Basically it's a wiki where you describe you test cases, putting the test data into tables. You can then execute the test(s) by the inbuild Fit testrunner.
 
HS Thomas
Ranch Hand
Posts: 3404
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Illja .
Would this work in a non XP environment ?
 
Ilja Preuss
author
Posts: 14112
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by HS Thomas:
Would this work in a non XP environment ?


Of course - what should prevent it from doing so?
I am currently not working in an XP environment, though we get more agile month by month...
 
Christian Hargraves
Ranch Hand
Posts: 42
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
HS Thomas,
I don't believe there is anything out there that is similar to Jameleon. Using something like FIT is much easier to get started in, but Jameleon attempts to make things more reusable and more immune to changes to the application. It also attempts to solve the documentation problems that come with automating tests.
I am planning on writing a white paper about the ideas behind Jameleon. But I keep spending my time adding more features to Jameleon so I haven't been able to document it as much as I would like to (especially the part on debugging strategies).
 
Ilja Preuss
author
Posts: 14112
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Christian Hargraves:
Using something like FIT is much easier to get started in, but Jameleon attempts to make things more reusable and more immune to changes to the application.


I am still not convinced by this. In FIT, the fixtures present a quite valuable abstraction from the interface of the application, so I'd think that it could be made as immune to changes. And reusability can be achieved by using reusable pages in FitNesse, so it seems to me?

It also attempts to solve the documentation problems that come with automating tests.


On the other hand, the documentation effect of FitNesse might suffice for many projects.

I am planning on writing a white paper about the ideas behind Jameleon.


I would really like to read that paper...
 
Christian Hargraves
Ranch Hand
Posts: 42
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

I am still not convinced by this. In FIT, the fixtures present a quite valuable abstraction from the interface of the application, so I'd think that it could be made as immune to changes. And reusability can be achieved by using reusable pages in FitNesse, so it seems to me?


You're right, you could make your tests more immune to changes in almost any framework. That's a programming practice and the person writing the tests must adhere to this practice to keep their tests as resuable as possible.
Jameleon, however, was designed with the idea that not everyone thinks about keeping their tests resuable. Jameleon helps foster the practice to those that aren't as familiar with OOP (such as testers) as most developers are.

On the other hand, the documentation effect of FitNesse might suffice for many projects.


This is a very nice feature in FitNesse. I believe keeping your test case docs and your automated scripts together in the same place is crucial in keeping test automation central to your testing efforts. Without good docs, it's so easy to forget what exactly the test is doing.
However, I don't see how this solution compares to how Jameleon attempts to solve the problem of the docs becoming out of sync with the scripts.

I would really like to read that paper...


I can tell ...
 
Don't get me started about those stupid light bulbs.
reply
    Bookmark Topic Watch Topic
  • New Topic