wood burning stoves*
The moose likes Testing and the fly likes randomness in test data Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Murach's Java Servlets and JSP this week in the Servlets forum!
JavaRanch » Java Forums » Engineering » Testing
Bookmark "randomness in test data" Watch "randomness in test data" New topic
Author

randomness in test data

manuel aldana
Ranch Hand

Joined: Dec 29, 2005
Posts: 308
hi,

i am having a unit test, which uses a list of objects as test data.

i am testing that this list forces a method to be called a certain amount of times depending on the list's size and objects type inside the list. i am using a mocks to isolate my class.

because i want to have random objects in this list i am calling a build method which generates a certain number of fixed objects whose type is depending on a int return value of a random function.

now my question:
is it alright if some randomness is included in my test data or should it be avoided at any time?

pros:
-i use random (in my case realistic) test data.
cons:
-a failing test is very difficult (or even impossible) to reproduce.
-unit tests shouldn't be dynamic, running them once or thousand times must show the same results, which is not 100% sure if test data can change during different test runs

of course i would use only random data in very rare cicumstances! but i thought it my case useful, because it is very time consuming and annoying to generate a good mixed and big list in my test suite.

maybe the solution to my problem could be a little code generater and copy/paste it into my tests...

thanks for your thoughts.


aldana software engineering blog & .more
Frank Carver
Sheriff

Joined: Jan 07, 1999
Posts: 6920
maybe the solution to my problem could be a little code generater and copy/paste it into my tests...

That's probably what I would do in this case.

Before that, though, I would investigate to see if I could repeatably "seed" the random number generator to produce the same pseudo-random sequence each time. If necessary, you could then fairly easily repeat the test for different seeds.


Read about me at frankcarver.me ~ Raspberry Alpha Omega ~ Frank's Punchbarrel Blog
Jeanne Boyarsky
internet detective
Marshal

Joined: May 26, 2003
Posts: 30076
    
149

Manuel,
For unit tests, I prefer to use the same data each time as it results in repeatable tests. For integration tests, I don't mind some randomness.


[Blog] [JavaRanch FAQ] [How To Ask Questions The Smart Way] [Book Promos]
Blogging on Certs: SCEA Part 1, Part 2 & 3, Core Spring 3, OCAJP, OCPJP beta, TOGAF part 1 and part 2
manuel aldana
Ranch Hand

Joined: Dec 29, 2005
Posts: 308
thanks, i got your points.
Jack Lee
Ranch Hand

Joined: Jun 06, 2006
Posts: 38
manuel aldana,

My suggestion is when you have no choice but use randomness in your unit testing, you can write extra code to log the list information when a failure occurs, which may make it easy to reproduce the failure.


SCJP 5.0<br />SCWCD 1.4
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: randomness in test data
 
Similar Threads
Refactoring Advice
how to convince people to unit test?
Non J2EE transactional management in Java
How to test more difficult input data like images?
Should unit testing be given up?