*
The moose likes Testing and the fly likes randomness in test data Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Engineering » Testing
Bookmark "randomness in test data" Watch "randomness in test data" New topic
Author

randomness in test data

manuel aldana
Ranch Hand

Joined: Dec 29, 2005
Posts: 308
hi,

i am having a unit test, which uses a list of objects as test data.

i am testing that this list forces a method to be called a certain amount of times depending on the list's size and objects type inside the list. i am using a mocks to isolate my class.

because i want to have random objects in this list i am calling a build method which generates a certain number of fixed objects whose type is depending on a int return value of a random function.

now my question:
is it alright if some randomness is included in my test data or should it be avoided at any time?

pros:
-i use random (in my case realistic) test data.
cons:
-a failing test is very difficult (or even impossible) to reproduce.
-unit tests shouldn't be dynamic, running them once or thousand times must show the same results, which is not 100% sure if test data can change during different test runs

of course i would use only random data in very rare cicumstances! but i thought it my case useful, because it is very time consuming and annoying to generate a good mixed and big list in my test suite.

maybe the solution to my problem could be a little code generater and copy/paste it into my tests...

thanks for your thoughts.


aldana software engineering blog & .more
Frank Carver
Sheriff

Joined: Jan 07, 1999
Posts: 6920
maybe the solution to my problem could be a little code generater and copy/paste it into my tests...

That's probably what I would do in this case.

Before that, though, I would investigate to see if I could repeatably "seed" the random number generator to produce the same pseudo-random sequence each time. If necessary, you could then fairly easily repeat the test for different seeds.


Read about me at frankcarver.me ~ Raspberry Alpha Omega ~ Frank's Punchbarrel Blog
Jeanne Boyarsky
author & internet detective
Marshal

Joined: May 26, 2003
Posts: 31076
    
233

Manuel,
For unit tests, I prefer to use the same data each time as it results in repeatable tests. For integration tests, I don't mind some randomness.


[Blog] [JavaRanch FAQ] [How To Ask Questions The Smart Way] [Book Promos]
Blogging on Certs: SCEA Part 1, Part 2 & 3, Core Spring 3, OCAJP, OCPJP beta, TOGAF part 1 and part 2
manuel aldana
Ranch Hand

Joined: Dec 29, 2005
Posts: 308
thanks, i got your points.
Jack Lee
Ranch Hand

Joined: Jun 06, 2006
Posts: 38
manuel aldana,

My suggestion is when you have no choice but use randomness in your unit testing, you can write extra code to log the list information when a failure occurs, which may make it easy to reproduce the failure.


SCJP 5.0<br />SCWCD 1.4
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: randomness in test data