This week's book giveaway is in the Testing forum. We're giving away four copies of Data Structures the Fun Way: An Amusing Adventure with Coffee-Filled Examples and have Jeremy Kubica on-line! See this thread for details.
I want to split all my JUnits up between "must run" and "nice to run" tests.
The reason being that with over 3000 tests our build process is now taking way too long and sometimes it is a pain to even run them. Now getting very selective about what new code gets tested at all.
Solution would be, when ime allows run all tests ( Eg nightly with no developers around ) and when I want to smoke test the code base during the day I could run more corse grained tests.
So is it possible for JUnit to be setup in such a away as to run diff tests at different times?
If not is there any programatic way to achieve this?
Bottom line is we're in the real world and don't have time to be running Unit tests against setters and getters that will almost never change but I don't want a single method in our code base that goes untested, so rock, hard place comes to mind.
The situation, you described, when there is not time to execute all unit tests, indicates that there is a bigger problem with your project.
Since all build is made in one go, I presume, the project is not modularised properly.
So the only long-term solution IMO, should be refactoring: dividing it into small modules, built independently.
At the beginning you can try identifying the code which changes rarely. For example any utilities you created for your project, like XML, file, sql or JNDI operations.
Since you made separate module including such utilities, your main project will be smaller and faster to build and test.
Module containing utilities will be built much rarely, only when necessary. The main project will use it as compiled jar stored in repository.
You can also modularise your application by business domains, i.e. each module contains functionality for different domain, like: accounting, sales and so one.
This way, since you finish work on one functional module, it is no longer changed and tested. You build and test only the part of your application which is currently developed.
It will fix the problem of long build including long unit testing. Maybe it is not easy solution, but will give you many more advantages, like easier maintenance, higher quality and so on.
Thanks for reply.
The project is modularized and well sturctured. The reason there are so many unit tests is that every concievable permutation of input for every method is tested. Even if a method has a public contract that requires a non null value we still test it with a null value and verify the correct exceptions are thrown.
So with 20 classes with around 10 public methods each and most methods having an average of 10 - 15 tests that is where we get the number of tests that we have.
I would say that there is only maybe 1 - 2 of these tests that really need to be run each time but as in OP we don't want to remove any of the others.
I did come across a means by where the TestCaseRunner is extended and a filter mechanism added but this is a bit hackish for my liking
We use classpath suite runners all the time to set up different ways of running tests - one project, all unit tests, all fast running unit tests (we have a few tests that we call unit tests but do work with jdepends and run really slow) and database tests.