This week's book giveaway is in the Agile and Other Processes forum. We're giving away four copies of Darcy DeClute's Scrum Master Certification Guide: The Definitive Resource for Passing the CSM and PSM Exams and have Darcy DeClute on-line! See this thread for details.
Lately I have been assisting developers on systems with close to 0% unit test coverage. These systems are big and they are either in production or production ready. But the client woke up late and are ask the team to increase unit test coverage. I am not too convinced about the the effectiveness of the tool like JTest or AppPerfect, even if the client is willing to buy one. I have seen AppPerfect test code it seem simple but there seems to be some overhead for maintain them (without the help of the tool). My question is, Did any of you had experience using any of the automated unit test creation tool? How effective and easy are the test case generated? How dificult is it to port to JUnit and execute them using just JUnit libraries (In case of AppPerfect the hack was easy) and what is the maintenance overhead? Are there any opensource tool that is capable of doing that? (I saw the ones listed in JUnit.org, all those projects are frozen for a while).
If any of you had similar experience and had written unit test manually to increase the coverage, how did you go by doing that? (set of steps to achieve that in shorter duration)
Ok I have asked too many questions, I guess. [ April 25, 2006: Message edited by: Vinod John ]
In general, I'm not a believer in generated unit tests but you might want to evaluate whether Agitator would be of use in your situation.
Personally, I'd start by figuring out the client's true motivation for increasing unit test coverage. Getting the client tell you why he wants higher test coverage might help in focusing the available resources to where would produce the most assistance regarding the stated need.
Once upon a time, I had a week-long consulting gig at a client. It was supposed to be about coaching their developers in TDD and unit testing techniques. On the first day, it turned out the client's project manager (who wasn't the one with whom the gig was agreed--and also not the one paying my fee) just wanted me to come and write some unit tests for them and that "they'll pick up from there, mimicking how I've written unit tests".
The gig lasted that one day. There was no point in me coming in to write some unit tests for their existing codebase, raising test coverage from a non-significant percentage to a less non-significant percentage, when none of the client's developers will actually learn from that experience.
The teaching behind the story is that the client's project manager wanted a different thing than the party who called me up and made the gig happen. The project manager's interest was to raise test coverage towards a number that was required of him, not to improve the product's quality and maintainability.
Vinod, JTest throws arguments at your code to try and break it (passing in nulls and the like.) A human is still needed to write unit tests as you know the expected value.
I agree with Lasse that a tool cannot automatically solve the testing problem. Even if you get some coverage with JTest, the most it can do is look for regression changes if you don't tell it the proper values. Of course, when the tests fail, is it because the functionality changed or a defect occured. You never really know if you didn't set up expected values in the first place.
Does the client really want unit test coverage in the true sense of the word? Maybe they really want end to end (like GUI) tests to increase coverage. If not, the best thing is to slowly add tests over time.
just wanted me to come and write some unit tests for them and that "they'll pick up from there, mimicking how I've written unit tests".
Lasse, this was the exact situation I am was in, but with one difference I work for a consulting firm. After explain the manager "Why it is not possible in a short duration?", the reply I got was "you may be correct, but we promised the client we would have x% unit test coverage?" So I figured out there is not point in explaining. At times I hate those "coverage reports", a most people (in management) tend to think these are real and not realize the background.
Originally posted by Jeanne Boyarsky: Vinod, JTest throws arguments at your code to try and break it (passing in nulls and the like.)
I think Agitator does the same thing. I was curious, though not convinced, on the usefulness of these tools over a period on time. I thought I would ask you guys.
Does the client really want unit test coverage in the true sense of the word?
This is the second time in 2 years I in this kind of situation, In the first (I was part of the client team) it was more to please the upper management. Knowledge level, I would say most of them (even large number of developers) were arguing the use of unit tests. For them it was another overhead. Now, I think, this is more of a "Contractual thing", our company might have promised them the deliver code with unit test. Quality wise I suspect the do care, they just want good coverage report and the test should be using some open source tool (they don't want to pay). I am just planning to suggest them something that is probably unethical now?
In most of the companies I worked, Unit Testing is mandatory. In my previous contract I have written some Base classes and some JUNITS for other developers to follow the same Procedures for writing JUNIT tests for the code they developed. From my experince I found that the developers learned lot about JUNIT tests when they followed my sample JUNIT tests instead of mentoring them for long hours.This way, in very short time, we are able to have 85% of test coverage.
I think he's gonna try to grab my monkey. Do we have a monkey outfit for this tiny ad?
a bit of art, as a gift, that will fit in a stocking