xUnit - assertion failures

[Jeanne Boyarsky]

I have been using shUnit (UNIX korn shell unit) in one of my classes. It runs all of the assertions in a method regardless of whether previous assertions fail. I found it interesting that all the xUnit frameworks don't behave the same way in this.

Personally, I like the way jUnit functions better in all this. Mainly, because I only care about the first error. shUnit's system encourages me to write "bad" tests that test a bunch of different things.

I'm curious which way people here prefer? Just the first assertion failing or all of them running?

[Lasse Koskela]

Originally posted by Jeanne Boyarsky:
I'm curious which way people here prefer? Just the first assertion failing or all of them running?

I'd prefer stopping on the first failing assertion because the rest of the assertions are making assumptions about the fixture's state that may or may not be true since a previous assertion failed. I can imagine that one might want to run a set of assertions regardless of whether the others failed or not, but when that happens (once in a year?), it's usually enough to just temporarily comment out the first failing assertion and re-run the test.

[Pradeep bhatt]

Why would it be useful to run an assertion when the previous one failed ? Is the second one independent of the first?

[Alexandru Popescu]

Usually assertion mechanism is used during development to mark indispensable/mandatory conditions. It is said that a true assertion marks the fact that the system reached/is going through a wrong execution path (that will make the system unusable). It will be no benefit to continue.

./pope

[Jeanne Boyarsky]

"Why would it be useful to run an assertion when the previous one failed ? Is the second one independent of the first?"
Pradeep,
In well crafted tests, it generally isn't. Sometimes during testing, I'll want to see if a later test fails to give me more insight into the problem. In that case, I do what Lasse does an uncomment it. If the assert doesn't fail, I can cram everything into one test method. This is a bad practice and I like that junit doesn't encourage it.

"It will be no benefit to continue."
Ali,
In the "real" code, this is true. In test code, sometimes assertions are independent. Although as I described above, this is often a code smell.

[Alexandru Popescu]

Wouldn't it be the smell of at least 2 tests? (i feel i'm loosing it)

./pope

[Nicholas Cheung]

I have a question on assertion. Does it really that useful for testing private method? It is useful, if we know some methods may encounter problem, but how should we know? Is that mean, I need to put down the assertion checking for each private method?

Nick

[Alexandru Popescu]

There is already a very long (but very very interesting discussion) on testing private methods.
It may answer also your question . Mines were answered .

./pope

[Jeanne Boyarsky]

Ali,
Yes, that is the code smell I was referring to.

Nick,
I want to clarify that I was talking about assert methods in the tests (to check test results) rather than assertions in the real code for parameter checking.