I would say that differently.
There are many tests that don't treat the classes/methods under test (CUT/MUT) as black boxes but rather white boxes. That is, the tests have intimate knowledge about details of the CUT's implementation. For example, when you use mock objects to simulate what the CUT's collaborators would be called upon to do, then you are writing a white box test. These are not necessarily unsuitable tests. I write white box tests as I develop the functionality inside my classes incrementally while doing TDD.
White box tests are, however,
more brittle since any change to the implementation of the CUT has the potential to cause a corresponding change in the test code. Black box tests are at a higher level of abstraction than white box tests and are more resilient to implementation detail changes in the CUT. Black box tests specify the API of the CUT.
In short, white box tests become detailed implementation design specs that should give you an idea of how the CUT is implemented. Black box tests are detailed API specs that give you an idea of how the CUT API is supposed to be used and how it behaves as far as client code is concerned.
You should keep white box and black box tests separate. Ideally, you would keep them in separate test classes, e.g. MyClassTest (API specs, black box tests) and MyClassImplTest (implementation specs, white box tests)
For example, a list of JUnit test cases for ArrayList might look like this:
These are just a "for instance" - I don't know if ArrayList actually has these unit tests.
That said, it's conceivable that you can write a white box test that specifies that a certain type of exception should be handled inside the method and not be thrown by the CUT. If you're using JUnit 4.x, you should be able to use an
@Rule with the
ExpectedException class, an example for which is given in the JUnit API documentation here:
http://junit.org/javadoc/latest/org/junit/rules/ExpectedException.html
One case where it might be appropriate to test that an Exception is not thrown is when the CUT is a subclass that overrides some behavior of its superclass. For example, one of the methods that
ArrayList overrides is the
AbstractList.add().
AbstractList.add() throws an
UnsupportedOperationException whereas
ArrayList.add() does not. It's conceivable that you would write a test to explicitly document this behavior. I would still classify this as a black box test though since it has to do with what a client may or may not expect to see when invoking the ArrayList API. That is, the test code only shows usage of the API and no intimate knowledge of what actually goes on inside the add method.
Again, refer to the JUnit API documentation for ExpectedException for an example.