• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

unit testing without brittle code

 
Trailboss
Posts: 23773
IntelliJ IDE Firefox Browser Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
It seems to me that some people will write test first code making the class that they
intend to test have a lot of "package private" attributes and/or methods that should
really be private. They do this exposing for the convenience of writing the unit test.
When the time comes to add something to the class, or do some refactoring, you're doing a
lot of research trying to figure out who all is using what - what is safe to change? The
code is more difficult to change because of all the tests looking at the innards of the
class. Some folks will keep everything private, but will use reflection methods to dig
that private stuff out to take a look at it. So if you refactor something, you find that
you have just broken a pile of tests that should still work fine because the public
methods all still behave the same way they always behaved.
Sometimes I feel the need to write a test for some code buried in my class. I don't want
to use the above techniques, so I compose a mountain of mock classes and a detailed
environment . I call a public method and pick apart a mountain of output looking for my
one little detail that shows that deep down algorithm did its stuff. This seems to be
about as fragile as the invasive methods and takes about twenty times longer to
implement.
I'm no expert, but there are conclusions that my gut has reached:
  • Making stuff not private for the sake of unit testing is bad
  • Using reflection to get at private stuff in a unit test is bad
  • Writing a mountain of pre and post stuff for a simple unit test is bad
    Clearly, the solution will be some kind of invasiveness. I'm thinking the goals are:
  • be able to write a simple test
  • the test will be included in the regression suite
  • minimize invasiveness
  • make any invasiveness obvious and clear that it is only for testing purposes


  • I have some ideas and I'm concerned I haven't thought them all the way through...
    Suppose I have a class called Spanky that has a lot of private stuff. And then
    TestSpanky that extends the junit TestCase class.
    How about putting a method in Spanky: void unitTest(TestSpanky test) ??
    I could call the method from TestSpanky passing in "this". Within the method, I could do
    things like "test.assertEquals(27, myPrivateMethod(5533))"
    My private stuff is still private. If I want to refactor the class, the tests are easier
    to understand, and it is clear that no other classes in the package are using my private
    stuff.
    This is just an idea - does it suck?
     
    Ranch Hand
    Posts: 775
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    More and more I've come to the conclusion that a unit test should only test against interfaces. In a certain sense you don't care how something is implemented, only if it delivers the functionality you want. The public interface is a declaration of functionality.
    That said, it is still nice to know if fragments of the implementation are doing what you expected. For those I think assertions, class invariant checks, and design-by-contract mechanisms are probably more suitable than (e.g.) JUnit tests.
    I've really only found one exception to this approach so far, and I think even it could be resolved if I spent some time to tackle it properly. If you want to create unit tests for a distributed application with simulated network problems, I've found I do want tests at a level other than at the public API, and I've gone for changing things to default access. The exposure is pretty limited (unlikely that other parts of an app will depend on the internals of a proxy on one side of an RMI connection.
    Regression tests are a bit of an open question, haven't really thought a lot about that. I suppose you could argue that a bug really only mattered if it broke something apparent at the public interface level, and use the approach described above.
     
    Sheriff
    Posts: 7001
    6
    Eclipse IDE Python C++ Debian Java Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Clearly, the solution will be some kind of invasiveness
    I'm not sure about that. One of the things you might be missing is that needing to test the internals of a class or method is often a "code smell" which indicates that it ought to be broken up a bit more. If those intermediate steps or values are important enough to be tested, then they could well be important enough to be used on their own - why are you limiting the flexibility of the class by lumping them together?
    I'd look hard at refactoring the class to make simple testing easier rather than building elaborate test frameworks. I'd certainly avoid putting any test code in the class under test. In a well-tested system there can easily be more test code than real code, and tangling them up is a big signpost pointing to code bloat.
    This is just an idea - does it suck?
    I'm afraid I think it does. I know asking for specifics can be a cheap shot in a general discussion, but if you can find an "untestable" example for us, maybe we could work through it.
     
    author
    Posts: 14112
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    I sometimes use a pattern I call Test Probe to test the innards of a class. It's simply an inner class providing access to otherwise private features.

    So I can access bar in a test by using foo.new TestProbe().getBar().
    I try to use it as seldom as possible and most often in the early phases of a new class, when I don't see in which direction it is evolving. Later I try to remove it, most often by breaking up the class the way Frank suggested.
    I like this pattern because its rather obvious from the code that it is a dirty hack for testing purposes. It's ugly, but certainly better than not writing a test...
     
    Frank Carver
    Sheriff
    Posts: 7001
    6
    Eclipse IDE Python C++ Debian Java Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Ilja wrote: It's ugly, but certainly better than not writing a test...
    I'm not entirely sure about that either. I really don't like delivering test code in a deployed system, I'd sooner make something public, or make it "package" scope and put a test in the same package, than put test code in delivered system.
    Give that by your own admission, this is only used when the design of the class is still very unstable, I'd accept the hit of needing to alter the tests with the code in preference to the worry of remembering to remove test-specific code after the design stabilises (whenever that might be).
     
    Ilja Preuss
    author
    Posts: 14112
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Originally posted by Frank Carver:
    Ilja wrote: It's ugly, but certainly better than not writing a test...
    I'm not entirely sure about that either. I really don't like delivering test code in a deployed system, I'd sooner make something public, or make it "package" scope and put a test in the same package, than put test code in delivered system.
    Give that by your own admission, this is only used when the design of the class is still very unstable, I'd accept the hit of needing to alter the tests with the code in preference to the worry of remembering to remove test-specific code after the design stabilises (whenever that might be).


    Remember that inner classes get compiled into their own class files. So by deleting all *$TestProbe.class files, you can remove all the test-specific code from the production system!
    OTOH, I really do like to include the tests in the distribution. That way if the system doesn't behaves like expected, you can easily run the testsuite on the production system. That can make it much easier to analyze environment-specific problems.
    For example, we recently discovered that one of our tests provokes an endless recursion on some JDKs (not the one we generally used for testing). That would have been much harder to find out without the tests.
     
    Ranch Hand
    Posts: 80
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Have you ever tried AspectJ?
    I haven't, but from some articles it seems it is perfect for your test needs.
     
    Ilja Preuss
    author
    Posts: 14112
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Originally posted by Eduard Jodas:
    Have you ever tried AspectJ?
    I haven't, but from some articles it seems it is perfect for your test needs.


    Mhh, how do you think it should be applied to testing?
     
    Eduard Jodas
    Ranch Hand
    Posts: 80
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    This is some of what I understand from what I have read about AspectJ:
    1. It has an event-like mechanism that notifies you about method calls, field access/assignment, exceptions, etc. So you can know when a private field, not accessible from a test case, is changed. The same with private methods
    2. It lets you define pre and post conditions
    3. It lets you change a class, for example you can add methods to access private fields or to check the state of the private fields of a class. You can also add code before/after a method.
    4. All this is done in separate files, so there is no need to modify the source code. You just need a development compilation and a production compilation -> clearer/cleaner source code and no performance decrease
    Now I am going to read this article:
    Aspectj and Testing
     
    paul wheaton
    Trailboss
    Posts: 23773
    IntelliJ IDE Firefox Browser Java
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    I have searched far and wide .... into the mountains and valleys .... seeking truth .... and.... now I'm boring myself.
    I came to a conclusion that puts a lot of this stuff into a comfort zone for me.
    I have verified that exposing your private stuff does violate encapsulation and make your code more brittle (less agile).
    BUT! Suppose you sacrifice 15% in agility only to gain 300% in testability. And along the way, your tests become 50% more agile? Would this be a worthwhile sacrifice?
    I think so.
    When you do test first design, you often make lots of small little classes that have lots of things exposed that you probably would not expose if you did the testing last. In fact, if you threw out all of the tests, you would probably make a lot of those classes private inner classes to some other class because there is only one class using them.
    I think a step I would like to see next is something discussing agility in code. After all, exposing otherwise private data and with the wrong kind of unit tests, you can make code very brittle (difficult to change). It seems we need better understanding of how to have good testing *and* have agile code.
     
    Ilja Preuss
    author
    Posts: 14112
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Paul, I think you are on the right way...

    I think a step I would like to see next is something discussing agility in code. After all, exposing otherwise private data and with the wrong kind of unit tests, you can make code very brittle (difficult to change). It seems we need better understanding of how to have good testing *and* have agile code.


    Yes, discussing that would certainly be worthwhile. A good way to start such a discussion would probably be to show us an example of how exposing otherwise private data made your code more brittle...
     
    Frank Carver
    Sheriff
    Posts: 7001
    6
    Eclipse IDE Python C++ Debian Java Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    One practice which can help cut through this dilemma, is to keep all member variables strictly private, write methods which perform some active and useful function, and make (nearly?) all your methods public.
    I despair sometimes at the proliferation of "JavaBeans" classes. If you habitually write all those getters and setters, you are only a small step from making the values public, which is itself only a small step from the dreaded global variables. JavaBeans seems to positively encourage thinking of objects as groups of data items, rather than groups of behaviour.
     
    Bartender
    Posts: 612
    7
    Mac OS X Python
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    For the internal tests like you have described, I
    set all "private" methods to default. I then
    create a TestClassA class that performs the wanted
    tests against ClassA.
    My build creates two jars: one with all code and
    one without Test* code. I then run two test sessions,
    the internals tests which runs against TestPackage.jar
    and the unit and transation tests which runs
    against Package.jar.
    -steve
     
    Ilja Preuss
    author
    Posts: 14112
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Originally posted by Steve Fahlbusch:
    For the internal tests like you have described, I
    set all "private" methods to default. I then
    create a TestClassA class that performs the wanted
    tests against ClassA.


    Steve, could you post an example of such an "internal test"? It could be very interesting to discuss wether there would be a reasonable way to "publicize" that tested behaviour. Thanks!
     
    Ranch Hand
    Posts: 1874
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    nice to hear from you steve after such a long time.
    I second Ilja's opinion. will it be possible for you to discuss the things in the forum ?
     
    Don't get me started about those stupid light bulbs.
    reply
      Bookmark Topic Watch Topic
    • New Topic