I am using TDD to write J2EE code using spring, strutstestcase, hibernate and so on, and I realize that writing the tests takes too much time (junit configuration, mock creation, write the test, configure database,..)
I realize that tests provide quality to the code, but it is really agile? What I am doing wrong?
Years ago, I had an architect over our team that was pretty horrible. But she said one thing to me that I've never forgotten.
Me: "I don't have time to write all these tests"
Her: "You don't have time not to write them"
Writing good tests takes practice and, often times, more time than they appear to be worth. But appearances can be deceiving. You may not be doing anything wrong at all. I could write a small book for every time a test has saved my butt. I could write a similar book for every time not writing a test has come back to bite me in the butt.
If you want to get more specific about your environment and how you're writing your tests, maybe we can help you reduce the pain a bit.
Well, it's hard to say if you are doing wrong without seeing your work. However, you are right. Once you feel that you've spent too much time writing tests, then you should review your work. Here are few tips:
(1) How does your code adhere to good programing principles and practices like separation of concern, single responsibility, open-closed principles etc?
(2) Are you writing unit tests or functional tests every time?
(3) Do your classes have too many dependencies that you have to mock too many of those?
(4) How often do you need to use persistent storage (i.e. database/file systems) to run your tests? (should be as few as possible, none is the best).
Well, right now I can only think of these four. Let me know if it's useful for you anyway.
“Everything should be as simple as it is, but not simpler.” Albert Einstein
Joined: Oct 15, 2010
well, I am writing tests for an already not-TDD-written J2EE web application (jboss applications are horribly to test because of jdni). I have written tests for each layer: dao (junit+spring), ejbs(junit+spring), action (strutstestcase+spring) and web (selenium). I have dozens of integration tests in each layer (dao, dao+ejbs, dao+ejbs+action) and often some tests get obsolete. Modifying the dao layer means writing new tests for each of the 4 layers. Methods tipically have 4 or 5 dependencies (having dependencies IMO is a good sign of low coupling). Writing dao tests without persistent storage for me is not an option, as dao beans already have hibernate queries.
And sometimes I simply don't realize of use cases which I have not considered and so haven't been tested.
I definitively like the TDD approach, but as I said modifying a few lines of code in a layer it may mean rewriting tests in each of the layers above, which takes time (I may spent sometimes 1 hour to write a test because of unexpected surprises (learning to mock static methods calls)
I have read that unit tests should be written in 2 minues. But how much time should an integration test take?
I guess that to become an expert in writing tests efficiently it takes time and expertise.
Joined: Nov 20, 2008
I understand that if you are not looking for a theory and/or full list of good practices on how to write tests. A forum would not be enough for those. So I just stick to your case then.
In your list of technologies, there is no mention of any mocking framework. Are you using one? (you should)
It doesn't seem you writing unit tests but functional tests, hence the issues of changing code in on layer involving changing tests in other layers. Such scenarios should only occur if you change the API between layers (i.e. refactoring signatures of interface methods and/or add/remove interface methods and/or changing attributes of transfer objects etc).
The main idea of writing unit test is that you test only and just only that section of the code that you are about to change without worrying about changing of its dependencies. All dependencies should be controllable i.e. created/managed with some mocking framework. For example, if method mymethod() of class A depending on class B, C, D, E, you should write test on mymethod() of class A with mocked objects with controllable states of B,C,D,E because you are not testing B,C,D,E. Which also forces you to think of what do you want that code section to accomplish instead of what if the dependencies change their behaviors.
It's also important to write testable code: i.e. private methods are generally not testable; methods which depend directly on class attributes are generally hard to test; methods which uses direct connections to persistent storage (i.e. file, jdbc) are generally hard to test. Also, if dependencies are automatically injected into your test object (i.e. via JNDI, Annotations, etc), you may want to consider to create some protected set methods to inject the mocked dependencies to make it testable.
Now, it may be hard to follow with some code sample. How about, you post up here some code that you find it hard to test and your current approaches to test it. Let's see how we can improve them.
Joined: Oct 15, 2010
Thanks for your suggestions Duc,
Yes I have tried mockito, powermorck and easymock for unit testing.
But I realise that, as you noticed, I am doing too much funcional testing and less unit testing. I am trying now to invert the efforts.
Also, I have reorganized and added an script to run the tests automatically, as until now I was launching the test cases by hand.
I guess it's harder to test an existing application (which has not been designed to be tested at all) than start an application from scratch using tests.
I can't give you an specific code that I can't test, I achieve it at the end. I guess is a question of learning to mock specific situations (static methods, void methods, threads) and, at the end, my own speed/capacity to organize and write test code.
I've been programming Java since 1997. I don't like formal testing, for my types of projects, and here's why. I did it once with a team of four developers and a large code base (roughly 250,000 lines). For us, it was a waste of time. Almost six months to start from scratch and build a complete testing solution using junit, etc. for our existing code. We found one bug during the initial test writing, and another bug later on. Yes, it would have gone faster if we were testing experts and had written tests from the very beginning, but the moral of my story is that a formal testing framework is not the only way to produce good code.
The reason why so few bugs were found is because our normal routines prevented bugs. We generally tested any new code or changes we made by feeding test data into our system and verifying results manually. Since the code was fresh in our head, we had a good idea on how to stress the code in question. The occasional missed condition would quickly be caught unexpectedly during some other test… each end-to-end test would touch so many components that we were effectively doing lots or unintentional testing. (One slight advantage of doing it this way is that the test conditions are more random.)
There are lots of arguments for automated testing, but I don't believe saving time or superior product are always valid reasons, especially when compared to good manual testing for some projects.
I found out later that I was not alone this belief. I read a book that described the early programming world. There were two grand experiments… in the U.S. people were moving towards formal unit testing. In Japan (who was more of a powerhouse programming nation then), they were using pragmatic end-to-end manual testing of their systems. The result?… Both programming methodologies produced the same quality and time-to-completion. (The reasons for Japan falling out of the software race were explained by the author... they had nothing to do with their testing methodology.)
So my advice: Do what you think feels right for your project. I'm sure very rigid projects will fit nicely into a junit kind of approach, but more fluid projects may benefit from a hands-on simulation of use instead.
Remember, in the end you are judged on how well your code works today, and tomorrow. No customer really cares how you achieve that.
There's two benefits to having an automated test suite. One is quality/time to completion. The other is regression. I agree - for some projects - aka the kind that are quick dirty and will be in production only a few months or so - testing has less value. However, for longer lived projects, quality and time to completion for later releases need to be measured if trying to say testing doesn't have value for that project.
Another alternative for testing that I don't think I've seen mentioned on this thread are automated system tests. Since the application you are testing is a web application, you should be able to use a tool like selenium to build a nice suite of tests at the application layer.
Selenium has a firefox plugin that allows you to record user actions (ie. login, click button a, type into box b, etc). Once those actions are recorded, you can save those recordings as junit tests that you can setup to run nightly or hourly, etc.
I have used this technique when given a large legacy code base that had no real tests, and was architected in such a way as it was very difficult to test (for all of the reasons that you cite above - tight coupling, numerous dependencies, etc)
The framework works with ajax-enabled interfaces in addition to standard interfaces. It took a minimal amount of modification to the recorded scripts to get them to be re-runnable, and the benefit was tremendous. In my case we ran a suite of read-only actions in production as part of our install procedure to verify our install was correct. (Not what I would recommend, but I wasn't the boss) If you have a test environment available, then read and write actions could be run in that environment at whatever frequency is good for your situation.
I did this over two years ago, so the feature set of selenium could be even richer now.
Well, it's a question of whether the developer must invest time yourself this or the specialized testers... Automated tests are really necessary if you need to know how your application reacts under repeated execution of the same operations.
that is a story pretty close to my heart: I spend half a year as "embedded QA", which means I was the developer writing and extending the testing framework for an already existing software and I faced the same problem. I got that job because, well, I was the TDD guy (and nowaddays I still am, just as a normal developer).
Here is what I came to believe after doing this job:
TDD does not work well when extending systems that were developed without this methodology. Because usually, those are very closely coupled and, well, ugly (this is a technical term describing something more ugly than close coupling, in the "you call your uncle daddy" kind of way - you know, code that looks like it is taken from an example of "don't try mocking this because half your code is in here and half my code is in your part"). In my case, even the most basic actions needed sessions or database connections or, later on, OSGI services to be present. If you start mocking stuff there, you'll get to the point where you do not know whether you are looking for bugs in your mocks or the actual system.
So I currently advocate (sounds good - not that anyone is listening) a gentle CYA approach (if you do not want to call it that, call it "separation of concerns" or simply "at least it is not my bundle's fault") for legacy systems:
Do TDD for stuff that you build anew. Build your testing framework up as far as you need to go to verify your own stuff. Spent not much more for tests than the usually recommended 50% of your time.
Do opportunistic testing for newly occuring bugs in the legacy part: Extend your own framework if possible to cover the system from bottom up. If not, don't feel ashamed to do top-down stuff from the highest layer (with Selenium in my case) and live wit the fact that while this gives you a lot of coverage pretty quickly, you'll have to search for the actual bug much longer than if you have a unit test that points to the right place.
Don't go through all layers (so tests for layer N involve layers 1 to N-1) like you mentioned. If you start wondering why a bug occurs in tests for layer N-1 but not in layer N (or even worse: the other way around), you got too much complexity in your tests. And a simple API change (or, since "API change" sounds so unprofessional, let's call it "API clarification") means changing a lot of test code.
Otherwise, as you point out, you will spent too much time on that.
I don't feel to bad about advocating this: If you manage to work on a legacy project long enough to see problems like this, it is probably not completely broken. In my case, the system is gradually getting better, new bundles are surprisingly easy to test, sometimes even developed using TDD. The old code gets improved whenever there is a bug (and time... hahaha). And the software sells well and has outpaced most of its competitors, which seems a good indicator that this mix of methodologies is not too shabby.
I also believe from my own experience that a non-trivial software project can have 90% code coverage if started with TDD and that a legacy project can be happy to go to about 50%. After that, the utility of additional tests declines rapidly.
In my humble opinion there is a little misunderstanding about what is Test Driven Development and Test Coverage. TDD is a different way of thinking development process and it goes far ahead of just having a couple of test for a class.
In TDD we write the tests as soon as we get the component's interface and it's a way to think in advance the component dependencies, what the methods algorithms will be, better ways to decouple and evaluate our code granularity. Based on our tests and our mocks expectations we build our new component knowing what it will have to adhere to, it's like a contract, except that the contract is a test. That's why TDD don't rely on integration tests, although it's a good practice to have them AFTER you build your component. Again, in my opinion that is the great difference: to use TDD we have to build tests BEFORE.
When we have a legacy system, all we can do is build or increment those components test coverage, but in most cases you are not rethink how you gonna rebuild that class, you just want to be sure it's working like it's supposed to do.
Joined: Sep 16, 2008
Jan Cumps wrote:Do you know a faster way to test?
(and to retest when needed)
Depends on what you are trying to test. Is it decoupled code? If it is, you can use mock tools like Mockito, JMock, EasyMock and so. If it's not you'll probably should use a context. If your project uses Spring, that's half the answer, you should give spring-testing a try, even if your project doesn't use this DI container. I usually work with a set of tools to create integration tests, like HSQLDB, DBUnit, Spring Testing, Struts Test Case, etc, etc, etc, all depends on project scope and what should be tested.