File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Testing and the fly likes JUnit Pocket Guide: Where to put your tests Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Engineering » Testing
Bookmark "JUnit Pocket Guide: Where to put your tests" Watch "JUnit Pocket Guide: Where to put your tests" New topic
Author

JUnit Pocket Guide: Where to put your tests

Petr Blahos
Ranch Hand

Joined: Apr 28, 2004
Posts: 131
Hi Kent and others,

I have seen different aproaches as to where the test classes
are put. I tend to put them to a completly separate directory
(like, I have my source in src/... and I put my tests in tests/...).
In some projects I saw tests directly in src (like
src/com/company/project/sub1 contained sources for project
and
src/com/company/project/sub1/tests contained tests for sub1)
And I think I even saw tests in the same directory as sources.

Does the book talk about this topic? Thanks and best regards,
Petr


Get a better web browser:<br /><a href="http://www.mozilla.org/products/firefox/switch.html" target="_blank" rel="nofollow">http://www.mozilla.org/products/firefox/switch.html</a>
Kishore Dandu
Ranch Hand

Joined: Jul 10, 2001
Posts: 1934
If you have the tests in the same directory as sources, its better to keep them in a specific naming convention(all test classes have a specific pattern that will not fall in the actual source file names)

You can compile only the sources by setting your build file(your build.xml invoked by ant) to ignore the test files if needed, for production purposes etc.


Kishore
SCJP, blog
Alexandru Popescu
Ranch Hand

Joined: Jul 12, 2004
Posts: 995
I would rather say that the package/naming organization is more important than the physical project structure.

./pope


blog - InfoQ.com
Jeanne Boyarsky
internet detective
Marshal

Joined: May 26, 2003
Posts: 30537
    
150

Petr,
I do what you are doing. However, as long as the unit tests are in the same package name as the code under test, it doesn't really matter. More of a personal preference.


[Blog] [JavaRanch FAQ] [How To Ask Questions The Smart Way] [Book Promos]
Blogging on Certs: SCEA Part 1, Part 2 & 3, Core Spring 3, OCAJP, OCPJP beta, TOGAF part 1 and part 2
Layne Lund
Ranch Hand

Joined: Dec 06, 2001
Posts: 3061
With my limited experience with JUnit, I think I prefer putting the tests in a separate directory structure. Although, you probably should keep the tests in the same package in order to test package-private methods. In other words, I like to set up two parallel directory structures, such as:

src/foo/bar
tests/foo/bar

I prefer this mostly from an organizational standpoint; the number of files in a single directory could get out of hand, otherwise.

-Layne


Java API Documentation
The Java Tutorial
Lasse Koskela
author
Sheriff

Joined: Jan 23, 2002
Posts: 11962
    
    5
I'm also putting my tests in a separate source folder but in the same Java package as the production code, i.e.

However, "JUnit Recipes" by J.B. Rainsberger, for example, recommends having the tests in a different package because that forces you to test only through the public interface.


Author of Test Driven (2007) and Effective Unit Testing (2013) [Blog] [HowToAskQuestionsOnJavaRanch]
Alexandru Popescu
Ranch Hand

Joined: Jul 12, 2004
Posts: 995
Same settings as Lasse is using.

Is Mr.Rainsberger explaining why the testing should be done only through public API? (this will give us a hint on the long thread of testing private methods ;-) ).

./pope
Rickard Sundin
Greenhorn

Joined: Mar 10, 2003
Posts: 16
I have heard people say that the argument for only testing the public interface of a unit is that you will be able to refactor the internal structure of the unit without having to change the tests. I guess that this comes from bad experience in projects where test discouraged people from doing codechanges since they would also have to rewrite the test (i.e. more work).

I myself find this a bit strange since a solid testbed should give developers the courage to refactor code and be confident they did not break anything.

/Rickard
Lasse Koskela
author
Sheriff

Joined: Jan 23, 2002
Posts: 11962
    
    5
Originally posted by Ali Pope:
Is Mr.Rainsberger explaining why the testing should be done only through public API? (this will give us a hint on the long thread of testing private methods ;-) )


Originally posted by Rickard Johansson:
I have heard people say that the argument for only testing the public interface of a unit is that you will be able to refactor the internal structure of the unit without having to change the tests. I guess that this comes from bad experience in projects where test discouraged people from doing codechanges since they would also have to rewrite the test (i.e. more work).

I myself find this a bit strange since a solid testbed should give developers the courage to refactor code and be confident they did not break anything.

I don't have the book in my hand right now, but I think I remember one of the argued benefits being exactly what Rickard is talking about.
Alexandru Popescu
Ranch Hand

Joined: Jul 12, 2004
Posts: 995
So I would conclude from the aboves that this reason is in a way deprecated :-).

./pope
Lasse Koskela
author
Sheriff

Joined: Jan 23, 2002
Posts: 11962
    
    5
Originally posted by Ali Pope:
So I would conclude from the aboves that this reason is in a way deprecated :-).

How so?
Petr Blahos
Ranch Hand

Joined: Apr 28, 2004
Posts: 131
Originally posted by Lasse Koskela:
I'm also putting my tests in a separate source folder but in the same Java package as the production code, i.e.


Cool! I haven't thought about that. I have actually always had my tests
separated (in no package).

Still, does anybody know if the book (JUnit Pocket Guide) discusses that?

Thanks, Petr
Lasse Koskela
author
Sheriff

Joined: Jan 23, 2002
Posts: 11962
    
    5
Kent posted the TOC in this thread. Based on that, it doesn't look like he discusses too much about placing your tests. I'd recommend JUnit Recipes for a more thorough discussion of various JUnit "best practices".
Alexandru Popescu
Ranch Hand

Joined: Jul 12, 2004
Posts: 995
Originally posted by Lasse Koskela:

How so?


I have just used/interpreted this fragment:


I guess that this comes from bad experience in projects where test discouraged people from doing codechanges since they would also have to rewrite the test (i.e. more work).


Moreover, the discussion on the testing private methods thread seems to prove that the unit testing is not meant only for public API.

./pope
Alexandru Popescu
Ranch Hand

Joined: Jul 12, 2004
Posts: 995
Originally posted by Lasse Koskela:
I'd recommend JUnit Recipes for a more thorough discussion of various JUnit "best practices".


Wow... many horseshoes. I think this will become a mandatory reading. I think I'll go for it.

./pope
Lasse Koskela
author
Sheriff

Joined: Jan 23, 2002
Posts: 11962
    
    5
Originally posted by Ali Pope:
Wow... many horseshoes.

And there's more to come once I get to writing my review. So far, it's definitely a 9 or 10.
Lasse Koskela
author
Sheriff

Joined: Jan 23, 2002
Posts: 11962
    
    5
I guess that this comes from bad experience in projects where test discouraged people from doing codechanges since they would also have to rewrite the test (i.e. more work).

Well, regardless of how good a state your project is in with regard to unit testing, it's always a good idea to make sure that your tests don't become more brittle than they need to be. With that in mind, I'd say testing only through the public interface is a good rule. When a test knows too much about the code under test, it becomes brittle by definition ("too much"). And then we're back to the point about wanting to test a private method being a code smell saying there's functionality that would benefit from being extracted into its own class.

Originally posted by Ali Pope:
Moreover, the discussion on the testing private methods thread seems to prove that the unit testing is not meant only for public API.

Some programming languages don't even have the concept of public vs private methods...
Alexandru Popescu
Ranch Hand

Joined: Jul 12, 2004
Posts: 995
You are in advantage here ;-). I think I will trust your reviews and go for the book.

./pope
Lasse Koskela
author
Sheriff

Joined: Jan 23, 2002
Posts: 11962
    
    5
Originally posted by Ali Pope:
I think I will trust your reviews and go for the book.
You won't regret it, I guarantee.
Alexandru Popescu
Ranch Hand

Joined: Jul 12, 2004
Posts: 995
Originally posted by Lasse Koskela:

Well, regardless of how good a state your project is in with regard to unit testing, it's always a good idea to make sure that your tests don't become more brittle than they need to be. With that in mind, I'd say testing only through the public interface is a good rule. When a test knows too much about the code under test, it becomes brittle by definition ("too much"). And then we're back to the point about wanting to test a private method being a code smell saying there's functionality that would benefit from being extracted into its own class.


Fully agree :-). However for the private method testing we have the other thread ;-).

./pope
Alexandru Popescu
Ranch Hand

Joined: Jul 12, 2004
Posts: 995
Originally posted by Lasse Koskela:
You won't regret it, I guarantee.


Even if for the beginning (some years ago) I didn't trust the idea of forums, this one changed a lot the way I'm seing. IMO JavaRanch is the best for java guys and more. And sure.... I trust your reviews ;-).

./pope
J. B. Rainsberger
author
Ranch Hand

Joined: Aug 05, 2004
Posts: 87
Originally posted by Rickard Johansson:
I have heard people say that the argument for only testing the public interface of a unit is that you will be able to refactor the internal structure of the unit without having to change the tests. I guess that this comes from bad experience in projects where test discouraged people from doing codechanges since they would also have to rewrite the test (i.e. more work).

I myself find this a bit strange since a solid testbed should give developers the courage to refactor code and be confident they did not break anything.


Originally I did not want to reply to this, because I didn't want to stretch out this discussion; however, I feel it's important to point something out about Rickard's objection.

Let's describe what happens when I have to change a test because I refactored.

When I start, my codebase is GREEN (all tests pass). Now I want to refactor some internal implementation detail inside a private method that has tests. My first instinct is to change the private method, then recompile. Since it is merely an implementation detail, I expect the codebase to remain GREEN; however, because I have tests for this implementation detail, there is a good chance that either the tests no longer compile, or they compile but fail. This is because I'm changing behavior upon which the tests depend.

So now I have two choices, neither of which I particularly like:

  • Test-drive a new private method from scratch, then change the production code to invoke the new method, then remove the old method. This is among the weakest kinds of refactorings: Replace Algorithm. It requires having two sets of code that both pass semantically equivalent but syntactically different tests -- that is, the tests are written differently (APIs are different), but mean the same thing. Do you think we might get that wrong? This sounds like a lot more effort than I expected when I started.
  • Change the production code, then change the tests to match. Hm. Do we really want to surrender the value that comes from writing the tests first? The more we pursue this option, the further we slide back into testing our code after we write it, and I know I'm less productive when I do that.


  • When refactoring, the ideal rhythm is to change production code without changing tests. This means that the design needs to be flexible enough to minimize the number of tests that fail when the production code changes. One of the ways we achieve this is by a combination of interface-rich designs and testing entirely through the public API. It works very well, and the alternatives are less desirable.

    That's why I recommend it. As always, your mileage may vary.


    Author of <a href="http://www.amazon.com/exec/obidos/ASIN/1932394230/ref=jranch-20" target="_blank" rel="nofollow">JUnit Recipes: Practical Methods for Programmer Testing</a>
    Alexandru Popescu
    Ranch Hand

    Joined: Jul 12, 2004
    Posts: 995
    Thanks for insights. What about a combination of the two solutions:
    1. add the new test
    2. refactor the method
    3. remove the old test
    Shouldn't be more appropriate?

    ./pope
    J. B. Rainsberger
    author
    Ranch Hand

    Joined: Aug 05, 2004
    Posts: 87
    I'm not saying that that won't work. I'm merely pointing out that when I think of refactoring, I don't think of writing new tests; instead, I think about changing code so that it still passes the existing tests. When I'm writing new tests, I assume it's to add new features. The generally accepted view of things is that one either refactors, or adds new features, but never both simultaneously.
    Alexandru Popescu
    Ranch Hand

    Joined: Jul 12, 2004
    Posts: 995
    Originally posted by J. B. Rainsberger:
    I'm merely pointing out that when I think of refactoring, I don't think of writing new tests; instead, I think about changing code so that it still passes the existing tests. When I'm writing new tests, I assume it's to add new features. The generally accepted view of things is that one either refactors, or adds new features, but never both simultaneously.


    Indeed one of the most valuable things brought by unit testing is to ease the refactoring process. But from time to time in evolving systems, I think you will need to follow one of the 3 methods exposed above.

    ./pope
    [ October 14, 2004: Message edited by: Ali Pope ]
    Alexandru Popescu
    Ranch Hand

    Joined: Jul 12, 2004
    Posts: 995
    Originally posted by Lasse Koskela:
    You won't regret it, I guarantee.


    Lasse good news: right now it is on my download manager! After finishing the current one I will go for it!

    ./pope
    Lasse Koskela
    author
    Sheriff

    Joined: Jan 23, 2002
    Posts: 11962
        
        5
    Originally posted by Ali Pope:
    Lasse good news: right now it is on my download manager! After finishing the current one I will go for it!

    Download manager? Interesting. How fully have you converted into e-books?
    Alexandru Popescu
    Ranch Hand

    Joined: Jul 12, 2004
    Posts: 995

    How fully have you converted into e-books?


    I am not I am getting this . Manning is also offering the book in pdf format which is the only possiblity (for my country) to have the book asap.

    ./pope
    Lasse Koskela
    author
    Sheriff

    Joined: Jan 23, 2002
    Posts: 11962
        
        5
    Originally posted by Ali Pope:
    I am not I am getting this . Manning is also offering the book in pdf format which is the only possiblity (for my country) to have the book asap.

    I meant to ask whether you're mostly buying books in PDF format (i.e. e-books). I've heard of some people buying all their books in electronic form nowadays, but personally I'd much rather take a hardcopy. I guess it just takes some time to get used to.
    Alexandru Popescu
    Ranch Hand

    Joined: Jul 12, 2004
    Posts: 995
    Unfortunately I usualy must buy e-books cause i'm leaving in one of the not-supported countries and for buying the hard copies I have to rely on my friends (and I rely on them but I cannot bother them all the time ).
    I surely like to have a real book feeling, but .... c'est la vie .

    ./pope
    Kent Beck
    author
    Ranch Hand

    Joined: Nov 07, 2003
    Posts: 45
    The JUnit Pocket Guide does briefly discuss where to put tests. The two options I settle on are:
    * Same package, different directory
    * Same directory structure, different package

    Kent Beck
    Three Rivers Institute
    www.threeriversinstitute.org


    Author of <a href="http://www.amazon.com/exec/obidos/ASIN/0596007434/ref=jranch-20" target="_blank" rel="nofollow">JUnit Pocket Guide</a>
    Alexandru Popescu
    Ranch Hand

    Joined: Jul 12, 2004
    Posts: 995
    Kent are there any particular reasons behind these types of organization or it is just the natural ways?

    ./pope
    [ October 15, 2004: Message edited by: Ali Pope ]
    Lasse Koskela
    author
    Sheriff

    Joined: Jan 23, 2002
    Posts: 11962
        
        5
    Originally posted by Ali Pope:
    Kent are there any particular reasons behind these types of organization or it is just the natural ways?
    There aren't too many ways to separate test classes from production classes. You can separate them by naming convention or by their physical location.

    The naming convention might work, but there are a few little things that make it far from perfect. For example, you're cluttering your "view" with test classes when you're implementing new features and vice versa.

    The separation based on physical location can be applied to either the root or leaf of a file path, i.e. using a different root folder for tests or using a different "leaf folder" for tests. It doesn't really make sense to apply both at the same time.
    [ October 16, 2004: Message edited by: Lasse Koskela ]
    Alexandru Popescu
    Ranch Hand

    Joined: Jul 12, 2004
    Posts: 995
    Thanks Lasse. Your answer confirms my beliefs.

    ./pope
     
    I agree. Here's the link: http://aspose.com/file-tools
     
    subject: JUnit Pocket Guide: Where to put your tests