Win a copy of Micro Frontends in Action this week in the Server-Side JavaScript and NodeJS forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Paul Clapham
  • Bear Bibeault
  • Junilu Lacar
Sheriffs:
  • Jeanne Boyarsky
  • Tim Cooke
  • Henry Wong
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • salvin francis
  • Frits Walraven
Bartenders:
  • Scott Selikoff
  • Piet Souris
  • Carey Brown

JUnit Pocket Guide: future of JUnit?

 
author
Posts: 799
5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Greetings Kent,

Any word on a next major version of JUnit?

Do you see any value in taking advantage of Java annotations a la NUnit?

thanks,
Jeff
 
author
Posts: 11962
5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
There's been a few rumors about merging features from TestNG into JUnit 4.0 but Cedric (the author of TestNG) says he isn't aware of anyone actually doing something about it -- neither the merging nor the 4.0.
 
Ranch Hand
Posts: 995
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Even if JUnit and TestNG seem to work and help in the same field, I think there are some conceptual mismatches between the two. One of the most important is the chain of testing (enabled by TestNG) against the independency of each test method in JUnit.

./pope
 
Lasse Koskela
author
Posts: 11962
5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Originally posted by Ali Pope:
Even if JUnit and TestNG seem to work and help in the same field, I think there are some conceptual mismatches between the two. One of the most important is the chain of testing (enabled by TestNG) against the independency of each test method in JUnit.


Regarding test ordering, the OrderedTestSuite from gsbase provides the ability to select a subset of your tests to be executed in a developer-specified order. Yes, it is a bit of a hack but still.
 
Alexandru Popescu
Ranch Hand
Posts: 995
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Interesting one Lasse. Thanks. I think that the big issue here is that (afaik - correct me if i'm wrong) Junit uses a new instance per test (not per test case, but per test method), while TestNG use an instance per test case. There are some discussions about the correctness/usage of both these strategies.

./pope
 
Lasse Koskela
author
Posts: 11962
5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
There certainly are differences and I'm not expecting the two projects to literally merge into one anytime soon. Instead, I'm expecting for JUnit to adopt some features from TestNG -- mainly the use of metadata to describe and select tests to be executed.
 
Alexandru Popescu
Ranch Hand
Posts: 995
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Originally posted by Lasse Koskela:
[...]mainly the use of metadata to describe and select tests to be executed.



This points are really powerfull. I think JUnit has some little help for this in JUnitDoclet, ain't it?

./pope
 
Lasse Koskela
author
Posts: 11962
5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Originally posted by Ali Pope:
I think JUnit has some little help for this in JUnitDoclet, ain't it?

I haven't used JUnitDoclet but from what I read from the website, it's intended to generate test method skeletons for all public methods of your production code which isn't quite what I'm looking for in TestNG's metadata approach. In fact, I don't value auto-generated test method skeletons at all, really, since I don't want to test "each public method in my production code". I want to test the intended behaviour of my production code which may or may not have anything to do with which public methods my production code has. Besides, I'd have to rename the test methods anyway to better represent what I'm testing and I doubt that JUnitDoclet has the slightest chance of guessing how many tests I'll come up with for a given method.

In other words, I don't think JUnitDoclet would fit my way of doing things.
 
Alexandru Popescu
Ranch Hand
Posts: 995
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ohhh yes I got tricked by the fact they are saying they use javadoc (but in fact they are not using it in the way TestNG used annotations). From here my wrong idea on it. Sorry.

./pope

PS: I thought at one moment that porting TestNG for J2SE < 1.5 would be a great idea, but till now the time didn't allow me.
 
Lasse Koskela
author
Posts: 11962
5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Originally posted by Ali Pope:
I thought at one moment that porting TestNG for J2SE < 1.5 would be a great idea, but till now the time didn't allow me.


Well, since in J2SE < 1.5 we don't have runtime access to, say, javadoc tags, we can't use them in the way TestNG uses @Attributes. However, one could write a custom TestRunner for JUnit that goes through a given set of classes, picks those that implement JUnit's Test interface, and further filter the selected classes based on whether they implement your custom interfaces like IgnoreThisTestCaseForNow, LongLastingTest, AcceptanceTest, etc.

I wouldn't mind seeing such a utility. Also, a lot of the groundwork has been done by Mike Bowler's gsbase library.
 
Alexandru Popescu
Ranch Hand
Posts: 995
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
My imagined solution goes some other directions:
  • extract javadoc tags and make them available for TestRunner (properties, java classes,(?))
  • either use bytecode engineering to add annotation like structures on the test classes, this infos being retrieved at load time (oh yes custom classloader).


  • waiting for opinions (who knows made we can organize a team to start this project *- ).

    ./pope
     
    author
    Posts: 14112
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator

    Originally posted by Ali Pope:
    Interesting one Lasse. Thanks. I think that the big issue here is that (afaik - correct me if i'm wrong) Junit uses a new instance per test (not per test case, but per test method), while TestNG use an instance per test case. There are some discussions about the correctness/usage of both these strategies.



    This is a common confusion. A TestCase class isn't a testcase - an instance of the class is (similar to the JFrame class not being a single frame). The naming convention for methods is just a convenient way to define more than one testcase inside one class, so that they can share a common fixture (setUp and tearDown).
     
    Ilja Preuss
    author
    Posts: 14112
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator

    Originally posted by Jeff Langr:
    Any word on a next major version of JUnit?

    Do you see any value in taking advantage of Java annotations a la NUnit?



    I don't think it needs a new version of JUnit to use annotations. Due to the modularity of JUnit, all it would need is an Adapter that makes your annotated test implement the Test interface, and a Suite or TestRunner that automatically collects your tests and wraps them in the adapter.

    I think it would be a fun thing to implement. Unfortunately there are some more important projects in the pipeline for me.
     
    author
    Posts: 46
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator
    Hi guys,

    Thought I would take this opportunity to share my views about TestNG and JUnit.

    The two tools have a few differences in purpose: while JUnit focuses exclusively on unit testing, TestNG tries to provide a set of features that cover the entire testing domain: unit, regression, functional, etc...

    The annotations allow you to design your testing hierarchy with zero intrusion from the testing framework. You are free to use your own hierarchy, your own base classes and name your methods whatever you like. This is something that has received a lot of praises from its users.

    Finally, I recently added parallelism and time-outs to TestNG, so that testing asynchronous code (such as JMS) is absolutely trivial. I'll blog about this very soon, stay tuned.

    --
    Cedric
    (TestNG)

    TestNGWeblog
     
    Alexandru Popescu
    Ranch Hand
    Posts: 995
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator
    Thank you Cedric for your insights. Do you have any intentions to extend TestNG for woking with jdks previous to 1.5? (forgive me if you answered this too many times)

    ./pope
     
    Cedric Beust
    author
    Posts: 46
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator
    Actually, I have never answered this question before because it's never been asked

    I guess it would be possible to use Javadoc tags instead of annotations, and that TestNG's dependencies on the JDK 5.0 runtime (e.g. java.util.concurrent) could be backported.

    The question is: is there any interest in a JDK 1.4 TestNG?

    --
    Cedric
     
    Jeff Langr
    author
    Posts: 799
    5
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator

    Originally posted by Ilja Preuss:
    I don't think it needs a new version of JUnit to use annotations. Due to the modularity of JUnit, all it would need is an Adapter that makes your annotated test implement the Test interface, and a Suite or TestRunner that automatically collects your tests and wraps them in the adapter.

    I think it would be a fun thing to implement. Unfortunately there are some more important projects in the pipeline for me.



    Is there a resistance to a new version? My guess might be that since JUnit has become a de facto standard for testing within several IDEs, it might be more difficult to gain acceptance for a major upgrade.

    Having looked at the underlying code and design, I'd love to see a rewrite. A few things I tried to do with it recently were more difficult than I'd hoped.

    -Jeff-
     
    Alexandru Popescu
    Ranch Hand
    Posts: 995
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator
    Sure it is an interest in this (for the beginning you can count me ). Some post above I already stated that I have this in my future plan and I am waiting to reach the moment.

    ./pope
     
    Lasse Koskela
    author
    Posts: 11962
    5
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator

    Originally posted by Jeff Langr:
    Is there a resistance to a new version? My guess might be that since JUnit has become a de facto standard for testing within several IDEs, it might be more difficult to gain acceptance for a major upgrade.


    I don't think there would be too much resistance as long as we'd have backwards compatibility, i.e. the new JUnit "4.0" would still run TestCases based on naming conventions if there's no metadata available (indicating that the particular TestCase was written for the new version).
     
    Cedric Beust
    author
    Posts: 46
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator

    Originally posted by Jeff Langr:

    Having looked at the underlying code and design, I'd love to see a rewrite. A few things I tried to do with it recently were more difficult than I'd hoped.
    -Jeff-



    Jeff, can you expand on your difficulties with JUnit? I'm curious to see if TestNG would fix your problems...

    --
    Cedric
     
    Cedric Beust
    author
    Posts: 46
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator

    Originally posted by Lasse Koskela:

    I don't think there would be too much resistance as long as we'd have backwards compatibility, i.e. the new JUnit "4.0" would still run TestCases based on naming conventions if there's no metadata available (indicating that the particular TestCase was written for the new version).


    For the record, TestNG has a JUnitConverter tool that will recursively convert all your tests to the TestNG annotations.

    Converting your entire test codebase to TestNG should be a matter of seconds.

    --
    Cedric
     
    Jeff Langr
    author
    Posts: 799
    5
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator

    Originally posted by Cedric Beust:


    Jeff, can you expand on your difficulties with JUnit? I'm curious to see if TestNG would fix your problems...

    --
    Cedric



    Greetings Cedric,

    Other than nits about the implementation and design, one thing I've wanted to do is easily determine whether or not a suite contained a specific test case. So I've had to code something like:



    Which seems to work fine, but I would've expected that this would've been required by some test for JUnit itself.

    Also, when hooking into the test collector, my specific need in one case ended up being inefficient (which mattered against the large number of test classes we had). The isTestClass method has the class name string as an argument. In order for me to determine some specific details of the class, I had to use Class.forName, which means that this must happen twice. I could be wrong on this one, since I'm recalling it from memory.

    -Jeff-
     
    Cedric Beust
    author
    Posts: 46
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator

    Originally posted by Jeff Langr:
    Which seems to work fine, but I would've expected that this would've been required by some test for JUnit itself.



    Considering that JUnit is a test framework, I have to say the quality of testing it has is... alarming, to say the least.



    Also, when hooking into the test collector, my specific need in one case ended up being inefficient (which mattered against the large number of test classes we had). The isTestClass method has the class name string as an argument. In order for me to determine some specific details of the class, I had to use Class.forName, which means that this must happen twice. I could be wrong on this one, since I'm recalling it from memory.
    -Jeff-[/QB]



    So you need to plug into the test framework. Again, JUnit is more on the blackbox side in this area.

    I did my best to make TestNG very open and extensible, so you have a bunch of interfaces and factories that allow you to pretty much redefine completely the way it works (this is how it supports both annotations and the JUnit way).

    Closer to what you are looking for, you would probably be intersted in the Listening API which allows you to be notified whenever a test starts, when it finishes, whether is succeeded, etc... This is how the HTML reports are generated, and this is how one of the TestNG developers hooked JUnitReports.

    I'm curious, though: why did you have to write this contains() method?

    --
    Cedric
     
    Jeff Langr
    author
    Posts: 799
    5
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator
    Hi Cedric,

    I wanted the contains method to support testing of my own JUnit modifications.

    I looked at TestNG several months ago and liked some of the things I saw in it. What would sell it to me is a good Eclipse plugin for it (is there one yet?).

    thanks,
    Jeff
    [ October 13, 2004: Message edited by: Jeff Langr ]
     
    Cedric Beust
    author
    Posts: 46
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator

    Originally posted by Jeff Langr:
    Hi Cedric,
    I wanted the contains method to support testing of my own JUnit modifications.

    I looked at TestNG several months ago and liked some of the things I saw in it. What would sell it to me is a good Eclipse plugin for it (is there one yet?).

    [ October 13, 2004: Message edited by: Jeff Langr ]



    I am working on it (just created a separate project on java.net).

    Don't let that hold you back, though, I find that the HTML reports generated by TestNG are an acceptable substitute to the green bar.

    But yes, an Eclipse plug-in is next on my TODO list. What would you like to see in it exactly?

    --
    Cedric
     
    Jeff Langr
    author
    Posts: 799
    5
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator

    Originally posted by Cedric Beust:
    But yes, an Eclipse plug-in is next on my TODO list. What would you like to see in it exactly?



    Nothing more than seamless integration with Eclipse, of course. The little things matter: a new template; a new classpath variable; the ability of quick fix to supply appropriate completions; an addition to the run menu; double-click walkback to link back to code; filter stack trace; etc.

    A minor detail: One thing that Eclipse improved on but is only halfway there is the Result Comparison dialog. This dialog allows you to visually compare two strings on a failed assertEquals. A great thing to have, but it should (a) show the expected and actual above each other instead of side-by-side (b) using a non-proportional font and (c) supporting synchronized scrolling.

    -Jeff-
     
    Alexandru Popescu
    Ranch Hand
    Posts: 995
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator
    Together with the JUnitConvertor tool, an Eclipse plugin will open more doors to TestNG. Unfortunatelly I am not gonna profit from it as we (my company) will not be able to use JDK1.5 till (at least) next year summer .
    I also think there are other companies in this position too, that's why I have thought some time ago of making TestNG compatible with previous JDKs.

    ./pope
     
    Cedric Beust
    author
    Posts: 46
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator

    Originally posted by Ali Pope:
    Together with the JUnitConvertor tool, an Eclipse plugin will open more doors to TestNG. Unfortunatelly I am not gonna profit from it as we (my company) will not be able to use JDK1.5 till (at least) next year summer .
    I also think there are other companies in this position too, that's why I have thought some time ago of making TestNG compatible with previous JDKs.
    ./pope



    Well, if you feel like contributing, you are more than welcome to join the project. Processing Javadoc annotations instead of JDK5 ones should be easy to add to TestNG...

    --
    Cedric
     
    Alexandru Popescu
    Ranch Hand
    Posts: 995
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator
    This is good news! I will prioritize again my future plans, trying to pull TestNG to the top. I will use the mailing list for further communication.

    ./pope
     
    Whatever. Here's a tiny ad:
    Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop
    https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
    reply
      Bookmark Topic Watch Topic
    • New Topic