aspose file tools*
The moose likes Design and the fly likes BDD in Action: Question about different frameworks Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Engineering » Design
Bookmark "BDD in Action: Question about different frameworks" Watch "BDD in Action: Question about different frameworks" New topic
Author

BDD in Action: Question about different frameworks

Burk Hufnagel
Ranch Hand

Joined: Oct 01, 2001
Posts: 814
    
    3
John,
You've mentioned several different BDD frameworks for Java developers, and I was wondering is you could say a few words about them. I've used Spock, but have no experience with JBehave or Thucydides, and I was hoping you could compare them briefly - or at least tell me if there's one you prefer and why.

I also noticed you haven't mentioned easyB and was wondering if it's because you don't have enough experience to talk about it, or if you just don't recommend it.

Thanks,
Burk


SCJP, SCJD, SCEA 5 "Any sufficiently analyzed magic is indistinguishable from science!" Agatha Heterodyne (Girl Genius)
John Smart
Author
Ranch Hand

Joined: Aug 06, 2013
Posts: 33
    
    5
OK, here goes. There are two levels of BDD as you know: one for high-level requirements and automated acceptance criteria, and one more focused on coding. While you could use the same tooling for both, the focus of tools tends to be better suited for one of these two approaches. Here a few of the main ones:

1) High-level BDD tools: Usually separate the specifications (e.g. .story or .feature files for JBehave, Cucumber etc, or HTML pages for Concordion):
- JBehave is the evolution of Dan North's original BDD tool: it uses the classic BDD "Given..When..Then" notation. Specifications are in .story files, and the step implementations are in Java classes. This approach work very well as a collaboration and communication tool to define the specs, but would usually be too much overhead for low-level coding.
- Cucumber is a widely-used tool from the Ruby world, which now has a JVM version that works in many languages. It uses a very similar "Given..When..Then" notation. Ports of Cucumber exist for .NET (SpecFlow), Python (Behave and Lettuce), PHP (Behat) and other languages. These tools use the same approach as JBehave, and the same comments apply.
- In Concordion you write the specifications in the form of tagged HTML, and implement the actual tests in Java. I've mentioned Concordion in another post: I find the HTML page side of Concordion a bit hard to scale for larger projects, when you need to be able to slice and dice your requirements (e.g. have views by feature, by release, by iteration, etc...)
- Fitnesse is an older tool where you write specifications in a Wiki and implement the steps in Java. Haven't used it much myself, but I find places that use it tend to end up expressing everything in tabular format, which I find not appropriate for all sorts of requirements.
- Easyb is a Groovy-based BDD DSL where the "Given-When-Then" specs are embedded in a Groovy script alongside the implementation. This has advantages and disadvantages. Easyb can be used for both high-level and coding level BDD, but is harder for non-developers to work with.

Thucydides (http://www.thucydides.info) is a bit different: it is a reporting tool that helps you organize your automated acceptance tests, or just helps generate better living documentation. It's goal is to work with all of the main BDD tools: currently it has deep integration and reporting capabilities with JBehave, JUnit and easyb, and can generate living documentation for tools like Cucumber, Specflow or Behave.

2) TDD-style (coding-level) BDD tools are unit-testing tools with a BDD flavor: they are more focused on developer productivity and technical documentation than communicating with the business. Some use a "given-when-then" structures and support tabular data, whereas others use a more primitive style where you say things like 'it "should do this".
- Spock: very nice light-weight BDD unit testing in Groovy for Groovy or Java code
- Spec2: code-level BDD in Scala
- RSpec: low-level BDD in Ruby, using a simpler structure
- Jasmine: BDD for Javascript (more like rspec than spock)
- Kiwi: BDD for iOS (uses an rspec style)

There are a few native Java BDD tools (e.g. JDave), but I find they really struggle with the limits of the Java syntax, and contain a lot of boiler-plate code. Note you can also do BDD-style tests in JUnit or NUnit just by using expressive naming conventions and structuring your tests appropriately. The other tools just provide stronger built-in support for this.

Personally at the moment I use JBehave and Thucydides at the requirements level and Spock at the coding level for JVM projects, Specflow and Thucydides for .Net, Jasmine (and Karma) when I'm working with AngularJS. I'd be happy to use Cucumber as well but I haven't got round to getting the Thucydides/Cucumber integration done yet.
Burk Hufnagel
Ranch Hand

Joined: Oct 01, 2001
Posts: 814
    
    3
Wow. Thanks for the info dump on frameworks. Way more than I had expected.

Sounds like I need to get some experience with Thucydides.

Wondering about your statement that the Given-When-Then format is too much overhead for low level coding. For me, it makes the context very clear. I don't get how the 'it should' version does that. Can you help me understand?

Thanks again, John. This has been a very illuminating week.

With appreciation,
Burk
John Smart
Author
Ranch Hand

Joined: Aug 06, 2013
Posts: 33
    
    5
I didn't mean that the Given-When-Then is overhead for unit tests - in fact it works really nicely with Spock. When you use tools like JBehave or Cucumber, the scenarios (Given-When-Thens) are in plain text files, and the test implementations is in source code form. Putting the requirements in text (.story or .feature) files is more overhead, as you need to keep the specification texts and the implementation code in sync, but the overhead is justified for BDD acceptance criteria because it makes them much easier to use as a communication tool. For unit tests, something like Spock is better.
Burk Hufnagel
Ranch Hand

Joined: Oct 01, 2001
Posts: 814
    
    3
Ah. That makes a lot more sense.

It seems odd though to have the scenarios in a separate plain text file - I'd be afraid that the scenarios and the implementations would get out of synch. Is there something in the framework that helps prevent that problem? Or is it up to the developers to make sure everything stays synched up?
John Smart
Author
Ranch Hand

Joined: Aug 06, 2013
Posts: 33
    
    5
If they get out of sync, it's easy to spot: the tests don't run :-). For BAs and testers, the separation keeps the requirements clean and isolated from the implementation, which makes them easier to work with collaboratively. But you do have to be well organized with larger projects to avoid stepping on everyones toes. I'll include some techniques to do this in JBehave in the book somewhere.
Burk Hufnagel
Ranch Hand

Joined: Oct 01, 2001
Posts: 814
    
    3
John Smart wrote:If they get out of sync, it's easy to spot: the tests don't run :-).

I don't understand. If the scenarios are in a separate, plain text file, then as long as the tests match the system's behavior shouldn't they pass? How does the content of the scenarios file impact the tests?
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: BDD in Action: Question about different frameworks