File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
The moose likes Testing and the fly likes Better Automated Testing Tool Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Engineering » Testing
Bookmark "Better Automated Testing Tool" Watch "Better Automated Testing Tool" New topic

Better Automated Testing Tool

Syed Saifuddin
Ranch Hand

Joined: Sep 01, 2003
Posts: 130
Can anyone help me in identifying that which one is the best automated testing tool or framework for testing load, perfformance and security of the web application written in jsp and struts.

I developed web based HR & Payroll System and now it my responsibility to finalized and assured that this system cannot make mistake in calculating salary and other critical issue such as security factor and performance.

Thank You
Syed Saifuddin
[ August 25, 2005: Message edited by: Syed Saifuddin ]

Thank You & Best Regards,

Syed Saifuddin,
Senior Software Engineer

SAP Oracle AIX & Java Training
Lasse Koskela

Joined: Jan 23, 2002
Posts: 11962
Originally posted by Syed Saifuddin:
I developed web based HR & Payroll System and now it my responsibility to finalized and assured that this system cannot make mistake in calculating salary and other critical issue such as security factor and performance.

For web applications, you have basically two categories of tools: those that simulate the HTTP protocol and those that control a real web browser.

The power of the former lies in that you can write your tests in Java/Ruby/Python/etc. and run them from anywhere (since they're "headless"). The downside with these tools is that they duplicate the functionality of real web browsers and thus may not be a good enough representation of a browser's capabilities. For example, HttpUnit and JWebUnit rely on the Rhino JavaScript engine which doesn't implement all Internet Explorer specific JavaScript functions. In other words, even if all your automated end-to-end tests written with a tool like HttpUnit/JWebUnit/HtmlUnit/Canoo WebTest/etc. pass without a single failure, you can't really say that the same tests would pass if executed manually using a specific browser like IE or Firefox. In short, these are great tools but mostly suitable for applications that are strict about using only those features of JavaScript that are supported by the underlying JavaScript engine.

The power of the latter group--those tools that control a real browser--obviously is in how they use the real thing for testing and thus give a better representation of reality in terms of JavaScript. The downside is that executing tests through the browser is rather difficult to do with current tools in a platform/browser independent manner. For example, WTR, PAMIE, SAMIE, and Jiffie all let you control Internet Explorer on Windows but none of those tools work on Linux, for example, nor do they have any kind of support for Mozilla/Firefox. Selenium is platform independent as it supports Internet Explorer, Mozilla/Firefox, and Safari on Windows, Linux, and OS X (where applicable) but it has its own issues (the "TestRunner mode" requires you to manually start your server, launch a browser, and click a button in order to run the tests; the "Driven mode" lets you write your tests in Java/Ruby/Python/etc and execute them with the same tools you use to run your unit tests without manual steps but the driven mode is significantly slower to execute than any of the alternatives (due to its rather exotic architecture).

Now, you can use either class of these tools for functional testing, i.e. verifying that the right things happen when a user clicks through a certain sequence of steps. For performance testing, you might consider wrapping your HttpUnit/JWebUnit/HtmlUnit tests with JPerfUnit or go with a specialized load testing tool such as Grinder or JMeter. There's no way to use one of the browser-controlling tools for performance testing in a meaningful way.

For security, your best bet is to write a couple of "regular" functional tests to verify that you can't access certain areas of the web application unless you've logged in but that's not really a security issue--just regular functionality. The so-called "real" security testing would involve a security consultant analysing your architecture and running penetration tests against your system with tools like Nessus.

Having said all that, I'd like to suggest that you test domain logic like salary calculations in your *unit* tests instead of functional tests.

Author of Test Driven (2007) and Effective Unit Testing (2013) [Blog] [HowToAskQuestionsOnJavaRanch]
Syed Saifuddin
Ranch Hand

Joined: Sep 01, 2003
Posts: 130
Hello Lasse Koskela

I am very thankful to you for the detailed answer. One more thing I really need from you is the single name(not a verity of)framework to use because learning all these framework make extra burden.
If you please identify the name of one that cover max of the testing (security, performance and load) in web application it become easier for me to start.

Thankyou Again
Syed Saifuddin
Lasse Koskela

Joined: Jan 23, 2002
Posts: 11962
None of the tools mentioned cover all your needs. That's a fact of life you'll just have to live with.

In order to cover most of your needs, I'd suggest looking into HttpUnit and JWebUnit. The latter is built on top of the former, offers a higher level of abstraction, and therefore a bit nicer API while still giving you (some) access to the underlying (more powerful) HttpUnit API.
Scott Glaser

Joined: Jan 20, 2003
Posts: 10
There are a lot of test tools out there. For starters take a look at:
Kishore Dandu
Ranch Hand

Joined: Jul 10, 2001
Posts: 1934
Irrespective of JSP or other technologies you have used to render your web content; I would suggest 'Load Runner' for a reasonable load test on you system.

For performance and memory aspects you can do profiling using JProfile or something similar.

SCJP, blog
Michael Jameson

Joined: Sep 30, 2005
Posts: 1
I've only just started looking into Selenium but I love it so far.
Regarding the comment about having to start TestRunner mode manually, this is from the documentation:

Continuous Integration

Selenium can be integrated with an automated build. When the parameter "auto=true" is added to the URL, Selenium will run the entire suite of tests, and then post the results to a handling URL. The default URL is "/postResults", but an alternative handler location can be provided by specifying a "resultsUrl" parameter.

Therefore, the steps for continuous integration are:


Create a servlet-type application at the url /postResults which can read the parameters above and write them to a file

Create a script which can start up a brower and send to to the URL: selenium?auto=true

Generally, this can be done by merely calling the browser with the URL as an argument:

firefox.exe http://localhost/selenium?auto=true


Make your continuous build:
* Call the script from step 2, preferably using more than one browser
* Wait for it to finish, possibly by checking for the existence of the file(s) from step 1
* Parse these files to determine whether the build passed or failed
* Act accordingly (send emails, update a build web page, etc.)
I agree. Here's the link:
subject: Better Automated Testing Tool
It's not a secret anymore!