• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Better Automated Testing Tool

 
Ranch Hand
Posts: 130
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello,
Can anyone help me in identifying that which one is the best automated testing tool or framework for testing load, perfformance and security of the web application written in jsp and struts.

Detailed:
I developed web based HR & Payroll System and now it my responsibility to finalized and assured that this system cannot make mistake in calculating salary and other critical issue such as security factor and performance.

Thank You
Syed Saifuddin
[ August 25, 2005: Message edited by: Syed Saifuddin ]
 
author
Posts: 11962
5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Syed Saifuddin:
I developed web based HR & Payroll System and now it my responsibility to finalized and assured that this system cannot make mistake in calculating salary and other critical issue such as security factor and performance.


For web applications, you have basically two categories of tools: those that simulate the HTTP protocol and those that control a real web browser.

The power of the former lies in that you can write your tests in Java/Ruby/Python/etc. and run them from anywhere (since they're "headless"). The downside with these tools is that they duplicate the functionality of real web browsers and thus may not be a good enough representation of a browser's capabilities. For example, HttpUnit and JWebUnit rely on the Rhino JavaScript engine which doesn't implement all Internet Explorer specific JavaScript functions. In other words, even if all your automated end-to-end tests written with a tool like HttpUnit/JWebUnit/HtmlUnit/Canoo WebTest/etc. pass without a single failure, you can't really say that the same tests would pass if executed manually using a specific browser like IE or Firefox. In short, these are great tools but mostly suitable for applications that are strict about using only those features of JavaScript that are supported by the underlying JavaScript engine.

The power of the latter group--those tools that control a real browser--obviously is in how they use the real thing for testing and thus give a better representation of reality in terms of JavaScript. The downside is that executing tests through the browser is rather difficult to do with current tools in a platform/browser independent manner. For example, WTR, PAMIE, SAMIE, and Jiffie all let you control Internet Explorer on Windows but none of those tools work on Linux, for example, nor do they have any kind of support for Mozilla/Firefox. Selenium is platform independent as it supports Internet Explorer, Mozilla/Firefox, and Safari on Windows, Linux, and OS X (where applicable) but it has its own issues (the "TestRunner mode" requires you to manually start your server, launch a browser, and click a button in order to run the tests; the "Driven mode" lets you write your tests in Java/Ruby/Python/etc and execute them with the same tools you use to run your unit tests without manual steps but the driven mode is significantly slower to execute than any of the alternatives (due to its rather exotic architecture).

Now, you can use either class of these tools for functional testing, i.e. verifying that the right things happen when a user clicks through a certain sequence of steps. For performance testing, you might consider wrapping your HttpUnit/JWebUnit/HtmlUnit tests with JPerfUnit or go with a specialized load testing tool such as Grinder or JMeter. There's no way to use one of the browser-controlling tools for performance testing in a meaningful way.

For security, your best bet is to write a couple of "regular" functional tests to verify that you can't access certain areas of the web application unless you've logged in but that's not really a security issue--just regular functionality. The so-called "real" security testing would involve a security consultant analysing your architecture and running penetration tests against your system with tools like Nessus.

Having said all that, I'd like to suggest that you test domain logic like salary calculations in your *unit* tests instead of functional tests.
 
Syed Saifuddin
Ranch Hand
Posts: 130
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello Lasse Koskela

I am very thankful to you for the detailed answer. One more thing I really need from you is the single name(not a verity of)framework to use because learning all these framework make extra burden.
If you please identify the name of one that cover max of the testing (security, performance and load) in web application it become easier for me to start.

Thankyou Again
Syed Saifuddin
 
Lasse Koskela
author
Posts: 11962
5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
None of the tools mentioned cover all your needs. That's a fact of life you'll just have to live with.

In order to cover most of your needs, I'd suggest looking into HttpUnit and JWebUnit. The latter is built on top of the former, offers a higher level of abstraction, and therefore a bit nicer API while still giving you (some) access to the underlying (more powerful) HttpUnit API.
 
Greenhorn
Posts: 10
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
There are a lot of test tools out there. For starters take a look at:

http://www.softwareqatest.com/qatweb1.html
 
Ranch Hand
Posts: 1934
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Irrespective of JSP or other technologies you have used to render your web content; I would suggest 'Load Runner' for a reasonable load test on you system.

For performance and memory aspects you can do profiling using JProfile or something similar.
 
Greenhorn
Posts: 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I've only just started looking into Selenium but I love it so far.
Regarding the comment about having to start TestRunner mode manually, this is from the documentation:

Continuous Integration

Selenium can be integrated with an automated build. When the parameter "auto=true" is added to the URL, Selenium will run the entire suite of tests, and then post the results to a handling URL. The default URL is "/postResults", but an alternative handler location can be provided by specifying a "resultsUrl" parameter.


Therefore, the steps for continuous integration are:

1.

Create a servlet-type application at the url /postResults which can read the parameters above and write them to a file
2.

Create a script which can start up a brower and send to to the URL: selenium?auto=true
*

Generally, this can be done by merely calling the browser with the URL as an argument:

firefox.exe http://localhost/selenium?auto=true

3.

Make your continuous build:
* Call the script from step 2, preferably using more than one browser
* Wait for it to finish, possibly by checking for the existence of the file(s) from step 1
* Parse these files to determine whether the build passed or failed
* Act accordingly (send emails, update a build web page, etc.)
 
See ya later boys, I think I'm in love. Oh wait, she's just a tiny ad:
a bit of art, as a gift, the permaculture playing cards
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic