This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes Testing and the fly likes any strong opinions on web load testing Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Engineering » Testing
Bookmark "any strong opinions on web load testing" Watch "any strong opinions on web load testing" New topic
Author

any strong opinions on web load testing

Geoff Koch
Ranch Hand

Joined: Mar 19, 2006
Posts: 35
All,

I'm working on my STP Magazine Best Practices column and thought I'd check in to JavaRanch. The topic du jour is Web load testing. I'm wondering if there any strong opinions out there on the right and wrong ways to do Web load testing?

I'm looking for folks who wouldn't mind being quoted in the column, though I'm happy to be vague in attributions or even take stuff off the record if that's your preference. I guess one thing I'm curious about: say you've run your Web site for years on expensive and well-tested hardware and software. Now you're looking to move to low cost, commodity servers and possibly more open source software. How do you start?

Thanks for any help on this. Deadline is the end of next week.


--------------------<br />Geoff Koch<br />Freelance technical writer, journalist<br />Lansing, MI<br />AIM: geoffinmichigan<br />Yahoo: geoff_pdx<br />MSN: geoffarnoldkoch
Jeanne Boyarsky
internet detective
Marshal

Joined: May 26, 2003
Posts: 30136
    
150

Geoff,
In my mind, the most important thing is to come up with realistic scenarios of what should be tested. If it isn't representative of what real users do, the results are meaningless. This includes having simulated users do different tasks on the website and come up with different percentages of each task to mix in.

You don't want to turn your load test into a stress test. A stress test will tell you if the machine will crash, but not how it will respond for normal users.

In the case of an existing website, there is an advantage as you can take baseline numbers. Then you can easily see if the performance has degraded with the new hardware/software.

Jeanne Boyarsky
Developer at a bank in New York City

[edited to fix typo]
[ July 05, 2006: Message edited by: Jeanne Boyarsky ]

[Blog] [JavaRanch FAQ] [How To Ask Questions The Smart Way] [Book Promos]
Blogging on Certs: SCEA Part 1, Part 2 & 3, Core Spring 3, OCAJP, OCPJP beta, TOGAF part 1 and part 2
Geoff Koch
Ranch Hand

Joined: Mar 19, 2006
Posts: 35
Thanks, Jeanne. Are you seeing any sort of trend towards deployment of low-cost commodity hardware and freeware/open source software to run Web sites and Web applications?
Lasse Koskela
author
Sheriff

Joined: Jan 23, 2002
Posts: 11962
    
    5
Originally posted by Geoff Koch:
Thanks, Jeanne. Are you seeing any sort of trend towards deployment of low-cost commodity hardware and freeware/open source software to run Web sites and Web applications?

We (Reaktor, a professional services company in Finland) certainly are seeing some kind of a trend. Even our finance domain clients are increasingly deploying systems on open source platforms and, in some cases, on commodity hardware.


Author of Test Driven (2007) and Effective Unit Testing (2013) [Blog] [HowToAskQuestionsOnJavaRanch]
Jeanne Boyarsky
internet detective
Marshal

Joined: May 26, 2003
Posts: 30136
    
150

Originally posted by Geoff Koch:
Thanks, Jeanne. Are you seeing any sort of trend towards deployment of low-cost commodity hardware and freeware/open source software to run Web sites and Web applications?

I don't have exposure to enough systems to comment on this. Google is famous for low-cost commodity hardware though. The whole grid computing thing!
Geoff Koch
Ranch Hand

Joined: Mar 19, 2006
Posts: 35
Do you have two or three sentences of wisdom, Lasse, for folks curious about how load testing will change when moving from proprietary/expensive hardware and software to commodity/open source platforms? Thanks!
Lasse Koskela
author
Sheriff

Joined: Jan 23, 2002
Posts: 11962
    
    5
Originally posted by Geoff Koch:
Do you have two or three sentences of wisdom, Lasse, for folks curious about how load testing will change when moving from proprietary/expensive hardware and software to commodity/open source platforms? Thanks!

Sentences of wisdom? Umm, not really. Mainly because I see no difference in the function of load testing systems deployed on Dell hardware instead of Sun hardware, for example. The technical interface through which load testing is performed is practically the same (HTTP for web apps, for example).

Sure, the lower cost of commodity hardware might have an effect on how willing companies are to build multiple testing environments in full scale (mirroring production) but there's still the additional cost of hardware SSL accelerators, single sign-on licenses, etc. which still make up a price tag way higher than the proprietary hardware was.

Having said that, open source platforms do make it somewhat easier to analyze performance bottlenecks because there's an open community to turn to instead of a commercial vendor with non-existent support.
Geoff Koch
Ranch Hand

Joined: Mar 19, 2006
Posts: 35
Interesting. So whether you have many cheap rack-mounted commodity servers or a few expensive proprietary boxes from Sun or IBM, load testing procedures are the same? I would have thought that "sizing" the Web server solution you need would be easier working with commercial vendors, which theoretically provide engineering support or at the very least, lots of documentation. No?
Lasse Koskela
author
Sheriff

Joined: Jan 23, 2002
Posts: 11962
    
    5
Originally posted by Geoff Koch:
cheap rack-mounted commodity servers or a few expensive proprietary boxes from Sun or IBM, ...

They're practically all rack-mounted these days

Originally posted by Geoff Koch:
I would have thought that "sizing" the Web server solution you need would be easier working with commercial vendors, which theoretically provide engineering support or at the very least, lots of documentation. No?

Sure, you can get engineering support for right-sizing your cluster but those poor IBM/Sun guys need to make their suggestions based on the same limited knowledge of your system's future requirements. The problem comes down to having prior experience with similar systems and, unfortunately, many of the sales engineers that you get have not had the follow-up necessary with their previous sales to know whether they were anywhere near the target.

In other words (and in my opinion), it's better to ask from Someone Who Knows rather than Someone Who You'd Think Would Know. The big question is whether you're more likely to get such someone through the hardware vendor, some big name consultancy, a smaller consultancy, or from your own IT ops... I'd personally recommend asking your own operations people first because they're more likely to have a clue based on the company's other deployments--assuming that there are other systems running on the same hardware already.
Geoff Koch
Ranch Hand

Joined: Mar 19, 2006
Posts: 35
Thanks, Lasse. I like that advice and do think DIY has merit. Thanks for your help on this!
Geoff Koch
Ranch Hand

Joined: Mar 19, 2006
Posts: 35
Actually, I'm sorry to ask one additional question here, but I'm curious: how (if at all) is AJAX and Web 2.0 changing Web load testing? It seems like Web load testing basically has always been about 1) availability and 2) response time. So has response time essentially been "solved"? Or, is Web load testing related to response time primarily now about non-application issues, like I/O or database bottlenecks, etc.? Or, is there anything (re: Web load testing) that did work pre-AJAX but does not work with advent of AJAX? Thanks.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: any strong opinions on web load testing
 
Similar Threads
Load test a web site
Software Quality Assurance Planet
No of Users
JMeter for Use with Regression Testing (Or other RT apps)
automation/load testing of a swing application