• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

any strong opinions on web load testing

 
Ranch Hand
Posts: 35
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
All,

I'm working on my STP Magazine Best Practices column and thought I'd check in to JavaRanch. The topic du jour is Web load testing. I'm wondering if there any strong opinions out there on the right and wrong ways to do Web load testing?

I'm looking for folks who wouldn't mind being quoted in the column, though I'm happy to be vague in attributions or even take stuff off the record if that's your preference. I guess one thing I'm curious about: say you've run your Web site for years on expensive and well-tested hardware and software. Now you're looking to move to low cost, commodity servers and possibly more open source software. How do you start?

Thanks for any help on this. Deadline is the end of next week.
 
author & internet detective
Posts: 41860
908
Eclipse IDE VI Editor Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Geoff,
In my mind, the most important thing is to come up with realistic scenarios of what should be tested. If it isn't representative of what real users do, the results are meaningless. This includes having simulated users do different tasks on the website and come up with different percentages of each task to mix in.

You don't want to turn your load test into a stress test. A stress test will tell you if the machine will crash, but not how it will respond for normal users.

In the case of an existing website, there is an advantage as you can take baseline numbers. Then you can easily see if the performance has degraded with the new hardware/software.

Jeanne Boyarsky
Developer at a bank in New York City

[edited to fix typo]
[ July 05, 2006: Message edited by: Jeanne Boyarsky ]
 
Geoff Koch
Ranch Hand
Posts: 35
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks, Jeanne. Are you seeing any sort of trend towards deployment of low-cost commodity hardware and freeware/open source software to run Web sites and Web applications?
 
author
Posts: 11962
5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Geoff Koch:
Thanks, Jeanne. Are you seeing any sort of trend towards deployment of low-cost commodity hardware and freeware/open source software to run Web sites and Web applications?


We (Reaktor, a professional services company in Finland) certainly are seeing some kind of a trend. Even our finance domain clients are increasingly deploying systems on open source platforms and, in some cases, on commodity hardware.
 
Jeanne Boyarsky
author & internet detective
Posts: 41860
908
Eclipse IDE VI Editor Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Geoff Koch:
Thanks, Jeanne. Are you seeing any sort of trend towards deployment of low-cost commodity hardware and freeware/open source software to run Web sites and Web applications?


I don't have exposure to enough systems to comment on this. Google is famous for low-cost commodity hardware though. The whole grid computing thing!
 
Geoff Koch
Ranch Hand
Posts: 35
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Do you have two or three sentences of wisdom, Lasse, for folks curious about how load testing will change when moving from proprietary/expensive hardware and software to commodity/open source platforms? Thanks!
 
Lasse Koskela
author
Posts: 11962
5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Geoff Koch:
Do you have two or three sentences of wisdom, Lasse, for folks curious about how load testing will change when moving from proprietary/expensive hardware and software to commodity/open source platforms? Thanks!


Sentences of wisdom? Umm, not really. Mainly because I see no difference in the function of load testing systems deployed on Dell hardware instead of Sun hardware, for example. The technical interface through which load testing is performed is practically the same (HTTP for web apps, for example).

Sure, the lower cost of commodity hardware might have an effect on how willing companies are to build multiple testing environments in full scale (mirroring production) but there's still the additional cost of hardware SSL accelerators, single sign-on licenses, etc. which still make up a price tag way higher than the proprietary hardware was.

Having said that, open source platforms do make it somewhat easier to analyze performance bottlenecks because there's an open community to turn to instead of a commercial vendor with non-existent support.
 
Geoff Koch
Ranch Hand
Posts: 35
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Interesting. So whether you have many cheap rack-mounted commodity servers or a few expensive proprietary boxes from Sun or IBM, load testing procedures are the same? I would have thought that "sizing" the Web server solution you need would be easier working with commercial vendors, which theoretically provide engineering support or at the very least, lots of documentation. No?
 
Lasse Koskela
author
Posts: 11962
5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Geoff Koch:
cheap rack-mounted commodity servers or a few expensive proprietary boxes from Sun or IBM, ...


They're practically all rack-mounted these days

Originally posted by Geoff Koch:
I would have thought that "sizing" the Web server solution you need would be easier working with commercial vendors, which theoretically provide engineering support or at the very least, lots of documentation. No?


Sure, you can get engineering support for right-sizing your cluster but those poor IBM/Sun guys need to make their suggestions based on the same limited knowledge of your system's future requirements. The problem comes down to having prior experience with similar systems and, unfortunately, many of the sales engineers that you get have not had the follow-up necessary with their previous sales to know whether they were anywhere near the target.

In other words (and in my opinion), it's better to ask from Someone Who Knows rather than Someone Who You'd Think Would Know. The big question is whether you're more likely to get such someone through the hardware vendor, some big name consultancy, a smaller consultancy, or from your own IT ops... I'd personally recommend asking your own operations people first because they're more likely to have a clue based on the company's other deployments--assuming that there are other systems running on the same hardware already.
 
Geoff Koch
Ranch Hand
Posts: 35
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks, Lasse. I like that advice and do think DIY has merit. Thanks for your help on this!
 
Geoff Koch
Ranch Hand
Posts: 35
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Actually, I'm sorry to ask one additional question here, but I'm curious: how (if at all) is AJAX and Web 2.0 changing Web load testing? It seems like Web load testing basically has always been about 1) availability and 2) response time. So has response time essentially been "solved"? Or, is Web load testing related to response time primarily now about non-application issues, like I/O or database bottlenecks, etc.? Or, is there anything (re: Web load testing) that did work pre-AJAX but does not work with advent of AJAX? Thanks.
 
reply
    Bookmark Topic Watch Topic
  • New Topic