Monitoring other objects

[Mark Herschberg]

Short version:
I need to monitor the time it takes for a number of instances of of an interface to perform some task.
Longer version:
I'm creating a test harness. We're going to simulate N clients on a single machine (more likely A clients across B machines, where B << A). A client sends a message (proprietary protocol) to the server being load tested, and expects a response. Our testing requires putting the server under a load defined by the numbers of users and or the frequency of messages sent to the server. Each client machine (which are centrally coordinated) will simulate it's share of clients by running multiple threads.
There are three things I need to measure:
1) Static client load
The client itself must do some work to create the message and send it, and to process the response. We will need to have the program calculate this, before the testing of the server begins, to the test harness can detrmine how many client PCs are needed in order to run at a given rate.
2) Dynamic load
Our static test (1) may not be accurate. We need to dyanmically monitor how long each client takes, to insure that the load we wanted to test is the load we actually applied.
3) Time
Finally, we need to simply measure how long it took for the server action to be performed, from the simulated client's perspective, so we can make measurements. (We'd probably need to do this for (2), anyway.)

Any ideas or patterns?
I could simply wrap all calls in a timer start and stop class for (3). Is there a better way?
For (2), there's the issue that (3) includes "dead time" when the client is waiting for the server to do it's work and reply, so it's not the true load factor. In that case I would need to wrap the sending and receving calls with a timer. is there a batter way?
For (1) I can simply run a single case of step (2). I'm making the assumption (which I think is reasonable in this case) that other resources (CPU, memory, badnwidth), won't be limiting factors when we scale up. Anything else I might be missing? I may need to take into account he thread swapping overhead.
Any other general ideas on ths topic? I'm also thinking of precomputing most work and storing it in memory (which I think I can do), to allow for a lighter client-side load.

--Mark
[e-mail]

[Junilu Lacar]

Mark,
Is buy (vs. build) an option? If it is, Compuware Corp has some pretty good test tools that do the kind of thing you are looking to do.
Junilu

[This message has been edited by JUNILU LACAR (edited May 29, 2001).]

[Mark Herschberg]

Originally posted by JUNILU LACAR:
Is buy (vs. build) an option? If it is, Compuware Corp has some pretty good test tools that do the kind of thing you are looking to do.

Buy is definately an option. However, most QA products (e.g. Mercury's, Rational) on the market are deisgned for load testing web servers (because everyone and their mom had an eCommerce site these last few years, so that's where the money way). They are not very good at loa testing proprietary client-server applications, let alone frameworks. While most technically can do it, it's not much saved effort from a custom system, and we get much more flexibility with the latter.
Looking at Compuware Corp's web site, what I want is a product like, http://www.compuware.com/products/strobe/package.htm but this is made for "OS/390-based applications."

--Mark
[e-mail]