File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Performance and the fly likes Can not get stable result when using multiple threads to calculate the average response time Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Performance
Bookmark "Can not get stable result when using multiple threads to calculate the average response time" Watch "Can not get stable result when using multiple threads to calculate the average response time" New topic
Author

Can not get stable result when using multiple threads to calculate the average response time

tuntony wei
Greenhorn

Joined: Jun 29, 2011
Posts: 13
Hi, guys:
I met a problem when I use multple threads to calculate the avaerage reponse time. My problem is I can not get the stable result, I put my code here and could anybody give me some suggestion.

1. My goal is to test the performance of a compnenet in our product. This is a J2ee product. Based on design, it is able to handle concurrent user's request.
2. my idea is to make a simple HTML, a link on the HTML can invoke one performance test case. In the test case, I record start time before the component starting to do the business logic, then I record the end time as soon as the component finished the job. So my response time = end - start. Finally I put it into a result.csv file.
3. I use JMeter to generate 100 concurrent user request to the HTML

Here is my code:

1. HTML:
<a href="TestServlet?command=KPI001" target="_self">KPI001</a>

2. TestServlet.java

3. TargetPerformance.java


My result:
if I run 100 user request through Jmeter 1 by 1, then my average response time is 2+ seconds, but if I run 100 concurrent user through Jmeter, then my average response time was around 4 seconds. This makes me worried and confused

Question:
1. Dirrectly feeling, should I worry about the start tiem and end time? When mutiple users come into the startTest method, is there any risk for them to be re-written?
2. In my code, I did not use any synchronized method/block because I think I do not need to. My understanding is that when concurrent users make requests, the TestServlet always did "TargetPerformance perf = new TargetPerformance ()", and then when the users did the "perf.startTest()", different user alwasy has its own copy of start time, end time and test result.
3. This is what I think. I think it should be this way but this part is also the part I worried most
Am I right? Could anybody give me some suggestion?
Thank you so so so much
Himanshu Gupta
Ranch Hand

Joined: Aug 18, 2008
Posts: 598

if I run 100 user request through Jmeter 1 by 1, then my average response time is 2+ seconds, but if I run 100 concurrent user through Jmeter, then my average response time was around 4 seconds. This makes me worried and confused


When you run one by run do you send the other request just after the first one or you wait to get the results of the previous request?

When you are running 100 concurrent threads you are increasing the load on the server. The request flood comes and server starts handling them. When it is one by one the server gets more time to process the 100 request.

My Blog SCJP 5 SCWCD 5
tuntony wei
Greenhorn

Joined: Jun 29, 2011
Posts: 13
Himanshu Gupta wrote:
if I run 100 user request through Jmeter 1 by 1, then my average response time is 2+ seconds, but if I run 100 concurrent user through Jmeter, then my average response time was around 4 seconds. This makes me worried and confused


When you run one by run do you send the other request just after the first one or you wait to get the results of the previous request?

When you are running 100 concurrent threads you are increasing the load on the server. The request flood comes and server starts handling them. When it is one by one the server gets more time to process the 100 request.


When I run it one by one, I run it through JMeter. Basically, Jmeter send the 1st requst to HTML and then wait for a HTTP code such as 200. And then, Jmeter will send the 2nd and so on... With this method, my response time was around 2 seconds

With concurrent threads, the response time I got was around 4 seconds. But my understanding is that under the concurrent test, the response time should be close to the 2 seconds. That's why I worried about if statime/endtime were messed up when multiple threads comming. Do you think my code, or my test method, is OK? Do you mean the longer response time supposes to happen when dealing with the multple threads?

Thank you so much
steve souza
Ranch Hand

Joined: Jun 26, 2002
Posts: 852
Being as your start and stop times are local variables in TargetPerformance your code is thread safe. Why would you be surprised that 100 simultaneous users would be slower than 1 sequential one. I would expect that. You should get an idea about this being correct simply by timing with a wall clock the total time of your tests and dividing by 100 to get average time.

Having said that I would use JAMon (or other tool of your choice) to do the monitoring. It is much more flexible. It can easily monitor your business methods (MonitorFactory.start("doKPI001()") and do a lot more than that such as monitor your jdbc, page hits, ejb calls without changing your code at all. And for this reason it allows you to monitor in production as well as dev which is where your problems are likely to occur.

In fact I just made a release today that also lets your monitor your unix/linux scripts with jamon by calling the unix scripts I provide such as jstart.sh/jstop.sh and jadd.sh (adds an arbitrary value).

Steve - www.jamonapi.com


http://www.jamonapi.com/ - a fast, free open source performance tuning api.
JavaRanch Performance FAQ
tuntony wei
Greenhorn

Joined: Jun 29, 2011
Posts: 13
steve souza wrote:Being as your start and stop times are local variables in TargetPerformance your code is thread safe. Why would you be surprised that 100 simultaneous users would be slower than 1 sequential one. I would expect that. You should get an idea about this being correct simply by timing with a wall clock the total time of your tests and dividing by 100 to get average time.

Having said that I would use JAMon (or other tool of your choice) to do the monitoring. It is much more flexible. It can easily monitor your business methods (MonitorFactory.start("doKPI001()") and do a lot more than that such as monitor your jdbc, page hits, ejb calls without changing your code at all. And for this reason it allows you to monitor in production as well as dev which is where your problems are likely to occur.

In fact I just made a release today that also lets your monitor your unix/linux scripts with jamon by calling the unix scripts I provide such as jstart.sh/jstop.sh and jadd.sh (adds an arbitrary value).

Steve - www.jamonapi.com


OK, great, now I am talking with our developers about my finding. And good to know the JAMon, we do need some good tool to monitor our KPI. I will try it. Thank you so much again
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Can not get stable result when using multiple threads to calculate the average response time
 
Similar Threads
InvalidMappingException
InvalidMappingException
strange.. !! plz help me..
InvalidMappingException
How to implement Request Time Out