• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

http client + thread save conn manager + execute get request time problems

 
mariqn nenchev
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi, all.
I am new with httpclient. I am using version 4. What i am trying to do
is the following:
Every 10 seconds i have to send usually about 10-15 get requsets. Every
Request returns me a response as (small) xml content that i parse and use.
So i create httpclient singleton with ThreadSaveClientConnManager, and i
have MyThread class that in its run method create HttpGet and the client
executes it. I have ThreadManager that starts the constructed threads.I
printed the times for building and executing the requests and they seems
to increase, and i can't understand why. Am i missing something, or i
don't free resources?
Here is my code:

In the thread class run method i do this:
HttpGet request = new HttpGet(query);// where query is some uri string
in format http://sample.com?id=1&id=2..........&id=50
HttpResponse response = httpClient.execute(request);
stream = response.getEntity().getContent(); // this input
stream is the xml i need.
in the finally block i only close the inputstream.

The docs says that if i close the input stream i close the underling
connection to be reused by the client is that right? Notice that my http
client and thread conn manager are singletons.
An the problem is that every request time is increased, but i can't
understand why some leak maybe, i don't free resources?!?
Here is the time in milliseconds for my first 2 threads:
first portion of get requests(the requests must be send every 10 seconds)
Createing and executing get for: 136
Createing and executing get for: 136
Createing and executing get for: 380
Createing and executing get for: 466
Createing and executing get for: 542
second portion of get requests(the requests must be send every 10 seconds)
Createing and executing get for: 677
Createing and executing get for: 755
Createing and executing get for: 784
Createing and executing get for: 1010
Createing and executing get for: 1015
third portion of get requests(the requests must be send every 10 seconds)
....here the execution of the request is still increasing. >1 sec


I have to say that the time is very important for the application. The
first requests times are acceptable, but they seems to increase. I
printed only the time in witch the request is constructed from query
string and is executed by the httpclient.
I also used debug config for that httpclient and here is part of it:
[DEBUG] ConnPoolByRoute - Notifying thread waiting on pool [HttpRoute[{}->http://sample.com]]
Could this be the reason that evey next request takes mote time? What the above means?
Please could you help with suggestions.
Regards.
 
Martijn Verburg
author
Bartender
Posts: 3275
5
Eclipse IDE Java Mac OS X
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi there!

Could you edit your post and UseCodeTags? That'll help us read your question better
 
mariqn nenchev
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
OK, here you are. I posted the code in code tags, now could you please help me.

Could you edit your post and UseCodeTags? That'll help us read your question better
 
Martijn Verburg
author
Bartender
Posts: 3275
5
Eclipse IDE Java Mac OS X
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi there!

Have you read the following articles on HttpClient and multithreading requests?

threading
performance
 
mariqn nenchev
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
OK, i read it and i added this param http.connection.stalecheck = false, i can't determine if there is some improvement in the performance time.
But my question is why every next request has bigger time to execute, please have a more detailed look at my first post. I think that the problem may be hidden in this debug line:
2009/04/21 14:23:05:789 EEST [DEBUG] ConnPoolByRoute - Notifying thread waiting on pool [HttpRoute[{}->http://api.intrade.com]]
What does it mean? The Thread is waiting to get connection from the pool, and that is why every next request is executed with bigger time?
Can you tell me how to initialize 5 connections that are ready to use immediately, because i think that the http client looks at the pool for connection and it either wait connection to be released or creates new on demand. How can i create for example 5 connections and if more are needed that the client will create them on demand. Or may be i am wrong, have a look at my code please.
 
Martijn Verburg
author
Bartender
Posts: 3275
5
Eclipse IDE Java Mac OS X
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It looks like from your logging that you're creating 5 connections per thread, don't you want a connection per thread?
 
mariqn nenchev
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
My fault, the log message is not correct. By that i mean th main thread that must wait for the other five threads to finish and that time = the longest request execution time. I will edit my first message.
Martijn Verburg wrote:It looks like from your logging that you're creating 5 connections per thread, don't you want a connection per thread?
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic