GeeCON Prague 2014*
The moose likes Other Open Source Projects and the fly likes http client + thread save conn manager + execute get request time problems Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


JavaRanch » Java Forums » Products » Other Open Source Projects
Bookmark "http client + thread save conn manager + execute get request time problems" Watch "http client + thread save conn manager + execute get request time problems" New topic
Author

http client + thread save conn manager + execute get request time problems

mariqn nenchev
Greenhorn

Joined: Apr 17, 2009
Posts: 5
Hi, all.
I am new with httpclient. I am using version 4. What i am trying to do
is the following:
Every 10 seconds i have to send usually about 10-15 get requsets. Every
Request returns me a response as (small) xml content that i parse and use.
So i create httpclient singleton with ThreadSaveClientConnManager, and i
have MyThread class that in its run method create HttpGet and the client
executes it. I have ThreadManager that starts the constructed threads.I
printed the times for building and executing the requests and they seems
to increase, and i can't understand why. Am i missing something, or i
don't free resources?
Here is my code:

In the thread class run method i do this:
HttpGet request = new HttpGet(query);// where query is some uri string
in format http://sample.com?id=1&id=2..........&id=50
HttpResponse response = httpClient.execute(request);
stream = response.getEntity().getContent(); // this input
stream is the xml i need.
in the finally block i only close the inputstream.

The docs says that if i close the input stream i close the underling
connection to be reused by the client is that right? Notice that my http
client and thread conn manager are singletons.
An the problem is that every request time is increased, but i can't
understand why some leak maybe, i don't free resources?!?
Here is the time in milliseconds for my first 2 threads:
first portion of get requests(the requests must be send every 10 seconds)
Createing and executing get for: 136
Createing and executing get for: 136
Createing and executing get for: 380
Createing and executing get for: 466
Createing and executing get for: 542
second portion of get requests(the requests must be send every 10 seconds)
Createing and executing get for: 677
Createing and executing get for: 755
Createing and executing get for: 784
Createing and executing get for: 1010
Createing and executing get for: 1015
third portion of get requests(the requests must be send every 10 seconds)
....here the execution of the request is still increasing. >1 sec


I have to say that the time is very important for the application. The
first requests times are acceptable, but they seems to increase. I
printed only the time in witch the request is constructed from query
string and is executed by the httpclient.
I also used debug config for that httpclient and here is part of it:
[DEBUG] ConnPoolByRoute - Notifying thread waiting on pool [HttpRoute[{}->http://sample.com]]
Could this be the reason that evey next request takes mote time? What the above means?
Please could you help with suggestions.
Regards.
Martijn Verburg
author
Bartender

Joined: Jun 24, 2003
Posts: 3274
    
    5

Hi there!

Could you edit your post and UseCodeTags? That'll help us read your question better


Cheers, Martijn - Blog,
Twitter, PCGen, Ikasan, My The Well-Grounded Java Developer book!,
My start-up.
mariqn nenchev
Greenhorn

Joined: Apr 17, 2009
Posts: 5
OK, here you are. I posted the code in code tags, now could you please help me.

Could you edit your post and UseCodeTags? That'll help us read your question better
Martijn Verburg
author
Bartender

Joined: Jun 24, 2003
Posts: 3274
    
    5

Hi there!

Have you read the following articles on HttpClient and multithreading requests?

threading
performance
mariqn nenchev
Greenhorn

Joined: Apr 17, 2009
Posts: 5
OK, i read it and i added this param http.connection.stalecheck = false, i can't determine if there is some improvement in the performance time.
But my question is why every next request has bigger time to execute, please have a more detailed look at my first post. I think that the problem may be hidden in this debug line:
2009/04/21 14:23:05:789 EEST [DEBUG] ConnPoolByRoute - Notifying thread waiting on pool [HttpRoute[{}->http://api.intrade.com]]
What does it mean? The Thread is waiting to get connection from the pool, and that is why every next request is executed with bigger time?
Can you tell me how to initialize 5 connections that are ready to use immediately, because i think that the http client looks at the pool for connection and it either wait connection to be released or creates new on demand. How can i create for example 5 connections and if more are needed that the client will create them on demand. Or may be i am wrong, have a look at my code please.
Martijn Verburg
author
Bartender

Joined: Jun 24, 2003
Posts: 3274
    
    5

It looks like from your logging that you're creating 5 connections per thread, don't you want a connection per thread?
mariqn nenchev
Greenhorn

Joined: Apr 17, 2009
Posts: 5
My fault, the log message is not correct. By that i mean th main thread that must wait for the other five threads to finish and that time = the longest request execution time. I will edit my first message.
Martijn Verburg wrote:It looks like from your logging that you're creating 5 connections per thread, don't you want a connection per thread?
 
 
subject: http client + thread save conn manager + execute get request time problems