This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes Tomcat and the fly likes Max concurrent requests on Tomcat Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Products » Tomcat
Bookmark "Max concurrent requests on Tomcat" Watch "Max concurrent requests on Tomcat" New topic
Author

Max concurrent requests on Tomcat

Ittai Zeidman
Greenhorn

Joined: Sep 08, 2009
Posts: 6
Hello all,
I'm pretty new to the whole multi-threading high performance issues so please excuse me if I confuse things.
I have a web-app which uses long polling (a client sends a request to the server and returns until there's new data or timeout and then requests again) because we need to be updated allways. For many reasons I can't use Tomcat's advanced NIO or Jetty continuations etc...
So now I have an app where I have to be able to serve a 1000 concurrent users which, as I understand it, will be sending a request and tying up a 1000 threads (please correct me if I'm wrong here).
My question is basically this:
How many concurrent threads can the Tomcat hold? Is 300 a reasonable number?
What kind of hardware requirements do I need from a server if I want it to hold 4 Tomcat instances and a load-balancer while remembering that I'll have around 1100 open threads at all times?

Thank you for any help you can give me and if any additional info is needed please don't hesitate to ask
Ittai
James Ward
Ranch Hand

Joined: Apr 27, 2003
Posts: 263
In my opinion 1000 is a very high number.

200-300 should be ok, and max number of request threads inside Tomcat can be configured.

Have you considered the following:
1. Instead of having long polling, why not poll the server every few seconds. This is likely to give you better scalability.
2. Load Balancing is a good idea.
A typical configuration could be 3 machines. One which has Apache (load balancer), and two machines behind it, each having 2 instances of Tomcat on it. A multiple CPU machine would be good. Usually the number of threads you can handle is significantly dependent on the speed and number of CPUs you have.

Just curious, are you using Blaze-DS/LCDS (Adobe real-time collaboration servers) by any chance. (They have a channel which is called amf/long-polling)?

Ittai Zeidman
Greenhorn

Joined: Sep 08, 2009
Posts: 6
Hi James,
First of all thank you for your reply.

James Ward wrote:In my opinion 1000 is a very high number.

This is what I need to be able to service, not what I want one instance to handle on its own

James Ward wrote:
200-300 should be ok

Do you know if it's more towards the 200 or the 300? As obviously 200 will be a bit of a problem for me

James Ward wrote:
1. Instead of having long polling, why not poll the server every few seconds. This is likely to give you better scalability.

I have considered polling but I can't poll every few seconds I need my information to be updated by a maximum of 500 milliseconds, give or take a few milliseconds, so polling will bring me to 2000 request per second which will clutter the network and problematic for me.

James Ward wrote:
A typical configuration could be 3 machines. One which has Apache (load balancer), and two machines behind it, each having 2 instances of Tomcat on it. A multiple CPU machine would be good. Usually the number of threads you can handle is significantly dependent on the speed and number of CPUs you have.

A colleague of mine told me that all the instances and load-balancer can all be on the same machine, that's incorrect?
I'm not really sure yet what kind of budget will have for the machine(s) but from what I gather we'll be able to spend for one very powerfull machine and not more.

James Ward wrote:
Just curious, are you using Blaze-DS/LCDS (Adobe real-time collaboration servers) by any chance. (They have a channel which is called amf/long-polling)?

No.


Thanks,
Ittai
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 15961
    
  19

sustained 500ms or less responses for web requests is pushing it. There's too much overhead, and none of the mechanisms - including HTML itself - were designed for ultrahigh performance. This is a protocol that was intended to permit humans to obtain information over a long distance through a chain of routers, switches and hosts, and although good HTTP apps should be able to provide turnaround in no more than a few seconds, milliseconds is probably asking too much.

For something like that, I'd recommend a more interactive, lower-overhead protocol such as RMI, where you could embed a client connection in an applet. Although realistically, I'm not sure that you can expect the humans looking at the displays to be capable of taking advantage of sub-second updating over extended periods of time.


Customer surveys are for companies who didn't pay proper attention to begin with.
James Ward
Ranch Hand

Joined: Apr 27, 2003
Posts: 263
A colleague of mine told me that all the instances and load-balancer can all be on the same machine, that's incorrect?

Yes ofcourse you can put all in one powerful machine.

Do you know if it's more towards the 200 or the 300? As obviously 200 will be a bit of a problem for me

Depends on underlying hardware too, but 300 should be ok.
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 15961
    
  19

James Ward wrote:
A colleague of mine told me that all the instances and load-balancer can all be on the same machine, that's incorrect?

Yes ofcourse you can put all in one powerful machine.

Do you know if it's more towards the 200 or the 300? As obviously 200 will be a bit of a problem for me

Depends on underlying hardware too, but 300 should be ok.


As a general rule, however, you shouldn't put more Tomcat instances on one machine than you have CPU cores. Otherwise they steal resources from each other. Actually, since Tomcat has multiple threads, they'll steal a few resources anyway, but the more instances, the more theft will occur.
Ittai Zeidman
Greenhorn

Joined: Sep 08, 2009
Posts: 6
Tim Holloway wrote:sustained 500ms or less responses for web requests is pushing it. There's too much overhead, and none of the mechanisms - including HTML itself - were designed for ultrahigh performance. This is a protocol that was intended to permit humans to obtain information over a long distance through a chain of routers, switches and hosts, and although good HTTP apps should be able to provide turnaround in no more than a few seconds, milliseconds is probably asking too much.

For something like that, I'd recommend a more interactive, lower-overhead protocol such as RMI, where you could embed a client connection in an applet. Although realistically, I'm not sure that you can expect the humans looking at the displays to be capable of taking advantage of sub-second updating over extended periods of time.

Hi Tim, I understand what you're saying; I think I might have exaggerated before with the maximum of 500ms, although 500ms is what we're aiming for. So, yeah you're right that trying for less is pushing it but I think 500ms is doable.
Secondly it's not that we expect humans to sit and wait for the updates but rather that there won't be more than a half a second (give or take a few ms) before updates appear on the screen as this is a Life supporting software.

And thank you for your additional info.

@James, thank you for your replies.

You two have been very helpful.

Thanks,
Ittai
Matthew Dyer
Greenhorn

Joined: Nov 22, 2009
Posts: 1
Hi Ittai,

What solution did you end up going for in this case as I have similar requirements?

Cheers,

Matt
 
It is sorta covered in the JavaRanch Style Guide.
 
subject: Max concurrent requests on Tomcat
 
Similar Threads
Webapps thread basic questions
how many users tomcat support concurrently
Creation of Servlet instances
WebServer's process management
Thread Pool Management in SocketServer.