• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Max concurrent requests on Tomcat

 
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello all,
I'm pretty new to the whole multi-threading high performance issues so please excuse me if I confuse things.
I have a web-app which uses long polling (a client sends a request to the server and returns until there's new data or timeout and then requests again) because we need to be updated allways. For many reasons I can't use Tomcat's advanced NIO or Jetty continuations etc...
So now I have an app where I have to be able to serve a 1000 concurrent users which, as I understand it, will be sending a request and tying up a 1000 threads (please correct me if I'm wrong here).
My question is basically this:
How many concurrent threads can the Tomcat hold? Is 300 a reasonable number?
What kind of hardware requirements do I need from a server if I want it to hold 4 Tomcat instances and a load-balancer while remembering that I'll have around 1100 open threads at all times?

Thank you for any help you can give me and if any additional info is needed please don't hesitate to ask
Ittai
 
Ranch Hand
Posts: 263
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
In my opinion 1000 is a very high number.

200-300 should be ok, and max number of request threads inside Tomcat can be configured.

Have you considered the following:
1. Instead of having long polling, why not poll the server every few seconds. This is likely to give you better scalability.
2. Load Balancing is a good idea.
A typical configuration could be 3 machines. One which has Apache (load balancer), and two machines behind it, each having 2 instances of Tomcat on it. A multiple CPU machine would be good. Usually the number of threads you can handle is significantly dependent on the speed and number of CPUs you have.

Just curious, are you using Blaze-DS/LCDS (Adobe real-time collaboration servers) by any chance. (They have a channel which is called amf/long-polling)?

 
Ittai Zeidman
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi James,
First of all thank you for your reply.

James Ward wrote:In my opinion 1000 is a very high number.


This is what I need to be able to service, not what I want one instance to handle on its own

James Ward wrote:
200-300 should be ok


Do you know if it's more towards the 200 or the 300? As obviously 200 will be a bit of a problem for me

James Ward wrote:
1. Instead of having long polling, why not poll the server every few seconds. This is likely to give you better scalability.


I have considered polling but I can't poll every few seconds I need my information to be updated by a maximum of 500 milliseconds, give or take a few milliseconds, so polling will bring me to 2000 request per second which will clutter the network and problematic for me.

James Ward wrote:
A typical configuration could be 3 machines. One which has Apache (load balancer), and two machines behind it, each having 2 instances of Tomcat on it. A multiple CPU machine would be good. Usually the number of threads you can handle is significantly dependent on the speed and number of CPUs you have.


A colleague of mine told me that all the instances and load-balancer can all be on the same machine, that's incorrect?
I'm not really sure yet what kind of budget will have for the machine(s) but from what I gather we'll be able to spend for one very powerfull machine and not more.

James Ward wrote:
Just curious, are you using Blaze-DS/LCDS (Adobe real-time collaboration servers) by any chance. (They have a channel which is called amf/long-polling)?


No.


Thanks,
Ittai
 
Saloon Keeper
Posts: 27762
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
sustained 500ms or less responses for web requests is pushing it. There's too much overhead, and none of the mechanisms - including HTML itself - were designed for ultrahigh performance. This is a protocol that was intended to permit humans to obtain information over a long distance through a chain of routers, switches and hosts, and although good HTTP apps should be able to provide turnaround in no more than a few seconds, milliseconds is probably asking too much.

For something like that, I'd recommend a more interactive, lower-overhead protocol such as RMI, where you could embed a client connection in an applet. Although realistically, I'm not sure that you can expect the humans looking at the displays to be capable of taking advantage of sub-second updating over extended periods of time.
 
James Ward
Ranch Hand
Posts: 263
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

A colleague of mine told me that all the instances and load-balancer can all be on the same machine, that's incorrect?


Yes ofcourse you can put all in one powerful machine.

Do you know if it's more towards the 200 or the 300? As obviously 200 will be a bit of a problem for me


Depends on underlying hardware too, but 300 should be ok.
 
Tim Holloway
Saloon Keeper
Posts: 27762
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

James Ward wrote:

A colleague of mine told me that all the instances and load-balancer can all be on the same machine, that's incorrect?


Yes ofcourse you can put all in one powerful machine.

Do you know if it's more towards the 200 or the 300? As obviously 200 will be a bit of a problem for me


Depends on underlying hardware too, but 300 should be ok.



As a general rule, however, you shouldn't put more Tomcat instances on one machine than you have CPU cores. Otherwise they steal resources from each other. Actually, since Tomcat has multiple threads, they'll steal a few resources anyway, but the more instances, the more theft will occur.
 
Ittai Zeidman
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote:sustained 500ms or less responses for web requests is pushing it. There's too much overhead, and none of the mechanisms - including HTML itself - were designed for ultrahigh performance. This is a protocol that was intended to permit humans to obtain information over a long distance through a chain of routers, switches and hosts, and although good HTTP apps should be able to provide turnaround in no more than a few seconds, milliseconds is probably asking too much.

For something like that, I'd recommend a more interactive, lower-overhead protocol such as RMI, where you could embed a client connection in an applet. Although realistically, I'm not sure that you can expect the humans looking at the displays to be capable of taking advantage of sub-second updating over extended periods of time.


Hi Tim, I understand what you're saying; I think I might have exaggerated before with the maximum of 500ms, although 500ms is what we're aiming for. So, yeah you're right that trying for less is pushing it but I think 500ms is doable.
Secondly it's not that we expect humans to sit and wait for the updates but rather that there won't be more than a half a second (give or take a few ms) before updates appear on the screen as this is a Life supporting software.

And thank you for your additional info.

@James, thank you for your replies.

You two have been very helpful.

Thanks,
Ittai
 
Greenhorn
Posts: 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Ittai,

What solution did you end up going for in this case as I have similar requirements?

Cheers,

Matt
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic