I would think that the choke point isn't going to be Tomcat, but rather your limited number of downstream connections.
If you can only connect 9 requests at a time, then Tomcat is going to have to park further requests somehow. And I hope that you're not simply doing a wait() or spin operation until a downstream connection comes available!
A spin would be awful, since you're burning CPU to little purpose, but a wait is bad, too. Aside from the fact that it's expressly forbidden by J2EE standards, it locks up the thread running the request, which can thus back up the request thread pool, and eventually the input connection pool. Expanding those parameters simply allows the backup to spread even further, not increase your throughput rate.
So if you must park requests, consider queuing them instead. Using the ServletContextListener you can create an "engine" at application start-up time and by sharing a (synchronized) queue with the servlets, servlets can dump requests into the engine queue and query the engine to see when they're complete. Normally this would be overkill for a turnaround of under 200ms, but I'm taking it as a given that the ACTUAL turnaround time is 100-150ms PLUS the amount of time spent waiting for an available downstream channel.
Also note that the "engine" can safely spawn sub-threads (as long as you clean them up at shutdown time!), so it's perfectly fine if you spawn 9 threads, each with an instance of a service class and have them pull requests from the engine's request queue.
"privilege" comes from the Latin words for "private" and "law" (legal) and dates to feudal times. To "claim privilege" meant that you were above the laws that applied to the common people.
posted 2 years ago
Many thanks for the suggestion.
Reading up on the technique, it may be just what I require.
They worship nothing. They say it's because nothing is worth fighting for. Like this tiny ad: