aspose file tools*
The moose likes Tomcat and the fly likes Tomcat multithread servlet Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Spring in Action this week in the Spring forum!
JavaRanch » Java Forums » Products » Tomcat
Bookmark "Tomcat multithread servlet" Watch "Tomcat multithread servlet" New topic
Author

Tomcat multithread servlet

Sard ukar
Greenhorn

Joined: Mar 16, 2012
Posts: 11
Hello, everyone.

I'm kind of stuck with one thing: I have tomcat server 7.0.26 running servlet applications. Using connector Catalina to them. I have block in server.xml:

<Connector port="80" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443"
maxThreads="3000"
disableUploadTimeout="true"
acceptCount="1000"
/>

Everything is fine, until the actual load has not consumed 1 CPU core. After that, requests start to stuck and response time is getting higher and higher.

Server has 1 process of tomcat running:

ps wax | grep tomcat
13627 pts/2 S+ 0:00 grep tomcat
27388 ? Sl 390:25 /usr/bin/java -Djava.util.logging.config.file=/usr/local/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/usr/local/tomcat/endorsed -classpath /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/usr/local/tomcat -Dcatalina.home=/usr/local/tomcat -Djava.io.tmpdir=/usr/local/tomcat/temp org.apache.catalina.startup.Bootstrap start

My question is: why the load is not being distributed to another CPU cores? Servlet is written not using SingleThreadModel and as I can see servlet is using only one thread to reply to tomcat, While tomcat is has multiple threads for clients and just keep them waiting.

I've read tons of docs and tried tons of methods trying to get multiple instances for servlet, but Tocat is always starting one. May be I'm doing something wrong?

If you will need any technical details, do not hasitate to ask I'm not sure what do you need, so was not willing to overload topic with them.

Thanks in advance.
Tim Moores
Rancher

Joined: Sep 21, 2011
Posts: 2408
There will always be only a single instance of a servlet. This instance will be used by multiple threads, so it needs to be written in a thread-safe manner.
Sard ukar
Greenhorn

Joined: Mar 16, 2012
Posts: 11
True, but the servlet is processing requests one by one. So multithreaded tomcat is just accepting data from clients in multithread, but then all requests are queued in one loooong queue to the servlet. And true, tomcat is accessing servlet from different threads. But the bottleneck is servlet which is in single instance and actually working as singlethread.

I think I'm missing some magic setting to make servlet accepting requests from tomcat in multithread manner.
Tim Moores
Rancher

Joined: Sep 21, 2011
Posts: 2408
If the servlet is working in a synchronized manner then that's because the code is making it so, maybe by synchronizing large parts of the code. Servlets themselves will be called by the container in a multi-threaded manner, i.e. concurrently.

Just because there's only a single instance of the servlet does not mean that it can't be run multi-threaded.
Sard ukar
Greenhorn

Joined: Mar 16, 2012
Posts: 11
Thanks for your replies.

So you're trying to say, that servlet by itself is limiting number of threads it can accept from tomcat?

I thought that tomcat do not care about thread safety unless asked explicitly at the servlet ( via SingleThreadModel for example ), and invoking as much copies of servlet as needed. Developer should care about thread safety by himself. Am I wrong?
Tim Moores
Rancher

Joined: Sep 21, 2011
Posts: 2408
You're right, sort of, but you seem a bit unclear about the terminology. The servlet has no way of limiting the number of threads that run it; the servlet container determines that. But since you said that the code is executed sequentially, there must be some code in the servlet that cause sit to act sequentially.

Not sure what you mean by "copies" - as I said, there is only a single instance of the servlet class, but that is invoked multiple times concurrently.
Sard ukar
Greenhorn

Joined: Mar 16, 2012
Posts: 11
That's exaclty that makes me confusing - terminology. If servlet has one copy to run, but invoked multiple times, then it should be able to contact with tomcat threads simultaneously. Meanwhile, servlet uses MySQL, at MySQL process list, there are no more than 1 simultaneous connection from servlet. At this time, tomcat has active hundred of threads with customer requests. Most of requests are POST, if it matters.

And about servlet container. In my case, as I understand - it is Catalina. May be I've configured it to work in single thread with tomcat? But the configuration is default.

And if there is an issue at the code. How to find where? Any tools for that?

Thanks in advance.
Tim Moores
Rancher

Joined: Sep 21, 2011
Posts: 2408
Is there any explicit synchronization in the code?

How did you determine that all the requests run sequentially? Have you tried adding System.out.println statements at strategic places to find out which methods get invoked in which order?
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 16145
    
  21

The old threading options for servlets and JSPs are strongly discouraged these days.

There is no servlet queue in Tomcat, so yes, the servlet request process method (which invokes doGet, doPost, etc.) can be executing concurrently in multiple threads and therefore and static or member variables in the servlet should only be accessed in a thread-safe manner. Better still, just don't define member or static variables. Thread-safe access slows things down and risks forming deadlocks.

It's also worth remembering that per the J2EE spec, servlets are forbidden from spawing threads themselves. The threads that process servlet requests come from a pool and are returned to that pool as soon as the response has been sent, so they shouldn't be carrying any long-term lint such as child threads.


Customer surveys are for companies who didn't pay proper attention to begin with.
Sard ukar
Greenhorn

Joined: Mar 16, 2012
Posts: 11
Looks like you're right. After rewriting servlet code, container started to occupy more than 1 core.

Thanks for you suggestions. Will see what will happen with load.
Sard ukar
Greenhorn

Joined: Mar 16, 2012
Posts: 11
Well, issue is not solved. When the load restored I've observed the following picture:
- serlvet is communicating to DB and I've seen a hundreds of connections to DB, meaning that it's running multi-threaded.
- tomcat do not occupy more than one core, but it's processed consumed 100% and reply time on requests start to increase, showing lags

So we're back to original question again: how can I force servlet to use multiple cores? Looks like each servlet has a separate thread. So in case I have 5 servlets I can occupy 5 cores in maximum.

I'm using connection pool with non-blocking access.

Looks like I'm missing something simple is server setup ...
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 16145
    
  21

Servlets do not "have" threads. The processing threads are in Tomcat's thread pool. When an HTTP request comes in, Tomcat pulls a thread from the pool, tells the thread to run the servlet code, then returns the thread to the pool when the servlet process() method is complete.

If there are multiple requests coming in, Tomcat will pull a thread from the pool for each request and dispatch a servlet (JSPs count as servlets here). If there are not, the threads remain idle.

Things that can max out a single core include:

1. Having only one active HTTP request in process

2. Having a servlet go into synchronized execution, putting other threads on hold.

3. Having a single runaway logic thread that eats disproportionate amounts of CPU.

4. Having the JVM mis-configured so that all the threads are executing as synchronous processes under one OS thread (what Microsoft called a "bundle").
Sard ukar
Greenhorn

Joined: Mar 16, 2012
Posts: 11
Thank you for your detailed reply. In my head, the logic is the same. But on practice ...

1. There are multiple requests ( 400 simultaneous )
2. This was checked and there is not syncronized code being executed. DB backend confirms that servlet has multiple threads running, cause each HTTP request produces separate DB connection.
3. Servlet is pretty simple, and as of 2 I doubt in this.
4. This is quiet possible. How to check that?

Thank you.
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 16145
    
  21

2a. Having the servlets single-thread on an external resource (such as a database). I trust you are using a database connection pool, at least.

4. Check out "green threads". I think this is obsolete now, and what little I learned on the subject, I've forgotten, but it might tell you something.
Sard ukar
Greenhorn

Joined: Mar 16, 2012
Posts: 11
Thank you for your reply.

2a. There is no connection pool. Just regular connect, query and close on service(). As far as I understand how it works, each thread of tomcat, create invocation of servlet. And so, all this connect()`s should be in separate invocations. And I can see that on DB. It just looks like green threads, you're right.

4. I've read about those, but looks like this were obsoleted at JVM 1.3 or somewhere around. I do not use green threads explicitly, but symptoms are similar.
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 16145
    
  21

A high-usage webapp should be using a connection pool. Connections are quite expensive to create.
Sard ukar
Greenhorn

Joined: Mar 16, 2012
Posts: 11
I agree. But still. I'm ready to pay price of establishing and droping connections on servlet invocation. But, I cannot do that for some reason. And I'd like to find out why. I do not think that this is normal behavior for the tomcat.

By the way, to clarify. I'm using thread pool for Catalina connector. And I'm not using pool for DB connections. Just not to confuse thread pool and connection pool.
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 16145
    
  21

You don't have an option on the servlet processor threads. Tomcat itself manages them and uses them to run the servlet code. Nor is there such a thing as a "connection pool" for incoming requests or outgoing responses, since these entities are too heterogeneous to pool, allowing for certain subtleties that would only cloud this message thread to discuss, since they work quite well automatically.

I will repeat, however:

A high-usage webapp should be using a connection pool. Connections are quite expensive to create.


Unlike the http connections, the database connections are limited in scope, being purely between the appserver and the database server(s). Those connections are extremely expensive to open and close, and any webapp worth developing in a language environment that's as expensive to use as Java should be using a database connection pool, not creating them from scratch every time. Otherwise, you might as well be cracking nuts with an 8-lb sledgehammer.

Tomcat has a long history of providing good performance with high reliability. So when people have a problem with Tomcat, we can pretty much assume that the real performance problem is in the applications. The environmental defaults on Tomcat are optimal for the vast majority of webapps, so before resorting to arcane solutions, the first thing that has to be done is to determine where the problem is.

This is critical, because too often, people "know" where the problem in. My experience - which goes way back in this field - and the experiences reported by many others, is that when you sit down with a decent set of monitoring and reporting tools and turn them loose on a problem app, the results are that the actual problem is almost never where you "know" it is, and that very frequently, the "known" problem is so insignificant that it isn't worth bothering about. Check our the historical record in our Performance forum, for examples.

In brief, I think we're probably not going to be able to help much more without knowing more about the internal details of the webapp itself. We recommend "best practices", but that's about all we can do without specifics.

Of course, if it was really important, I'm available as an independent consultant at suitably ruinous rates to dial in and/or replicate the offending system and pinpoint the real problems, but I mention that just because I do have extensive experience in such matters for those who'd rather pay for such things and resolve the problem in fairly quick order, not because I actually expect anyone to come rushing at me cash in hand. Especially since my recommendations on resolution usually don't sit well with companies that either want a simple, cheap out.
Sard ukar
Greenhorn

Joined: Mar 16, 2012
Posts: 11
Thanks for such detailed reply. Explains a lot. I'll force developers to make changes to application and use connection pool, as my vision is close to yours.

The important thing for me to know if that there is no mistake in my Tomcat config, making it single thread. Assuming of course, that config is close to default.

Will report back if connection pool will solve the problem.
Jayesh A Lalwani
Bartender

Joined: Jan 17, 2008
Posts: 2409
    
  28

Your problem may not be your code itself. It may be how MySQL is configured and how your application interacts with MySQL

You might want to monitor the locks on MySQL. If your queries are creating a lot of write locks, threads might be blocking each other. You might want to also monitor what MYSQL is doing with memory and IO. Also, remember that MySQL has thread pools and buffer pools of it's own, and sending too many requests to MySQL will cause MySQL to block some of them.

If you have a big machine that is running MySQL, you might want to start by using my-huge.cnf, and then tweak it to suit your needs.
Sard ukar
Greenhorn

Joined: Mar 16, 2012
Posts: 11
Thank you for you comment.

During synthetic tests I've done only read queries, without locks. Also, MySQL backend has no any visible load on it during synthetic tests, so I doubt that this is DB itself. I'm thinking that it is probably MySQL driver of Tomcat, which is managing those hundreds of connect/disconnect in a second.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Tomcat multithread servlet