In one of our web application we are having some issues with 100% CPU utilization and we are trying to figure out the cause. We've taken a thread dump at an interval of about half an hour and a few of the worker threads are runnable in both dumps and I'm trying to figure out whether they are serving the same request or different requests but don't have knowledge of thread dumps so looking for some help
Like this is the stack trace of one of the threads in the two dumps
From the looks of it the same thread is having a lock on different objects in both cases so it might be a different request but out of 50 worker threads same 5 threads are active in both dumps (and their stack trace is also the same apart from some hex codes) so its a little confusing. Can we know whether its the same request with thread dump? I can attach more thread dumps of the other 4 threads.
You can use VisualVM, it has a plugin for thread dump analysis. This can show blocked threads. You mentioned that you are investigating cause for 100% CPU utilization. Normally threads waiting on locks do not cause 100% CPU utilization. For 100% CPU utilization, it might be useful to look at GC activity. You can enable GC logs for this OR VisualVM can also be used.
Looks like the same thread serving two unrelated requests, since the various stream objects that are being locked are a completely different set in the each stack trace. If looks like the threads are just working on compressing the response data.
What kind of data are being returned here -- anything really large?
Thanks Rishi and Ernest. I'll take a memory dump and see if there's anything wrong there. As for the response, its just a normal web page though its heavier than a normal web page. I think the images don't matter and the size of the html itself is around 200KB in an extreme case (uncompressed). I'm using sitemesh, can sitemesh with the compression filter cause such problems?
subject: Web application worker threads running for a long time