Baski Reddy wrote:What version of JDK is in use? May be the standard JDK monitoring tools like jstack,jconsole... You need two things to troubleshoot further
- Thread dumps
- GC/Heap Dumps
Peter Johnson wrote:Pretty much anything that can make a URL would work. I've even seen JMeter used for stuff like this. However, it might be better if you tried to figure out the cause for the "hang". Things that I have seen (and how I figured out the root cause) are:
Peter Johnson wrote:a) An infinite loop in the application code. The developers swore to me (or actually, to the person I was interacting with) that there was no such infinite loop. I had them take several JVM thread dumps, several seconds apart and look for threads that were always busy in the same location. They found the loop. Such an issue can "steal" threads from your thread pool (because the threads are never released). And users often won't complain - all the'll see is that their request is taking a long time to complete so they'll try it again and maybe this time they won't hit the combination of factors that causes a loop.
Peter Johnson wrote:b) Heap space issues. Gather garbage collection statistics and use them to right-size your heap. If you end up filling up the heap, then the JVM will constantly perform major collections, which slows things down to a crawl.
Peter Johnson wrote:c) Poor database access schemes, poorly written queries or poorly planned database updates. You need to gather database statistics to track these down.
Valery Lezhebokov wrote:
massimiliano cattaneo wrote:
The problem is that if the method thrown an exception (RuntimeException) the Transaction associated is rolledback and also the message is put back in the Queue ( if the Queue is used as destination of the message)
According to spec all RuntimeException (System Exceptions) should be logged, so I think it would be visible in the logs if that was the case.
I don't know this for sure, because the server often leaves no error logs around the time of the crash, it just becomes non-responsive, yet the process is still running. I will often see CPU in use, which I have found to be JMS related processes still running in the background. I don't know for sure, maybe they're unrelated; but I still want to separate the bean processes into their own application server.
I don't know the details of the architecture, but I guess that the majority of the work is done on the JMS consumer side. In this case indeed spiting up the job among several consumers make sense, but before doing that you really need to be sure that it's necessary. Having everything in one JVM is
oftenalways much simpler (especially from maintenance pov).