We migrated to
JBoss 7.1 recently from JBoss 5.1. JVM gets hung in every couple days and stops accepting any requests. We need to restart to get it back to service. It happens when the total
thread utilization goes beyond 6-7K. We take thread dump and find majority of threads (90% of threads) stack traces look like this. Not sure what's going on.
This issue was not there when we were on JBoss 5.1. Any clue/help is appreciated.
"pool-696505-thread-3" prio=10 tid=0x0000000062b08000 nid=0x6c57 waiting on condition [0x0000000078bfc000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000007038b3400> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:315)
at com.sun.xml.messaging.saaj.util.ParserPool.get(ParserPool.java:88)
at com.sun.xml.messaging.saaj.soap.EnvelopeFactory.createEnvelope(EnvelopeFactory.java:91)
at com.sun.xml.messaging.saaj.soap.ver1_2.SOAPPart1_2Impl.createEnvelopeFromSource(SOAPPart1_2Impl.java:84)
at com.sun.xml.messaging.saaj.soap.SOAPPartImpl.getEnvelope(SOAPPartImpl.java:143)
at com.abc.myproject.integrator.uib.UnifiedMessagingIntegrator.createXMLStr(UnifiedMessagingIntegrator.java:425)
at com.abc.myproject.integrator.uib.UnifiedMessagingIntegrator.makeRemoteSystemsCall(UnifiedMessagingIntegrator.java:650)
at com.abc.myproject.integrator.RemoteRequestDispatcherConcurrent.call(RemoteRequestDispatcherConcurrent.java:69)
at com.abc.myproject.integrator.RemoteRequestDispatcherConcurrent.call(RemoteRequestDispatcherConcurrent.java:56)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
"pool-696504-thread-3" prio=10 tid=0x00002aaad5d72000 nid=0x6c52 waiting on condition [0x0000000078b57000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000007038b3400> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:315)
at com.sun.xml.messaging.saaj.util.ParserPool.get(ParserPool.java:88)
at com.sun.xml.messaging.saaj.soap.EnvelopeFactory.createEnvelope(EnvelopeFactory.java:91)
at com.sun.xml.messaging.saaj.soap.ver1_2.SOAPPart1_2Impl.createEnvelopeFromSource(SOAPPart1_2Impl.java:84)
at com.sun.xml.messaging.saaj.soap.SOAPPartImpl.getEnvelope(SOAPPartImpl.java:143)
at com.abc.myproject.integrator.uib.UnifiedMessagingIntegrator.createXMLStr(UnifiedMessagingIntegrator.java:425)
at com.abc.myproject.integrator.uib.UnifiedMessagingIntegrator.makeRemoteSystemsCall(UnifiedMessagingIntegrator.java:650)
at com.abc.myproject.integrator.RemoteRequestDispatcherConcurrent.call(RemoteRequestDispatcherConcurrent.java:69)
at com.abc.myproject.integrator.RemoteRequestDispatcherConcurrent.call(RemoteRequestDispatcherConcurrent.java:56)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)