aspose file tools*
The moose likes BEA/Weblogic and the fly likes Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Products » BEA/Weblogic
Bookmark ""stuck thread" issues in weblogic 9.2" Watch ""stuck thread" issues in weblogic 9.2" New topic
Author

"stuck thread" issues in weblogic 9.2

Shekar Atmakur
Ranch Hand

Joined: Oct 24, 2003
Posts: 36
Hello all,
We are having some trouble in production, where we have deployed our application on wl9.2. We have been encountering "stuck thread" issues and are hoping you can help us resolve this issue.

Basically we have configured the application server to use WL 8.1 thread pooling mechanism, where were can define user defined execute queues. Such thread pooling was replaced in WL9.2 to use work managers. We have set the default_ queue size at 50. During peak hours, all 50 threads on the Defualt execute queue are occupied. Processes running on these threads are taking a little while longer to execute causing the system response to slow down. The server logs are full of stuck threads.

I believe there is a fundamental configuration issue we are missing here or there is potentially a pacth for wl9.2. But i am just not able to place my finger on it. Could somebody please help me out with that.

Appreciate it.

Here are some snippets of the exceptions.

<blockquote>code:
<pre name="code" class="core"> <Jul 16, 2008 1:37:00 PM EDT> <Error> <WebLogicServer> <BEA-000337> <ExecuteThread: '34' for queue: 'default' has been busy for "671" seconds working on the request "weblogic.work.ExecuteRequestAdapter@50931a", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
java.lang.ClassLoader.loadClass(ClassLoader.java:295)
java.lang.ClassLoader.loadClass(ClassLoader.java:299)
java.lang.ClassLoader.loadClass(ClassLoader.java:251)
weblogic.utils.classloaders.GenericClassLoader.loadClass(GenericClassLoader.java:158)
weblogic.utils.classloaders.ChangeAwareClassLoader.loadClass(ChangeAwareClassLoader.java:35)
javax.xml.parsers.FactoryFinder.newInstance(FactoryFinder.java:88)
javax.xml.parsers.FactoryFinder.findJarServiceProvider(FactoryFinder.java:278)
</pre>
</blockquote>

<blockquote>code:
<pre name="code" class="core"> <Jul 16, 2008 1:40:24 PM EDT> <Error> <WebLogicServer> <BEA-000337> <ExecuteThread: '44' for queue: 'default' has been busy for "686" seconds working on the request "weblogic.work.ExecuteRequestAdapter@2e4333", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
weblogic.common.resourcepool.ResourcePoolImpl.reserveResourceInternal(ResourcePoolImpl.java:322)
weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:294)
weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:284)
weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:466)
weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:357)
weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:83)
</pre>
</blockquote>

<blockquote>code:
<pre name="code" class="core"> "ExecuteThread: '37' for queue: 'default'" daemon prio=10 tid=02f17860 nid=412 lwp_id=4622133 waiting for monitor entry [2d372000..2d36f578]
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:168)
at org.hibernate.util.ReflectHelper.classForName(ReflectHelper.java:108)
at org.hibernate.impl.SessionFactoryImpl.getImportedClassName(SessionFactoryImpl.java:722)
at org.hibernate.hql.classic.QueryTranslatorImpl.getEntityPersisterUsingImports(QueryTranslatorImpl.java:352)
at org.hibernate.hql.classic.WhereParser.doToken(WhereParser.java:389)
at org.hibernate.hql.classic.WhereParser.token(WhereParser.java:263)
at org.hibernate.hql.classic.ClauseParser.token(ClauseParser.java:86)
</pre>
</blockquote>
Marcos Maia
Ranch Hand

Joined: Jan 06, 2001
Posts: 977
Hi,

you have 2 log messages where you have threads 34 and 44 doing their jobs for more than 600 secs, the default configuret time for weblogic. Then you post a thread dump from thread 37? What you need to do is take some thread dumps 4 or 5 in an interval of 10 to 15 seconds each. Than from the log message you take a look on a stuck thread, for example the 34 from your log message and take a look at it's thread dump. See if you have same method holding lock or waiting for something.. Find the cause of it. Usually some db access, or slow network resource or system.
MadanMohan Bolla
Greenhorn

Joined: Nov 02, 2007
Posts: 19
Take 5-6 thread dumps and analyze the same in samurai tool so that you will exactly see which thread is actually blocking all the other threads and analyze the thread for what it is doing.
 
Don't get me started about those stupid light bulbs.
 
subject: "stuck thread" issues in weblogic 9.2