This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes BEA/Weblogic and the fly likes Thread stuck problems - StuckThreadMaxTime exceeded Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Products » BEA/Weblogic
Bookmark "Thread stuck problems - StuckThreadMaxTime exceeded" Watch "Thread stuck problems - StuckThreadMaxTime exceeded" New topic
Author

Thread stuck problems - StuckThreadMaxTime exceeded

Navjeet Nehra
Greenhorn

Joined: Sep 18, 2003
Posts: 13
Hi,
I am having the problems running continuous 24 hrs test against Weblogic 8.1. After some time the server starts complaining about the threads stuck
<Warning> <WebLogicServer> <BEA-000337> <ExecuteThread: '7' for queue: 'weblogic.kernel.Default' has been busy for "630" seconds working on the request "weblogic.ejb20.internal.JMSMessagePoller@d64412", which is
more than the configured time (StuckThreadMaxTime) of "600" seconds.>
This happens for a number of threads and then the server goes into a continuous period of inactivity trying desperately to free memory and then eventually it start giving "java.lang.OutOfMemoryException"
My queries are
1) How to detect on what the thread is stuck on
2) In case of a stuck thread - how to roll back/stop the thread so that the rest of the processing can continue.
Thanks
Navjeet
Jack Nicholson
Ranch Hand

Joined: Sep 07, 2002
Posts: 41
hi,
I am also facing similar problem.
This is problem reported due to JMS Server.
Cause the warning tells about JMS Server port only.
With me I am not facing OutOfMemoryError.
Navjeet, let me know were you able to solve ur problem or not.
Is there any tuning to be done on JMS server ?


Jack Nicholson
Purvesh Vora
Ranch Hand

Joined: Dec 10, 2003
Posts: 47
Hi,
I am also getting the same problem.
How did you resolve this issue?
Thanks,
Purvesh


Regards,<br />Purvesh<br />SCJP, SCBCD, IBM Exam #486
sathish kannan
Greenhorn

Joined: Jul 11, 2002
Posts: 29
Hi I am also getting the same problem while doing batch query on DELETE operation in a DAO class.

removeMetaDataDefinition-------------->DELETE FROM TDMS_METADATA_DEFN WHERE RESCLASSID ='05011132220000180' AND METADATADEFNID ='05011132190000179'
<Feb 5, 2005 12:09:20 PM IST> <Warning> <WebLogicServer> <BEA-000337> <ExecuteThread: '11' for queue: 'weblogic.kernel.Default' has been busy for "864" seconds working on the request "Http Request: /dmsweb/folder.do", which is more than the configured time (StuckThreadMaxTime) of "600" seconds.>
<Feb 5, 2005 12:09:21 PM IST> <Warning> <WebLogicServer> <BEA-000337> <ExecuteThread: '13' for queue: 'weblogic.kernel.Default' has been busy for "1,539" seconds working on the request "Http Request: /dmsweb/folder.do", which is more than the configured time (StuckThreadMaxTime) of "600" seconds.>
<Feb 5, 2005 12:19:21 PM IST> <Warning> <WebLogicServer> <BEA-000337> <ExecuteThread: '10' for queue: 'weblogic.kernel.Default' has been busy for "828" seconds working on the request "Http Request: /dmsweb/folder.do", which is more than the configured time (StuckThreadMaxTime) of "600" seconds.>
<Feb 5, 2005 12:19:21 PM IST> <Warning> <WebLogicServer> <BEA-000337> <ExecuteThread: '11' for queue: 'weblogic.kernel.Default' has been busy for "1,464" seconds working on the request "Http Request: /dmsweb/folder.do", which is more than the configured time (StuckThreadMaxTime) of "600" seconds.>


If you have solved the problem please let me know

thanks in advance
Mark Spritzler
ranger
Sheriff

Joined: Feb 05, 2001
Posts: 17249
    
    6

"sathish"-
Welcome to the JavaRanch! Please adjust your displayed name to meet the

JavaRanch Naming Policy.

You can change it

here.

Thanks! and welcome to the JavaRanch!

Mark


Perfect World Programming, LLC - Two Laptop Bag - Tube Organizer
How to Ask Questions the Smart Way FAQ
Carol Enderlin
drifter
Ranch Hand

Joined: Oct 10, 2000
Posts: 1364
I check on stuck threads in the console. There are other ways. See BEA Documentation.

Of course the console looks a little different in each version, I'm looking at 8.1 right now.

Go to the managed server node of interest in the node tree (list on left -- mine shows a world icon for my domain; under that is the "servers" folder; within the servers folder are my three servers).

Fo direct access to threads and what they are doing -- right click on the server node -- select "View Execute Threads". Look to see what the Default threads are "doing" in the "current request" column.

Alternatively you can go the the server's "Monitoring Tab". Select monitor all active Queues. There you can see the number of threads idle for each type of thread. Your user's requests will go to the default queue unless you configure it differently. Click the link for "weblogic.kernel.Default" to see what those threads are doing.

Then what to do? I have a process that runs a few hours and I get a stuck thread warning (1 thread), but no harm to the server. I don't do anything about it.

Check Tuning WebLogic Server Applications, there's a section on "Using Execute Queues to Control Thread Usage". Maybe you just need more threads. Maybe you need some of your application to use a different queue. Maybe your application needs tuning.

Once you know what your threads are stuck on you could run a profiling tool like Optimizeit or Jprobe. You could have tune your db use.

Regarding stopping threads. That's a tricky business and I'm pretty sure WebLogic doesn't support killing threads.
[ February 09, 2005: Message edited by: Carol Enderlin ]
Marcos Maia
Ranch Hand

Joined: Jan 06, 2001
Posts: 977
Originally posted by Navjeet Nehra:

1) How to detect on what the thread is stuck on
2) In case of a stuck thread - how to roll back/stop the thread so that the rest of the processing can continue.

Thanks
Navjeet



1) The best way to discover it is taking a thread dump, to do that you�ll have to start the server in foreground(so you can see the console messages) or redirect the standart output to a file, then all you need to do is hit CTR+BREAK on windows or use the command kill -3 on unix environments, it�ll provide you with a thread dump where you can see exactly where your thread�s are stucked.

2) You can�t roll back a stucked thread, usually as Carol mentioned you�ll have to tune the Default thread queue on weblogic and also tune your application, most of the problems like this i have faced were solved tunning the DB then the application and then, if still required weblogic where you can tune thread count, garbage collection, etc.
You can also consider programming a threshould so the node manager can reboot the managed server in case of too much thread count usage(this will actually kill all running threads of course).

best regards.
Vetrik Kumaran
Greenhorn

Joined: Jul 01, 2003
Posts: 7
Hi,

Found this thread to be interesting, I am facing similar problem, my application has 200 as thread queue and when this problem occurs , I was not even able to get into the console.

Could any one of you help me in solving this ?

If tuning in the database is required, what kind of tuning is to be made. I would appreciate your early response.

-- Vetrik


Try Try Try...
great veera
Greenhorn

Joined: May 14, 2001
Posts: 24
Thread dump would help to nail down the issue, also check is there any DB access your app is doing,is it waiting to get connection.,is there any operation going on in DB at that point of time..
Muthu Ponmozhi
Greenhorn

Joined: Dec 17, 2004
Posts: 21
Hi,
Our app also having same StuckThread problem.My senior said this is because of the weblogic is redirecting the request to a proxy server then to a external system.if the proxy is hanging the weblogic server is getting warning like "ExecuteThread: '17' for queue: 'default' has been busy for "816" seconds working on the request "de.dl.crl.facade.sb.CRLFacadeBeanEJB_r251p2_EOImpl",which is more than the configured time (StuckThreadMaxTime) of "600" seconds.>"

Now we want to track this problem and return back to the user with proper excpetion related information. Is there a way to get this proxy status from weblogic??
if anything is not clear please tell me.
Thanks in advance.


happy learning....<br>Ponmozhi</br> <br>SCJP 1.5,SCWCD 1.5(preparing)</br>
J Praveen Reddy
Greenhorn

Joined: Aug 22, 2006
Posts: 2
Hi all,
Any solution for this?
Fadia Ismael
Greenhorn

Joined: Mar 30, 2008
Posts: 1
Originally posted by J Praveen Reddy:
Hi all,
Any solution for this?


Please have a look at

https://support.bea.com/application_content/product_portlets/support_patterns/wls/JDBCCausesServerHangPattern.html

Where it mentions:
This does not interrupt the thread, as this is just a notification for the administrator

Hope this is useful..
mahesh vavilla
Greenhorn

Joined: Dec 02, 2008
Posts: 1
we still didn't find the solution for this?

I am having the same problem.

Its been revolving from long back.

Please find a solution for this.
Vikrant Korde
Greenhorn

Joined: Jun 16, 2008
Posts: 9
Check out this link
http://e-docs.bea.com/wls/docs81/perform/WLSTuning.html#1125714

to be precise, do the following
To configure WebLogic Server stuck thread detection behavior:
1. Start the Administration Server if it is not already running.
2. Access the Administration Console for the domain.
3. Expand the Servers node in the left pane to display the servers configured in your domain.
4. Click the name of the server instance that you want to modify for improved stuck thread detection.
Note: You configure stuck thread detection parameters on a per-server basis, rather than on a per-execute queue basis.
5. Select the Configuration —> Tuning tab in the right pane.
6. Modify the following attributes as necessary to tune thread detection behavior for the server:
* Stuck Thread Max Time: Enter the number of seconds, that a thread must be continually working before this server diagnoses the thread as being stuck. By default, WebLogic Server considers a thread to be "stuck" after 600 seconds of continuous use.
* Stuck Thread Timer Interval: Enter the number of seconds, after which WebLogic Server periodically scans threads to see if they have been continually working for the length of time specified by Stuck Thread Max Time. By default, WebLogic Server sets this interval to 600 seconds.
7. Click Apply to save your changes.
8. Reboot the server to use the new settings.
Hareesh Ram Chanchali
Ranch Hand

Joined: Jan 31, 2005
Posts: 110
I strongly agree with Marcos.

We need to take thread dumps at regular intervals and should analyse them.

You take the thread dumps at an interval of 1 or 2 minutes and take the help of samurai tool to analyse the thread dumps. Then you will be able to know what exactly happenning and possible reasons

1) DB locks
2) Native thread problems if any
3) Huge amount of data being pulled up
4) extended time in getting responses from interface systems

with all these try checking the Garbage collection also.


Hareesh Ram Chanchali
SCJP 5.0, IBM Certified Solution Designer
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Thread stuck problems - StuckThreadMaxTime exceeded
 
Similar Threads
problem while starting jboss5.0
"stuck thread" issues in weblogic 9.2
BEA-000337 StuckThreadMaxTime
StuckThreadMaxTime Error.
How to notify weblogic for StuckThreadMaxTime.