• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Struck thread and deadlock while getting connection

 
Greenhorn
Posts: 21
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi All,

We are getting following error. what is the cause for the below issue.

Full thread dump Java HotSpot(TM) Server VM (1.4.1_05-b01 mixed mode):

"20100314540051" daemon prio=5 tid=0x21f67e8 nid=0x627 waiting for monitor entry [52c00000..52c019bc]
at java.util.logging.StreamHandler.publish(StreamHandler.java:173)
- waiting to lock <916fcb18> (a weblogic.logging.ConsoleHandler)
at weblogic.logging.ConsoleHandler.publish(ConsoleHandler.java:107)
at java.util.logging.Logger.log(Logger.java:424)
at weblogic.logging.MessageLogger.log(MessageLogger.java:92)
at weblogic.jdbc.JDBCLogger.logConnCreatedInfo(JDBCLogger.java:129)
at weblogic.jdbc.common.internal.ConnectionEnvFactory.makeConnection(ConnectionEnvFactory.java:215)
at weblogic.jdbc.common.internal.ConnectionEnvFactory.createResource(ConnectionEnvFactory.java:111)
at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1115)
at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1052)
at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:332)
at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:258)
* at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:439)*
* at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:347)*
* at weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:78)*
at weblogic.jdbc.common.internal.RmiDataSource.getPoolConnection(RmiDataSource.java:288)
at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:310)
at com.fortressit.ejb.EJB11.EJB11Context.getConnection(EJB11Context.java:109)
at com.base.util.DBConnection.getConnection(DBConnection.java:206)
at com.cms.comn.api.CmsAPIAcctUser.retrieveUserId(CmsAPIAcctUser.java:1537)
 
Bartender
Posts: 6663
5
MyEclipse IDE Firefox Browser Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
How did you conclude this is a deadlock ?
 
skumar nand
Greenhorn
Posts: 21
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The below are full thread dump
"ExecuteThread: '42' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0xc6310 nid=0x36 waiting for monitor entry [6a180000..6a1819bc]
at weblogic.cluster.MemberManager.findOrCreate(MemberManager.java:285)
- waiting to lock <914676c8> (a weblogic.cluster.MemberManager)
at weblogic.cluster.MemberManager.resetTimeout(MemberManager.java:325)
at weblogic.cluster.replication.ReplicationManager.updateSecondary(ReplicationManager.java:724)
at weblogic.ejb20.swap.ReplicatedMemorySwap.sendUpdate(ReplicatedMemorySwap.java:212)
at weblogic.ejb20.manager.StatefulSessionManager.replicate(StatefulSessionManager.java:1071)
at weblogic.ejb20.manager.StatefulSessionManager.replicateAndRelease(StatefulSessionManager.java:1203)
at weblogic.ejb20.manager.StatefulSessionManager.afterCompletion(StatefulSessionManager.java:761)
at weblogic.ejb20.internal.TxManager$TxListener.afterCompletion(TxManager.java:720)
at weblogic.transaction.internal.ServerSCInfo.callAfterCompletions(ServerSCInfo.java:853)
at weblogic.transaction.internal.ServerTransactionImpl.callAfterCompletions(ServerTransactionImpl.java:2658)
at weblogic.transaction.internal.ServerTransactionImpl.afterCommittedStateHousekeeping(ServerTransactionImpl.java:2556)
at weblogic.transaction.internal.ServerTransactionImpl.setCommitted(ServerTransactionImpl.java:2588)
at weblogic.transaction.internal.ServerTransactionImpl.globalRetryCommit(ServerTransactionImpl.java:2386)
at weblogic.transaction.internal.ServerTransactionImpl.globalCommit(ServerTransactionImpl.java:2315)
at weblogic.transaction.internal.ServerTransactionImpl.internalCommit(ServerTransactionImpl.java:255)
at weblogic.transaction.internal.ServerTransactionImpl.commit(ServerTransactionImpl.java:221)
at weblogic.ejb20.internal.BaseEJBObject.postInvoke(BaseEJBObject.java:289)
at com.mycomp.pfk.ejb.PfkSessionBean_mz6mqm_EOImpl.getPortalRecord(PfkSessionBean_mz6mqm_EOImpl.java:367)
at com.mycomp.pfk.ejb.PfkSessionBean_mz6mqm_EOImpl_WLSkel.invoke(Unknown Source)
at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:477)
at weblogic.rmi.cluster.ReplicaAwareServerRef.invoke(ReplicaAwareServerRef.java:108)
at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:420)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:353)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:144)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:415)
at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:30)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)
"ExecuteThread: '35' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x9b6808 nid=0x2f waiting for monitor entry [6a881000..6a8819bc]
at java.util.logging.StreamHandler.publish(StreamHandler.java:173)
- waiting to lock <916fcb18> (a weblogic.logging.ConsoleHandler)
at weblogic.logging.ConsoleHandler.publish(ConsoleHandler.java:107)
at java.util.logging.Logger.log(Logger.java:424)
at weblogic.logging.MessageLogger.log(MessageLogger.java:92)
at weblogic.rmi.RMILogger.logAssociateTX(RMILogger.java:111)
at weblogic.rmi.internal.BasicServerRef.postInvoke(BasicServerRef.java:546)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:435)
at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:30)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)

"ExecuteThread: '34' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x9b5c58 nid=0x2e waiting for monitor entry [6a981000..6a9819bc]
at java.util.logging.StreamHandler.publish(StreamHandler.java:173)
- waiting to lock <916fcb18> (a weblogic.logging.ConsoleHandler)
at weblogic.logging.ConsoleHandler.publish(ConsoleHandler.java:107)
at java.util.logging.Logger.log(Logger.java:424)
at weblogic.logging.MessageLogger.log(MessageLogger.java:92)
at weblogic.rmi.RMILogger.logAssociateTX(RMILogger.java:111)
at weblogic.rmi.internal.BasicServerRef.postInvoke(BasicServerRef.java:546)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:435)
at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:30)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)

"ExecuteThread: '33' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x9b50a8 nid=0x2d waiting for monitor entry [6aa81000..6aa819bc]
at java.util.logging.StreamHandler.publish(StreamHandler.java:173)
- waiting to lock <916fcb18> (a weblogic.logging.ConsoleHandler)
at weblogic.logging.ConsoleHandler.publish(ConsoleHandler.java:107)
at java.util.logging.Logger.log(Logger.java:424)
at weblogic.logging.MessageLogger.log(MessageLogger.java:92)
at weblogic.rmi.RMILogger.logAssociateTX(RMILogger.java:111)
at weblogic.rmi.internal.BasicServerRef.postInvoke(BasicServerRef.java:546)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:435)
at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:30)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)

"ExecuteThread: '32' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x8f27d8 nid=0x2c waiting for monitor entry [6ab81000..6ab819bc]
at java.util.logging.StreamHandler.publish(StreamHandler.java:173)
- waiting to lock <916fcb18> (a weblogic.logging.ConsoleHandler)
at weblogic.logging.ConsoleHandler.publish(ConsoleHandler.java:107)
at java.util.logging.Logger.log(Logger.java:424)
at weblogic.logging.MessageLogger.log(MessageLogger.java:92)
at weblogic.rmi.RMILogger.logAssociateTX(RMILogger.java:111)
at weblogic.rmi.internal.BasicServerRef.postInvoke(BasicServerRef.java:546)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:435)
at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:30)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)

"ExecuteThread: '31' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x8f2028 nid=0x2b waiting for monitor entry [6ac81000..6ac819bc]
at weblogic.cluster.MemberManager.findOrCreate(MemberManager.java:285)
- waiting to lock <914676c8> (a weblogic.cluster.MemberManager)
at weblogic.cluster.MemberManager.resetTimeout(MemberManager.java:325)
at weblogic.cluster.replication.ReplicationManager.unregister(ReplicationManager.java:677)
at weblogic.ejb20.manager.ReplicatedStatefulSessionManager.remove(ReplicatedStatefulSessionManager.java:193)
at weblogic.ejb20.internal.StatefulEJBObject.remove(StatefulEJBObject.java:92)
at com.mycomp.pfk.ejb.PfkSessionBean_mz6mqm_EOImpl.remove(PfkSessionBean_mz6mqm_EOImpl.java:388)
at com.mycomp.pfk.ejb.PfkSessionBean_mz6mqm_EOImpl_WLSkel.invoke(Unknown Source)
at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:477)
at weblogic.rmi.cluster.ReplicaAwareServerRef.invoke(ReplicaAwareServerRef.java:108)
at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:420)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:353)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:144)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:415)
at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:30)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)

"ExecuteThread: '30' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x8f1478 nid=0x2a in Object.wait() [6ad81000..6ad819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144dd90> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144dd90> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '29' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x8f08c8 nid=0x29 in Object.wait() [6ae81000..6ae819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144de10> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144de10> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '28' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x8efd18 nid=0x28 in Object.wait() [6af81000..6af819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144de90> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144de90> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '27' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x8ef168 nid=0x27 in Object.wait() [6b081000..6b0819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144df10> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144df10> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '26' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x8ee5b8 nid=0x26 in Object.wait() [6b181000..6b1819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144df90> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144df90> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '25' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x8eda70 nid=0x25 in Object.wait() [6b281000..6b2819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144e010> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144e010> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '24' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x8ecb90 nid=0x24 in Object.wait() [6b381000..6b3819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144e090> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144e090> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '23' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x1b0d28 nid=0x23 in Object.wait() [6b481000..6b4819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144e110> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144e110> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '22' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x1b0178 nid=0x22 in Object.wait() [6b581000..6b5819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144e190> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144e190> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '21' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x1af5c8 nid=0x21 in Object.wait() [6b681000..6b6819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144e210> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144e210> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '20' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x1aea18 nid=0x20 in Object.wait() [6b781000..6b7819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144e290> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144e290> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '19' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x1ade68 nid=0x1f in Object.wait() [6b881000..6b8819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144e310> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144e310> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '18' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x1acab0 nid=0x1e in Object.wait() [6b981000..6b9819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144e390> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144e390> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '17' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x7f5f98 nid=0x1d in Object.wait() [6ba81000..6ba819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144e410> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144e410> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '16' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x7f53e8 nid=0x1c in Object.wait() [6bb81000..6bb819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144e490> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144e490> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '15' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x7f4838 nid=0x1b in Object.wait() [6bc81000..6bc819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144e510> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144e510> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '14' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x7f3c88 nid=0x1a in Object.wait() [6bd81000..6bd819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144e590> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144e590> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '13' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x7f30d8 nid=0x19 in Object.wait() [6be81000..6be819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144e610> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144e610> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '12' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x7f2600 nid=0x18 in Object.wait() [6bf81000..6bf819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144e690> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144e690> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '11' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x7f1358 nid=0x17 in Object.wait() [6c081000..6c0819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144e710> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144e710> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"ExecuteThread: '0' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x9c3640 nid=0xc in Object.wait() [6cb81000..6cb819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144ec90> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:145)
- locked <9144ec90> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:164)

"Signal Dispatcher" daemon prio=10 tid=0xdd020 nid=0x9 waiting on condition [0..0]

"Finalizer" daemon prio=8 tid=0xd9a78 nid=0x6 in Object.wait() [71e81000..71e819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144ee30> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:111)
- locked <9144ee30> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:127)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=10 tid=0xd8938 nid=0x5 in Object.wait() [71f81000..71f819bc]
at java.lang.Object.wait(Native Method)
- waiting on <9144ee98> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:426)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:113)
- locked <9144ee98> (a java.lang.ref.Reference$Lock)

"main" prio=5 tid=0x2d630 nid=0x1 in Object.wait() [ffbee000..ffbee58c]
at java.lang.Object.wait(Native Method)
- waiting on <9144f0e8> (a weblogic.t3.srvr.T3Srvr)
at java.lang.Object.wait(Object.java:426)
at weblogic.t3.srvr.T3Srvr.waitForDeath(T3Srvr.java:1170)
- locked <9144f0e8> (a weblogic.t3.srvr.T3Srvr)
at weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:384)
at weblogic.Server.main(Server.java:32)

"VM Thread" prio=5 tid=0xd7ce0 nid=0x4 runnable

"VM Periodic Task Thread" prio=10 tid=0xdaac8 nid=0x7 waiting on condition
"Suspend Checker Thread" prio=10 tid=0xdac18 nid=0x8 runnable

Found one Java-level deadlock:
=============================
"ExecuteThread: '68' for queue: 'weblogic.kernel.Default'":
waiting to lock monitor 0x1fe4b8 (object 0x916fcb50, a weblogic.logging.FileStreamHandler),
which is held by "ExecuteThread: '60' for queue: 'weblogic.kernel.Default'"
"ExecuteThread: '60' for queue: 'weblogic.kernel.Default'":
waiting to lock monitor 0x1fe410 (object 0x916fcb18, a weblogic.logging.ConsoleHandler),
which is held by "ExecuteThread: '68' for queue: 'weblogic.kernel.Default'"

Java stack information for the threads listed above:
===================================================
"ExecuteThread: '68' for queue: 'weblogic.kernel.Default'":
at java.util.logging.StreamHandler.publish(StreamHandler.java:173)
- waiting to lock <916fcb50> (a weblogic.logging.FileStreamHandler)
at weblogic.logging.FileStreamHandler.publish(FileStreamHandler.java:191)
at java.util.logging.Logger.log(Logger.java:424)
at weblogic.logging.MessageLogger.log(MessageLogger.java:121)
at weblogic.logging.MessageLogger.log(MessageLogger.java:112)
at weblogic.logging.WLErrorManager.error(WLErrorManager.java:48)
- locked <917bb0e0> (a weblogic.logging.WLErrorManager)
at java.util.logging.Handler.reportError(Handler.java:208)
at java.util.logging.StreamHandler.publish(StreamHandler.java:182)
- locked <916fcb18> (a weblogic.logging.ConsoleHandler)
at weblogic.logging.ConsoleHandler.publish(ConsoleHandler.java:107)
at java.util.logging.Logger.log(Logger.java:424)
at weblogic.logging.MessageLogger.log(MessageLogger.java:92)
at weblogic.servlet.internal.session.HTTPSessionLogger.logErrorGettingSession(HTTPSessionLogger.java:1173)
at weblogic.servlet.internal.session.ReplicatedSessionContext.getSessionInternal(ReplicatedSessionContext.java:525)
at weblogic.servlet.internal.ServletRequestImpl.getValidSession(ServletRequestImpl.java:2489)
at weblogic.servlet.internal.ServletRequestImpl.getSession(ServletRequestImpl.java:2153)
at weblogic.servlet.security.internal.SecurityModule$SessionRetrievalAction.run(SecurityModule.java:637)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:317)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)
at weblogic.servlet.security.internal.SecurityModule.getUserSession(SecurityModule.java:612)
at weblogic.servlet.security.internal.SecurityModule.beginCheck(SecurityModule.java:153)
at weblogic.servlet.security.internal.BasicSecurityModule.checkA(BasicSecurityModule.java:46)
at weblogic.servlet.security.internal.ServletSecurityManager.checkAccess(ServletSecurityManager.java:145)
at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3539)
at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2585)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)
"ExecuteThread: '60' for queue: 'weblogic.kernel.Default'":
at java.util.logging.StreamHandler.publish(StreamHandler.java:173)
- waiting to lock <916fcb18> (a weblogic.logging.ConsoleHandler)
at weblogic.logging.ConsoleHandler.publish(ConsoleHandler.java:107)
at java.util.logging.Logger.log(Logger.java:424)
at weblogic.logging.MessageLogger.log(MessageLogger.java:121)
at weblogic.logging.MessageLogger.log(MessageLogger.java:112)
at weblogic.logging.WLErrorManager.error(WLErrorManager.java:48)
- locked <917bb138> (a weblogic.logging.WLErrorManager)
at java.util.logging.Handler.reportError(Handler.java:208)
at java.util.logging.StreamHandler.publish(StreamHandler.java:182)
- locked <916fcb50> (a weblogic.logging.FileStreamHandler)
at weblogic.logging.FileStreamHandler.publish(FileStreamHandler.java:191)
at java.util.logging.Logger.log(Logger.java:424)
at weblogic.logging.MessageLogger.log(MessageLogger.java:92)
at weblogic.servlet.internal.session.HTTPSessionLogger.logErrorGettingSession(HTTPSessionLogger.java:1173)
at weblogic.servlet.internal.session.ReplicatedSessionContext.lookupROs(ReplicatedSessionContext.java:346)
at weblogic.servlet.internal.session.ReplicatedSessionContext.getSessionInternal(ReplicatedSessionContext.java:448)
at weblogic.servlet.internal.ServletRequestImpl.getValidSession(ServletRequestImpl.java:2489)
at weblogic.servlet.internal.ServletRequestImpl.getSession(ServletRequestImpl.java:2153)
at weblogic.servlet.internal.ServletRequestImpl.getRequestedSessionId(ServletRequestImpl.java:2208)
at weblogic.servlet.security.internal.SecurityModule.getCurrentUser(SecurityModule.java:267)
at weblogic.servlet.security.internal.SecurityModule.beginCheck(SecurityModule.java:156)
at weblogic.servlet.security.internal.BasicSecurityModule.checkA(BasicSecurityModule.java:46)
at weblogic.servlet.security.internal.ServletSecurityManager.checkAccess(ServletSecurityManager.java:145)
at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3539)
at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2585)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)

Found 1 deadlock.

Full thread dump Java HotSpot(TM) Server VM (1.4.1_05-b01 mixed mode):

"20100314540051" daemon prio=5 tid=0x21f67e8 nid=0x627 waiting for monitor entry [52c00000..52c019bc]
at java.util.logging.StreamHandler.publish(StreamHandler.java:173)
- waiting to lock <916fcb18> (a weblogic.logging.ConsoleHandler)
at weblogic.logging.ConsoleHandler.publish(ConsoleHandler.java:107)
at java.util.logging.Logger.log(Logger.java:424)
at weblogic.logging.MessageLogger.log(MessageLogger.java:92)
at weblogic.jdbc.JDBCLogger.logConnCreatedInfo(JDBCLogger.java:129)
at weblogic.jdbc.common.internal.ConnectionEnvFactory.makeConnection(ConnectionEnvFactory.java:215)
at weblogic.jdbc.common.internal.ConnectionEnvFactory.createResource(ConnectionEnvFactory.java:111)
at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1115)
at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1052)
at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:332)
at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:258)
at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:439)
at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:347)
at weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:78)
at weblogic.jdbc.common.internal.RmiDataSource.getPoolConnection(RmiDataSource.java:288)
at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:310)
at com.fortressit.ejb.EJB11.EJB11Context.getConnection(EJB11Context.java:109)
at com.mycomp.base.util.DBConnection.getConnection(DBConnection.java:206)
at com.mycomp.cms.comn.api.CmsAPIAcctUser.retrieveUserId(CmsAPIAcctUser.java:1537)
at com.mycomp.trade.maf.sau.model.RecipientAccount.listUsers(RecipientAccount.java:119)
at com.mycomp.trade.dec.sau.util.SAUAlertUtil.registerAlert(SAUAlertUtil.java:79)
at com.mycomp.trade.dec.sau.business.DECCHRManager.registerAlert(DECCHRManager.java:178)
at com.mycomp.trade.dec.sau.business.DECCHRManager.updateAlert(DECCHRManager.java:336)
at com.mycomp.trade.dec.sau.business.DECCHRManager.processDECCHR(DECCHRManager.java:127)
at com.mycomp.trade.dec.sau.business.DECCHRManager.processMsg(DECCHRManager.java:76)
at com.mycomp.trade.dec.sau.ejb.SAUDECMsgSvcBean.processMsg(SAUDECMsgSvcBean.java:108)
at com.mycomp.trade.dec.sau.ejb.SAUDECMsgSvcBean_s7aj8_EOImpl.processMsg(SAUDECMsgSvcBean_s7aj8_EOImpl.java:100)
at com.mycomp.trade.dec.sau.business.SAUDECDelegate.processMsg(SAUDECDelegate.java:351)
at com.mycomp.trade.dec.sau.business.DECMsgProcessor.process(DECMsgProcessor.java:244)
at com.mycomp.trade.sau.msg.ejb.MsgSvcBean.processMsg(MsgSvcBean.java:300)
at com.mycomp.trade.sau.msg.ejb.MsgSvcBean_5z2zq6_EOImpl.processMsg(MsgSvcBean_5z2zq6_EOImpl.java:908)
at com.mycomp.trade.sau.msg.util.MsgDelegate.processMsg(MsgDelegate.java:300)
at com.mycomp.trade.sau.msg.util.LinkMsgProcThread.run(LinkMsgProcThread.java:35)
at java.lang.Thread.run(Thread.java:536)
 
Deepak Bala
Bartender
Posts: 6663
5
MyEclipse IDE Firefox Browser Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
So a weblogic FileStreamHandler and a ConsoleHandler are deadlocked ? What is the logging implementation ? Is it commons logging or log4j ? If log4j then do you have a console appender and a file appender ?
 
skumar nand
Greenhorn
Posts: 21
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
we used log4j with file appender
 
Deepak Bala
Bartender
Posts: 6663
5
MyEclipse IDE Firefox Browser Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Post the contents of the log4j.xml here
 
skumar nand
Greenhorn
Posts: 21
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
#this section is mandatory, please do not change

log4j.categoryFactory=com.mycomp.base.util.logging.LoggerFactory
log4j.rootCategory=INFO
log4j.category.com.mycomp.trade.dec=INFO, A-DEC
log4j.appender.A-DEC=org.apache.log4j.RollingFileAppender
log4j.appender.A-DEC.layout=org.apache.log4j.PatternLayout
log4j.appender.A-DEC.layout.ConversionPattern=%d %-5p %c - %m%n

#----- You may change this section below ------- ----
log4j.appender.A-DEC.File=/mycomp/trade/sau/rel1_0/logs/saudec.log
log4j.appender.A-DEC.MaxFileSize=1000KB
log4j.appender.A-DEC.MaxBac
 
Deepak Bala
Bartender
Posts: 6663
5
MyEclipse IDE Firefox Browser Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Could be related to this -> https://issues.apache.org/bugzilla/show_bug.cgi?id=41214

Is that the full content ? Where is the console appender ? Try removing the Console appender. Are you able to reproduce the problem after that ?
 
skumar nand
Greenhorn
Posts: 21
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
We are not using the console appender.
 
Deepak Bala
Bartender
Posts: 6663
5
MyEclipse IDE Firefox Browser Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
From the logs, it looks like you do -> ExecuteThread: '60 is waiting for a lock on ConsoleHandler.

Also looks like java util logging calls the WL logging implementation
 
Don't get me started about those stupid light bulbs.
reply
    Bookmark Topic Watch Topic
  • New Topic