aspose file tools*
The moose likes Other Java Products and Servers and the fly likes GlassFish 2.1.1 server hangs up due to unknown broker shutdown request from Admin Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Spring in Action this week in the Spring forum!
JavaRanch » Java Forums » Products » Other Java Products and Servers
Bookmark "GlassFish 2.1.1 server hangs up due to unknown broker shutdown request from Admin" Watch "GlassFish 2.1.1 server hangs up due to unknown broker shutdown request from Admin" New topic
Author

GlassFish 2.1.1 server hangs up due to unknown broker shutdown request from Admin

shiv kavinda
Greenhorn

Joined: Apr 17, 2011
Posts: 1
Hi
I got a problem in glassfish 2.1.1 .
I caught this warning in one of our servers that runs glassfish 2.1.1:

This happens while in the running mode. I did not try to shutdown or restart . I have a payment gateway running in this server and
When the broker is down, GF does not try to bring it up. So it is down forever and my app hangs on it.
I've been hacking the server for a week day there is not much information about this problem.


sample server log is pasted below


============================================================================================
[#|2011-04-08T07:51:22.824+0100|INFO|sun-appserver2.1|javax.enterprise.system.container.ejb|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|Invoking wsimport with http://uljvae01:6021/EPDQWS/EpdqWSService?WSDL|#]
[#|2011-04-08T07:51:22.824+0100|INFO|sun-appserver2.1|javax.enterprise.system.container.ejb|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|wsimport successful|#]
[#|2011-04-08T07:52:21.667+0100|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|
parsing WSDL...
|#]
[#|2011-04-08T07:52:22.027+0100|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|
generating code...
|#]
[#|2011-04-08T07:52:22.245+0100|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|
compiling code...
|#]
[#|2011-04-08T07:52:22.792+0100|INFO|sun-appserver2.1|javax.enterprise.system.container.ejb|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|Invoking wsimport with http://uljvae01:6021/EPDQWS/EpdqWSService?WSDL|#]
[#|2011-04-08T07:52:22.792+0100|INFO|sun-appserver2.1|javax.enterprise.system.container.ejb|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|wsimport successful|#]
[#|2011-04-08T07:53:22.011+0100|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|
parsing WSDL...
|#]
[#|2011-04-08T07:53:22.042+0100|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|
generating code...
|#]
[#|2011-04-08T07:53:22.214+0100|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|
compiling code...
|#]
[#|2011-04-08T07:53:22.870+0100|INFO|sun-appserver2.1|javax.enterprise.system.container.ejb|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|Invoking wsimport with http://uljvae01:6021/EPDQWS/EpdqWSService?WSDL|#]
[#|2011-04-08T07:53:22.870+0100|INFO|sun-appserver2.1|javax.enterprise.system.container.ejb|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|wsimport successful|#]
[#|2011-04-08T07:54:21.980+0100|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|
parsing WSDL...
|#]
[#|2011-04-08T07:54:22.495+0100|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|
generating code...
|#]
[#|2011-04-08T07:54:22.636+0100|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|
compiling code...
|#]
[#|2011-04-08T07:54:23.120+0100|INFO|sun-appserver2.1|javax.enterprise.system.container.ejb|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|Invoking wsimport with http://uljvae01:6021/EPDQWS/EpdqWSService?WSDL|#]
[#|2011-04-08T07:54:23.120+0100|INFO|sun-appserver2.1|javax.enterprise.system.container.ejb|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|wsimport successful|#]
[#|2011-04-08T07:55:21.949+0100|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|
parsing WSDL...
|#]
[#|2011-04-08T07:55:22.105+0100|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|
generating code...
|#]
[#|2011-04-08T07:55:22.324+0100|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|
compiling code...
|#]
[#|2011-04-08T07:55:22.870+0100|INFO|sun-appserver2.1|javax.enterprise.system.container.ejb|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|Invoking wsimport with http://uljvae01:6021/EPDQWS/EpdqWSService?WSDL|#]
[#|2011-04-08T07:55:22.870+0100|INFO|sun-appserver2.1|javax.enterprise.system.container.ejb|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|wsimport successful|#]
[#|2011-04-08T07:56:11.574+0100|INFO|sun-appserver2.1|javax.resourceadapter.mqjmsra.lifecycle|_ThreadID=18;_ThreadName=iMQEventHandler-4;|MQJMSRA_RA1101: onEvent:Connection Event:E201:[E201]: Connection closed due to admin requested shutdown: localhost:6027(2021), com.sun.messaging.jms.notification.ConnectionClosedEvent[source=BrokerAddress=localhost:6027(2021), ConnectionID=6740532968453863424, ReconnectEnabled: false, IsConnectedToHABroker: false]|#]
[#|2011-04-08T07:56:11.574+0100|WARNING|sun-appserver2.1|javax.resourceadapter.mqjmsra.lifecycle|_ThreadID=18;_ThreadName=iMQEventHandler-4;_RequestID=fed5aa4c-cb5c-4a77-9bfa-0fd92b3669a3;|MQJMSRA_RA2001: onException:Connection Failed:[C4056]: Received goodbye message from broker.|#]
[#|2011-04-08T07:56:16.620+0100|WARNING|sun-appserver2.1|javax.resourceadapter.mqjmsra.lifecycle|_ThreadID=18;_ThreadName=iMQEventHandler-4;_RequestID=fed5aa4c-cb5c-4a77-9bfa-0fd92b3669a3;|MQJMSRA_RA2001: onException:Reconnecting...Loop Attempt# 1elayed 5000 milliseconds.|#]
[#|2011-04-08T07:56:17.605+0100|WARNING|sun-appserver2.1|javax.jms|_ThreadID=18;_ThreadName=iMQEventHandler-4;_RequestID=fed5aa4c-cb5c-4a77-9bfa-0fd92b3669a3;|[C4003]: Error occurred on connection creation [localhost:6027]. - cause: java.net.ConnectException: Connection refused: connect|#]
[#|2011-04-08T07:56:22.386+0100|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|
parsing WSDL...
|#]
[#|2011-04-08T07:56:22.589+0100|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|
generating code...
|#]
[#|2011-04-08T07:56:22.620+0100|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|
compiling code...
|#]
[#|2011-04-08T07:56:23.214+0100|INFO|sun-appserver2.1|javax.enterprise.system.container.ejb|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|Invoking wsimport with http://uljvae01:6021/EPDQWS/EpdqWSService?WSDL|#]
[#|2011-04-08T07:56:23.214+0100|INFO|sun-appserver2.1|javax.enterprise.system.container.ejb|_ThreadID=15;_ThreadName=httpSSLWorkerThread-6021-1;|wsimport successful|#]
[#|2011-04-08T07:56:23.511+0100|WARNING|sun-appserver2.1|javax.jms|_ThreadID=18;_ThreadName=iMQEventHandler-4;_RequestID=fed5aa4c-cb5c-4a77-9bfa-0fd92b3669a3;|[C4003]: Error occurred on connection creation [localhost:6027]. - cause: java.net.ConnectException: Connection refused: connect|#]
[#|2011-04-08T07:56:29.417+0100|WARNING|sun-appserver2.1|javax.jms|_ThreadID=18;_ThreadName=iMQEventHandler-4;_RequestID=fed5aa4c-cb5c-4a77-9bfa-0fd92b3669a3;|[C4003]: Error occurred on connection creation [localhost:6027]. - cause: java.net.ConnectException: Connection refused: connect|#]
[#|2011-04-08T07:56:35.324+0100|WARNING|sun-appserver2.1|javax.jms|_ThreadID=18;_ThreadName=iMQEventHandler-4;_RequestID=fed5aa4c-cb5c-4a77-9bfa-0fd92b3669a3;|[C4003]: Error occurred on connection creation [localhost:6027]. - cause: java.net.ConnectException: Connection refused: connect|#]
[#|2011-04-08T07:56:41.230+0100|WARNING|sun-appserver2.1|javax.jms|_ThreadID=18;_ThreadName=iMQEventHandler-4;_RequestID=fed5aa4c-cb5c-4a77-9bfa-0fd92b3669a3;|[C4003]: Error occurred on connection creation [localhost:6027]. - cause: java.net.ConnectException: Connection refused: connect|#]
[#|2011-04-08T07:56:47.245+0100|WARNING|sun-appserver2.1|javax.jms|_ThreadID=18;_ThreadName=iMQEventHandler-4;_RequestID=fed5aa4c-cb5c-4a77-9bfa-0fd92b3669a3;|[C4003]: Error occurred on connection creation [localhost:6027]. - cause: java.net.ConnectException: Connection refused: connect|#]
[#|2011-04-08T07:56:53.261+0100|WARNING|sun-appserver2.1|javax.jms|_ThreadID=18;_ThreadName=iMQEventHandler-4;_RequestID=fed5aa4c-cb5c-4a77-9bfa-0fd92b3669a3;|[C4003]: Error occurred on connection creation [localhost:6027]. - cause: java.net.ConnectException: Connection refused: connect|#]
[#|2011-04-08T07:56:59.167+0100|WARNING|sun-appserver2.1|javax.jms|_ThreadID=18;_ThreadName=iMQEventHandler-4;_RequestID=fed5aa4c-cb5c-4a77-9bfa-0fd92b3669a3;|[C4003]: Error occurred on connection creation [localhost:6027]. - cause: java.net.ConnectException: Connection refused: connect|#]
[#|2011-04-08T07:57:05.074+0100|WARNING|sun-appserver2.1|javax.jms|_ThreadID=18;_ThreadName=iMQEventHandler-4;_RequestID=fed5aa4c-cb5c-4a77-9bfa-0fd92b3669a3;|[C4003]: Error occurred on connection creation [localhost:6027]. - cause: java.net.ConnectException: Connection refused: connect|#]
[#|2011-04-08T07:57:05.074+0100|SEVERE|sun-appserver2.1|javax.resourceadapter.mqjmsra.lifecycle|_ThreadID=18;_ThreadName=iMQEventHandler-4;_RequestID=fed5aa4c-cb5c-4a77-9bfa-0fd92b3669a3;|MQJMSRA_RA2001: onException:Reconnect unsuccessfull on loop# 1:[C4003]: Error occurred on connection creation [localhost:6027]. - cause: java.net.ConnectException: Connection refused: connect|#]
[#|2011-04-08T07:57:05.120+0100|WARNING|sun-appserver2.1|javax.enterprise.system.stream.err|_ThreadID=18;_ThreadName=iMQEventHandler-4;_RequestID=fed5aa4c-cb5c-4a77-9bfa-0fd92b3669a3;|com.sun.messaging.jms.JMSException: [C4003]: Error occurred on connection creation [localhost:6027]. - cause: java.net.ConnectException: Connection refused: connect
at com.sun.messaging.jmq.jmsclient.ExceptionHandler.throwConnectionException(ExceptionHandler.java:274)
at com.sun.messaging.jmq.jmsclient.ExceptionHandler.handleConnectException(ExceptionHandler.java:220)
at com.sun.messaging.jmq.jmsclient.PortMapperClient.readBrokerPorts(PortMapperClient.java:241)
at com.sun.messaging.jmq.jmsclient.PortMapperClient.init(PortMapperClient.java:150)
at com.sun.messaging.jmq.jmsclient.PortMapperClient.<init>(PortMapperClient.java:92)
at com.sun.messaging.jmq.jmsclient.protocol.tcp.TCPConnectionHandler.<init>(TCPConnectionHandler.java:164)
at com.sun.messaging.jmq.jmsclient.protocol.tcp.TCPStreamHandler.openConnection(TCPStreamHandler.java:135)
at com.sun.messaging.jmq.jmsclient.ConnectionInitiator.createConnection(ConnectionInitiator.java:778)
at com.sun.messaging.jmq.jmsclient.ConnectionInitiator.createConnectionNew(ConnectionInitiator.java:254)
at com.sun.messaging.jmq.jmsclient.ConnectionInitiator.createConnection(ConnectionInitiator.java:208)
at com.sun.messaging.jmq.jmsclient.ConnectionInitiator.createConnection(ConnectionInitiator.java:158)
at com.sun.messaging.jmq.jmsclient.ProtocolHandler.init(ProtocolHandler.java:816)
at com.sun.messaging.jmq.jmsclient.ProtocolHandler.<init>(ProtocolHandler.java:1529)
at com.sun.messaging.jmq.jmsclient.ConnectionImpl.openConnection(ConnectionImpl.java:2327)
at com.sun.messaging.jmq.jmsclient.ConnectionImpl.init(ConnectionImpl.java:1024)
at com.sun.messaging.jmq.jmsclient.ConnectionImpl.<init>(ConnectionImpl.java:418)
at com.sun.messaging.jmq.jmsclient.UnifiedConnectionImpl.<init>(UnifiedConnectionImpl.java:60)
at com.sun.messaging.jmq.jmsclient.XAConnectionImpl.<init>(XAConnectionImpl.java:58)
at com.sun.messaging.XAConnectionFactory.createXAConnection(XAConnectionFactory.java:91)
at com.sun.messaging.XAConnectionFactory.createXAConnection(XAConnectionFactory.java:69)
at com.sun.messaging.jms.ra.ResourceAdapter.onException(ResourceAdapter.java:709)
at com.sun.messaging.jmq.jmsclient.notification.EventHandler.deliverException(EventHandler.java:210)
at com.sun.messaging.jmq.jmsclient.notification.EventHandler.run(EventHandler.java:184)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:519)
at java.net.Socket.connect(Socket.java:469)
at java.net.Socket.<init>(Socket.java:366)
at java.net.Socket.<init>(Socket.java:180)
at com.sun.messaging.jmq.jmsclient.PortMapperClient.makeSocketWithTimeout(PortMapperClient.java:268)
at com.sun.messaging.jmq.jmsclient.PortMapperClient.readBrokerPorts(PortMapperClient.java:220)
... 21 more
.

============================================================================================



when I check the broker log I found this
[23/Feb/2011:07:38:30 EST] [B1066]: Closing: admin@127.0.0.1:1548->admin:1545 because "[B0061]: Client exited without closing connections". Count: service=0 broker=0

what does it mean?
I have pasted the complete broker log here in domains\domain1\imq\instances\imqbroker\log

thank you in advance
regards
Shivantha



# 1298402506937 Do not modify this line
[23/Feb/2011:06:21:47 EST]
================================================================================
Sun Java(tm) System Message Queue 4.3
Sun Microsystems, Inc.
Version: 4.3 (Build 7-g)
Compile: Fri 11/07/2008
Copyright (c) 2007 Sun Microsystems, Inc. All rights reserved. Use is
subject to license terms.
================================================================================
Java Runtime: 1.6.0_12 Sun Microsystems Inc. C:\Java\JDK1.6.0\jre
[23/Feb/2011:06:21:47 EST] IMQ_HOME=C:\Sun\AppServer\imq
[23/Feb/2011:06:21:47 EST] IMQ_VARHOME=C:\Sun\AppServer\nodeagents\ULJVAE01\uk-payment-prod\imq
[23/Feb/2011:06:21:47 EST] Windows 2003 5.2 x86 ULJVAE01 (1 cpu) SYSTEM
[23/Feb/2011:06:21:47 EST] Java Heap Size: max=194432k, current=16256k
[23/Feb/2011:06:21:47 EST] Arguments: -Dimq.cluster.nowaitForMasterBroker=true -startRmiRegistry -rmiRegistryPort 6127 -Dimq.imqcmd.user=admin -passfile C:\WINDOWS\Temp\asmq9135564275555937686.tmp -save -name ukpaymentprodukpaymentprod -port 6027 -bgnd -silent
[23/Feb/2011:06:21:47 EST] [B1004]: Starting the portmapper service using tcp [ 6027, 50, * ] with min threads 1 and max threads of 1
[23/Feb/2011:06:21:47 EST] [B1060]: Loading persistent data...
[23/Feb/2011:06:21:47 EST] Using built-in file-based persistent store: C:\Sun\AppServer\nodeagents\ULJVAE01\uk-payment-prod\imq\instances\ukpaymentprodukpaymentprod\
[23/Feb/2011:06:21:50 EST] WARNING [B3168]: Invalid broker address for this broker to run in cluster: Loopback IP address is not allowed in broker address mq://127.0.0.1:6027/?instName=ukpaymentprodukpaymentprod&brokerSessionUID=6111717085383758848 for cluster
[23/Feb/2011:06:21:50 EST] WARNING [B1137]: Cluster initialization failed. Disabling the cluster service.
[23/Feb/2011:06:21:50 EST] [B1136]: Processing stored transactions
[23/Feb/2011:06:21:50 EST] [B1284]: 0 cluster transactions remain in PREPARED state, 0 in COMMITTED state waiting for remote broker completion
[23/Feb/2011:06:21:50 EST] Of 0 remote transactions, 0 in PREPARED state, 0 in COMMITTED state
[23/Feb/2011:06:21:50 EST] [B1013]: Auto Creation of Queues is enabled
[23/Feb/2011:06:21:50 EST] [B1151]: Loading destination mq.sys.dmq [Queue] with 0 messages
[23/Feb/2011:06:21:50 EST] [B1152]: Loading of destination mq.sys.dmq [Queue] complete
[23/Feb/2011:06:21:50 EST] [B1239]: Using platform MBean server
[23/Feb/2011:06:21:52 EST] RMI Registry started on port 6127
[23/Feb/2011:06:21:52 EST] JESMF classes not present - JESMF support will not be enabled.
[23/Feb/2011:06:21:52 EST] JMX Connector Server jmxrmi started successfully with url service:jmx:rmi://ULJVAE01/jndi/rmi://ULJVAE01:6127/ULJVAE01/6027/jmxrmi
[23/Feb/2011:06:21:52 EST] [B1004]: Starting the admin service using tcp(host = *, port=0, mode=dedicated) with min threads 4 and max threads of 10
[23/Feb/2011:06:21:52 EST] [B1227]: Using file user repository for admin service connection authentication
[23/Feb/2011:06:21:52 EST] [B1004]: Starting the jms service using tcp(host = *, port=0, mode=dedicated) with min threads 10 and max threads of 1000
[23/Feb/2011:06:21:52 EST] [B1227]: Using file user repository for jms service connection authentication
[23/Feb/2011:06:21:52 EST] [B1039]: Broker "ukpaymentprodukpaymentprod@ULJVAE01:6027" ready.
[23/Feb/2011:06:21:52 EST] [B1065]: Accepting: admin@127.0.0.1:1548->admin:1545. Count: service=1 broker=1
[23/Feb/2011:06:21:52 EST] [B1065]: Accepting: guest@127.0.0.1:1551->jms:1543. Count: service=1 broker=2
[23/Feb/2011:07:38:30 EST] [B1047]: Shutting down broker...
[23/Feb/2011:07:38:30 EST] [B1077]: Broadcast good-bye to all connections ...
[23/Feb/2011:07:38:30 EST] [B1078]: Flushing good-bye messages ...
[23/Feb/2011:07:38:30 EST] [B1007]: Stopping Service admin with protocol tcp(host = *, port=0, mode=dedicated)
[23/Feb/2011:07:38:30 EST] [B1007]: Stopping Service jms with protocol tcp(host = *, port=0, mode=dedicated)
[23/Feb/2011:07:38:30 EST] [B1066]: Closing: guest@127.0.0.1:1551->jms:1543 because "[B0060]: Service shutting down". Count: service=0 broker=1
[23/Feb/2011:07:38:30 EST] [B1066]: Closing: admin@127.0.0.1:1548->admin:1545 because "[B0061]: Client exited without closing connections". Count: service=0 broker=0
[23/Feb/2011:07:38:30 EST] JMX Connector Server jmxrmi stopped successfully
[23/Feb/2011:07:38:30 EST] [B1048]: Shutdown of broker complete.
[23/Feb/2011:07:41:23 EST]
.
 
It is sorta covered in the JavaRanch Style Guide.
 
subject: GlassFish 2.1.1 server hangs up due to unknown broker shutdown request from Admin