• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Tim Cooke
  • Liutauras Vilda
  • Jeanne Boyarsky
  • paul wheaton
Sheriffs:
  • Ron McLeod
  • Devaka Cooray
  • Henry Wong
Saloon Keepers:
  • Tim Holloway
  • Stephan van Hulst
  • Carey Brown
  • Tim Moores
  • Mikalai Zaikin
Bartenders:
  • Frits Walraven

Websphere Server Getting Down Every Saturday

 
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,
I am new to WebSphere and unable to know the exact causes of the issues.

On every Saturday the TCP Port listening is stopped and the deployed wars are shutting down.
[3/10/19 0:00:01:740 IST] 0000001f TCPChannel    I   TCPC0002I: TCP Channel TCP_4 has stopped listening on host *  (IPv6) port 9444.
[3/10/19 0:00:01:742 IST] 0000001f TCPChannel    I   TCPC0002I: TCP Channel TCP_2 has stopped listening on host *  (IPv6) port 9081.
[3/10/19 0:00:01:747 IST] 0000001f JMSProxyRunti I   XJMS0005I: XD JMS message pacing controller successfully stopped.
[3/10/19 0:00:01:747 IST] 0000001f ApplicationMg A   WSVR0217I: Stopping application: eda_war
[3/10/19 0:00:01:941 IST] 0000001f servlet       I com.ibm.ws.webcontainer.servlet.ServletWrapper doDestroy SRVE0253I: [eda_war] [/eda] [SLOSamlLogoutController]: Destroy successful.
[3/10/19 0:00:01:944 IST] 0000001f servlet       I com.ibm.ws.webcontainer.servlet.ServletWrapper doDestroy SRVE0253I: [eda_war] [/eda] [SloLogoutResponseConsumer]: Destroy successful.
[3/10/19 0:00:01:946 IST] 0000001f servlet       I com.ibm.ws.webcontainer.servlet.ServletWrapper doDestroy SRVE0253I: [eda_war] [/eda] [DownloadPDF]: Destroy successful.
[3/10/19 0:00:01:946 IST] 0000001f servlet       I com.ibm.ws.webcontainer.servlet.ServletWrapper doDestroy SRVE0253I: [eda_war] [/eda] [SSOSamlResponse]: Destroy successful.
[3/10/19 0:00:01:947 IST] 0000001f servlet       I com.ibm.ws.webcontainer.servlet.ServletWrapper doDestroy SRVE0253I: [eda_war] [/eda] [com.edist.common.servlet.UpLoadpdfServlet]: Destroy successful.
[3/10/19 0:00:01:947 IST] 0000001f servlet       I com.ibm.ws.webcontainer.servlet.ServletWrapper doDestroy SRVE0253I: [eda_war] [/eda] [Captcha]: Destroy successful.
[3/10/19 0:00:01:948 IST] 0000001f servlet       I com.ibm.ws.webcontainer.servlet.ServletWrapper doDestroy SRVE0253I: [eda_war] [/eda] [SSOSamlRequest]: Destroy successful.
[3/10/19 0:00:02:112 IST] 00000040 TransportAdap W   DCSV1117W: DCS Stack DefaultCoreGroup at Member ip-20-0-8-70Cell01\ip-20-0-8-70Node01\server1: The stream from Member ip-20-0-8-70Cell01\ip-20-0-8-70Node01\nodeagent has closed. The channnel is View|Ptp.
[3/10/19 0:00:02:139 IST] 00000040 VSyncAlgo1    I   DCSV2004I: DCS Stack DefaultCoreGroup at Member ip-20-0-8-70Cell01\ip-20-0-8-70Node01\server1: View synchronization completed successfully. The View Identifier is (3:0.ip-20-0-8-70Cell01\ip-20-0-8-70CellManager01\dmgr). The internal details are None.

[3/10/19 0:00:02:914 IST] 0000004a DiscoveryRcv  W   DCSV1115W: DCS Stack DefaultCoreGroup at Member ip-20-0-8-70Cell01\ip-20-0-8-70Node01\server1: Member ip-20-0-8-70Cell01\ip-20-0-8-70CellManager01\dmgr connection  was closed. Member will  be removed from view. DCS connection status is Discovery|Ptp, receiver closed.
[3/10/19 0:00:02:918 IST] 0000003f RoleViewLeade I   DCSV8053I: DCS Stack DefaultCoreGroup at Member ip-20-0-8-70Cell01\ip-20-0-8-70Node01\server1: View change in process. Excluded members are [ip-20-0-8-70Cell01\ip-20-0-8-70CellManager01\dmgr].
[3/10/19 0:00:02:919 IST] 00000040 VSyncAlgo1    I   DCSV2004I: DCS Stack DefaultCoreGroup at Member ip-20-0-8-70Cell01\ip-20-0-8-70Node01\server1: View synchronization completed successfully. The View Identifier is (4:0.ip-20-0-8-70Cell01\ip-20-0-8-70CellManager01\dmgr). The internal details are None.
[3/10/19 0:00:02:924 IST] 0000003f CoordinatorIm I   HMGR0218I: A new core group view has been installed. The core group is DefaultCoreGroup. The view identifier is (5:0.ip-20-0-8-70Cell01\ip-20-0-8-70Node01\server1). The number of members in the new view is 1.
[3/10/19 0:00:02:924 IST] 0000003f CoreGroupMemb I   DCSV8050I: DCS Stack DefaultCoreGroup at Member ip-20-0-8-70Cell01\ip-20-0-8-70Node01\server1: New view installed, identifier (5:0.ip-20-0-8-70Cell01\ip-20-0-8-70Node01\server1), view size is 1 (AV=1, CD=1, CN=1, DF=3)
[3/10/19 0:00:02:927 IST] 0000006c ViewReceiver  I   DCSV1033I: DCS Stack DefaultCoreGroup at Member ip-20-0-8-70Cell01\ip-20-0-8-70Node01\server1: Confirmed all new view members in view identifier (5:0.ip-20-0-8-70Cell01\ip-20-0-8-70Node01\server1). View channel type is View|Ptp.
[3/10/19 0:00:02:934 IST] 0000003f CoordinatorIm I   HMGR0206I: The Coordinator is an Active Coordinator for core group DefaultCoreGroup. The active coordinator set is [ip-20-0-8-70Cell01\ip-20-0-8-70Node01\server1].
[3/10/19 0:00:03:381 IST] 00000053 Peer          I   ODCF8534I: Removed neighbor ip=20.0.8.70 udp=11005 tcp=11006 ID=e61cb44475513adb8b69ea9d1dfc80b6754fbb94 version=0;cellName=ip-20-0-8-70Cell01;bridgedCells=[];properties={MEMBER_STARTUP_TIME=1552041121755, MEMBER_VERSION=4, memberName=ip-20-0-8-70Cell01\ip-20-0-8-70CellManager01\dmgr, epoch=1552041123328, inOdc=1, ODC_PUBLISHER_ONLY=false}, neighbor set is now 0 nodes
.
[3/10/19 0:00:03:387 IST] 00000088 P2PGroup      I   ODCF8041I: Detected process ip-20-0-8-70Cell01\ip-20-0-8-70CellManager01\dmgr stopped.
[3/10/19 0:00:03:389 IST] 00000088 ServerStopped I   ODCF0009I: Marked server /cell/ip-20-0-8-70Cell01/node/ip-20-0-8-70CellManager01/server/dmgr STOPPED due to transaction: remote - markServerDown: ip-20-0-8-70Cell01\ip-20-0-8-70CellManager01\dmgr
[3/10/19 0:00:03:396 IST] 00000055 NGUtil$Server I   ASND0003I: Detected server dmgr stopped on node ip-20-0-8-70CellManager01
[3/10/19 0:00:22:072 IST] 00000030 ServerStopped I   ODCF0009I: Marked server /cell/ip-20-0-8-70Cell01/node/ip-20-0-8-70Node01/server/nodeagent STOPPED due to transaction: remote - markServerDown: ip-20-0-8-70Cell01\ip-20-0-8-70Node01\nodeagent
[3/10/19 0:01:01:959 IST] 0000001f servlet       I com.ibm.ws.webcontainer.servlet.ServletWrapper doDestroy SRVE0253I: [eda_war] [/eda] [Faces Servlet]: Destroy successful.
[3/10/19 0:01:02:009 IST] 0000001f ApplicationMg A   WSVR0220I: Application stopped: eda_war
[3/10/19 0:01:02:010 IST] 0000001f ApplicationMg A   WSVR0217I: Stopping application: ibmasyncrsp
[3/10/19 0:01:02:021 IST] 0000001f servlet       I com.ibm.ws.webcontainer.servlet.ServletWrapper doDestroy SRVE0253I: [ibmasyncrsp] [/IBM_WS_SYS_RESPONSESERVLET] [rspservlet]: Destroy successful.
[3/10/19 0:01:02:031 IST] 0000001f ApplicationMg A   WSVR0220I: Application stopped: ibmasyncrsp
[3/10/19 0:01:02:031 IST] 0000001f ApplicationMg A   WSVR0217I: Stopping application: edamnpservice_war
[3/10/19 0:01:02:046 IST] 0000001f servlet       I com.ibm.ws.webcontainer.servlet.ServletWrapper doDestroy SRVE0253I: [edamnpservice_war] [/edaservice] [com.eda.service.service.StatusSearchService]: Destroy successful.
[3/10/19 0:01:02:046 IST] 0000001f servlet       I com.ibm.ws.webcontainer.servlet.ServletWrapper doDestroy SRVE0253I: [edamnpservice_war] [/edaservice] [com.edist.webservices.accountservice_jaxws.StatusSearchService]: Destroy successful.
[3/10/19 0:01:02:068 IST] 0000001f ApplicationMg A   WSVR0220I: Application stopped: edamnpservice_war
[3/10/19 0:01:02:073 IST] 0000001f SchedulerServ I   SCHD0040I: The Scheduler Service is stopping.
[3/10/19 0:01:02:074 IST] 0000001f SchedulerServ I   SCHD0002I: The Scheduler Service has stopped.
[3/10/19 0:01:02:075 IST] 0000001f AppProfileCom I   ACIN0009I: The application profiling service is stopping.
[3/10/19 0:01:02:075 IST] 0000001f ActivitySessi I   WACS0049I: The ActivitySession service is stopping.
[3/10/19 0:01:02:083 IST] 0000001f ObjectPoolSer I   OBPL0011I: The Object Pool service is stopping.
[3/10/19 0:01:02:086 IST] 0000001f distSecurityC I   securityServiceStarted is false
[3/10/19 0:01:02:103 IST] 0000001f CGBridgeSubsc I   CWRCB0104I: The core group bridge service has stopped the subscription router.
[3/10/19 0:01:02:103 IST] 0000001f CGBridgeServi I   CWRCB0103I: The core group bridge service has stopped.
[3/10/19 0:01:02:104 IST] 0000001f DragDropDeplo I   CWLDD0004I: Stopping monitored directory application deployment service...
[3/10/19 0:01:02:104 IST] 0000001f DragDropDeplo I   CWLDD0005I: Monitored directory application deployment service is stopped.
[3/10/19 0:01:02:106 IST] 0000001f DCSStackImpl  I   DCSV8011I: DCS Stack DefaultCoreGroup at Member ip-20-0-8-70Cell01\ip-20-0-8-70Node01\server1: Member received a request for shutdown.
[3/10/19 0:01:02:127 IST] 0000001f CfwTCPListene I   ODCF8525I: The unstructured overlay is stopping its inbound TCP channel.
[3/10/19 0:01:02:127 IST] 0000001f Peer          I   ODCF8526I: The unstructured overlay is stopping its outbound TCP channel.
[3/10/19 0:01:02:127 IST] 0000001f Peer          I   ODCF8550I: The current TCP protocol statistics for the unstructured overlay member {20.0.8.70  udp_port=11003  tcp_port=11004} are {connOpened=17, connClosed=17, read={buffAloc=32559, byteAloc=33523848, buffReld=32559, byteReld=33523848}, write={buffAloc=2989, byteAloc=17210368, buffReld=2989, byteReld=17210368} }.
[3/10/19 0:01:02:142 IST] 0000001f TCPChannel    I   TCPC0002I: TCP Channel TCPInboundChannel_ipcc.Default_IPC_Connector_Name has stopped listening on host localhost  (IPv4: 127.0.0.1) port 9634.
[3/10/19 0:01:02:183 IST] 0000001f SystemOut     O WARN: The method class org.apache.commons.logging.impl.SLF4JLogFactory#release() was invoked.
[3/10/19 0:01:02:183 IST] 0000001f SystemOut     O WARN: Please see http://www.slf4j.org/codes.html#release for an explanation.
[3/10/19 0:01:02:205 IST] 0000001f FailureScopeC A   WTRN0105I: The transaction service has shutdown successfully with no transactions requiring recovery.
[3/10/19 0:01:02:213 IST] 0000001f TCPChannel    I   TCPC0002I: TCP Channel sonIncomingTcpConnectorInboundChannel_SonIncomingTCPSubsystem_0_11004 has stopped listening on host ip-20-0-8-70.ap-south-1.compute.internal  (IPv4: 20.0.8.70) port 11004.
[3/10/19 0:01:02:214 IST] 0000001f TCPChannel    I   TCPC0002I: TCP Channel TCP_5 has stopped listening on host *  (IPv6) port 9353.
[3/10/19 0:01:02:217 IST] 0000001f ServerCollabo A   WSVR0024I: Server server1 stopped
 
Saloon Keeper
Posts: 15731
368
  • Likes 2
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
This almost looks like a scheduled job. There's nobody running scripts to update/restart services?
 
Bartender
Posts: 1381
39
IBM DB2 Netbeans IDE Spring Java
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I would suggest you to talk to your system administrator. The trace you posted looks like a normal shutdown operation issued by a remote agent controller,not a failure.
 
Sheriff
Posts: 28333
97
Eclipse IDE Firefox Browser MySQL Database
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:This almost looks like a scheduled job.



Especially since it appears to start at just about exactly midnight...

And, welcome to the Ranch, Naobisana!
 
Saloon Keeper
Posts: 3950
43
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
check which user runs WAS process in OS
run "crontab -l" as that user
inspect if WAS stopped by cron
 
If you are using a wood chipper, you are doing it wrong. Even on this tiny ad:
Gift giving made easy with the permaculture playing cards
https://coderanch.com/t/777758/Gift-giving-easy-permaculture-playing
reply
    Bookmark Topic Watch Topic
  • New Topic