File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes JBoss/WildFly and the fly likes Server stopped responding for almost 10 minutes and resumed to normal state. Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Products » JBoss/WildFly
Bookmark "Server stopped responding for almost 10 minutes and resumed to normal state." Watch "Server stopped responding for almost 10 minutes and resumed to normal state." New topic
Author

Server stopped responding for almost 10 minutes and resumed to normal state.

Uttam Chauhan
Greenhorn

Joined: Mar 27, 2011
Posts: 3
Hello,

We have faced an issue with production JBoss server. Server stopped responding for almost 10 minutes and returned back to normal state after it. During this interval clients were not able to connect and no error or warning messages were logged to log files. However we have an known and open bug in the deployed application. One connection is not closed and Resultset and Statement is also open on this connection, could this be cause of this problem ? Same application is deployed on other three JBoss server but none of these server had this issue, though waring "Closing a connection for you" is being logged frequently on all the four servers. Datasource is configured in mssql-ds.xml as non transactional DS. How JBoss Handles open resources or its Database Driver's call to handle the open resources? Here are the Environment and server details.

Environment Details :
Os : Linux 2.6.18-194.17.4.el5 (i386) CPU : 1

JVM Name : Java HotSpot(TM) Server VM

JVM version : 1.5.0_03-b07 (Sun Microsystems Inc.)

Max Memory : 512 MB

Min Memory : 128 MB

Database : SQL Server 2008

Driver being used : sqljdbc 2

JBoss Version: 4.0.4GA(build: CVSTag=JBoss_4_0_4_GA date=200605151000)

Version Name: Zion

I find this a very strange problem as we did not get any errors during the freeze.I hope there is someone who can help us.

Thanks in advance.


Peter Johnson
author
Bartender

Joined: May 14, 2008
Posts: 5845
    
    7

Sounds like a huge garbage collection pause. Are you by any chance running the CMS collector? There are ways of determining if GC is the issue, but not after the fact. For suggestions on how to gather GC data and analyze it, see these white papers:
Java Garbage Collection Statistical Analysis 101
Java Garbage Collection Performance Analysis 201
at http://www.cmg.org/cgi-bin/search.cgi?q=java+peter+johnson&x=30&y=10



JBoss In Action
Uttam Chauhan
Greenhorn

Joined: Mar 27, 2011
Posts: 3
Hi Peter,

Thanks a lot for you reply. We got right direction to work.

Could you please also provide some help on these queries.

1 I monitored logs there was continuous logging in server.log during the interval for which client was not able to connect, so looks like JBoss was working in this duration, So only client was not able to connect for 10 minutes, In what situation this could happen ?

2 How are the resources left opened like Resultset, Statements and Connection are handled by JBoss if they are configured in *-ds.xml in JBoss.(What if developer forgot to close them). ?



Peter Johnson
author
Bartender

Joined: May 14, 2008
Posts: 5845
    
    7

Usually if a request doesn't close a datasource connection (or actually, release it back into the connection pool), the app server will print a warning in the log file and release the connection for you. This is, of course, if you are using a data source (have a *-ds.xml) and the code looks up the data source in JNDI.

One way that you could end up not connecting for many minutes is if all of the threads in the thread pool are active, in which case other request will have to wait until one of the threads finishes an becomes free.

What kind of requests are coming in? HTTP to servlets/JSPs/JSF? EJB requests? MDB requests?
Uttam Chauhan
Greenhorn

Joined: Mar 27, 2011
Posts: 3
Hi Peter,

Yes exactly same warning we are getting for unclosed connection, But I am more interested to know about open Statements and open Resultsets on a pooled connection (configured in *-ds.xml and being retrieved from JNDI).

I think all the threads being active is most promising cause of server not responding as we have multiple long database conversations for a single request (each request spends more than 500 millis in database conversations). So If there are simultaneous long running requests than thread pool will be fully consumed. Am i right ?

Request coming in are HTTP request to a servlet, a stateless session bean works at middle layer.

Once again thank you.

 
 
subject: Server stopped responding for almost 10 minutes and resumed to normal state.