aspose file tools*
The moose likes Tomcat and the fly likes Tomcat Issue Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Soft Skills this week in the Jobs Discussion forum!
JavaRanch » Java Forums » Products » Tomcat
Bookmark "Tomcat Issue" Watch "Tomcat Issue" New topic
Author

Tomcat Issue

Sanyog Singh
Greenhorn

Joined: Oct 06, 2008
Posts: 2
Hi ,

I have recently faced a typical issue in my application recently. I would appreciate any guidance to progress further on troubleshooting of the issue.

I am trying to provide as much information as I can for your review below.

I have an application based on JSP and Servlets, the web container is Tomcat 6.0.

The front interface is Apache which redirects the incoming request to Tomcat 6.0. The Apache to Tomcat redirection is set web modjk connector.

Apache is configured to have a secured protocol - https with SSL certificate.

The application is having good error logging mechanism for exceptions, which gets stored in DB. There is also a provision to log DB connection errors in Tomcat logs so that if somehow the application could not write to DB due to a DB error, that log will appear in the Tomcat logs.

The application was running fine without any issues until recently when we suddenly had a situation in which some of the online processes of the application started behaving in unexpected manner. We had 5 steps in one process, which got lost after two steps, before moving to the third step. The users always saw an error message, which is a custom error message raised by the application on account of the exception. However, the most unexpected situation was that there were no logs in the DB error log OR Tomcat logs. There was no error indicated by those mechanisms.

It was looking like the application did encounter an error, every time after two steps in the online process but somehow couldn't log it anywhere. We are not sure how would that happen. We did the following checks to make sure that the application is having everything in place to handle the situation:

1. DB Error logging - we verified again that in case of exception, in any given situation the application would log the error stack trace and message in the DB. There are no chances that would be missed.

2. Alternative logging - The application is having a proper mechanism to log a DB connection related error in Tomcat logs. Therefore, if we consider a situation in which the application just couldn't write to DB for an error logging, that error would have gone to Tomcat logs.

3. Testing - The testing is carried out in two replica environments for couple of weeks but no issues in those environments.

4. Apache to Tomcat redirection - We also thought at one point of time that could this be a redirection issue. However, that was ruled out because the users see the application custom error message every time after the second step, which indicates that the request is reaching properly to Tomcat in all instances.

The problem got resolved by Tomcat restart. Which again doesn't explain the solution as the root cause is still unknown.

Based on my experience so far, there are two situations which Tomcat restarts normally resolves:

1. Out of Memory errors - When a memory leak in the application causes "Out of Memory" after a continuous processing over a period of time, Tomcat restart resolves the error as it clears the memory.

2. Connection over limit issues - When there is a connection leak, the application runs out of connection over a period of time. Either the DB cursors get over limit or the connections managed by the application get over the limit of total number of connections allowed by DB. Tomcat restarts releases the connections and resolves the issue.

However, in the recent issue of my application, none of these were the root cause as the application was progressing until the third step in the online process, every time. The first two stages do use DB connections.

I have searched for more than a week for similar issues or problems on internet but couldn't find anything so far. Based on that, I thought it is now time to take expert advise as I have done everything that I could. I am continuing my research further but any help would be very very helpful in the current situation.

Looking forward to your kind attention and comments/suggestions to help problem investigation.

Best regards,

Sanyog
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12835
    
    5
You might try the Tomcat Management application to get a feel for whats happening in terms of memory consumption and request Threads.

Bill
steve souza
Ranch Hand

Joined: Jun 26, 2002
Posts: 861
JAMon may be able to provide some info. I suggest using the tomcat jamon valve to track page hits and exceptions. Also use the jamon proxy jdbc driver which will monitor all sql statements as well as any exceptions they throw. You will also be able to tell which pages and queries are executing at any given time.

Both can be set up in a manner of minutes.

Here is how to setup monitoring in tomcat 6:
http://jamonapi.sourceforge.net/http_monitoring.html#intro

Here is how to setup the jdbc proxy monitoring driver.
http://jamonapi.sourceforge.net/#WhatsNew22


http://www.jamonapi.com/ - a fast, free open source performance tuning api.
JavaRanch Performance FAQ
Sanyog Singh
Greenhorn

Joined: Oct 06, 2008
Posts: 2
Thank you Steve.

I will install the tool and use it for troubleshooting.

I will revert back to you if I will require further information.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Tomcat Issue