wood burning stoves 2.0*
The moose likes Tomcat and the fly likes Can Not Keep Tomcat Up and running ~30 mins Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of The Java EE 7 Tutorial Volume 1 or Volume 2 this week in the Java EE forum
or jQuery UI in Action in the JavaScript forum!
JavaRanch » Java Forums » Products » Tomcat
Bookmark "Can Not Keep Tomcat Up and running ~30 mins" Watch "Can Not Keep Tomcat Up and running ~30 mins" New topic
Author

Can Not Keep Tomcat Up and running ~30 mins

Bob OConnor
Greenhorn

Joined: Mar 24, 2005
Posts: 2
Ok our fairly active website was moved to new servers
about a month ago and all is well. In the past on
about a monthly basis the tomcat would fail
and be rebooted, cause not found.

This is the first failure in under a month on the new servers....
So I rebooted:
catalina.sh stop -force
catalina.sh start

Runs for about 30 to 60 mins and fails AGAIN...
I suspect some type of memory leak OR OR....
I look forward to some help as to what is going
on, and why NOW...
We've done nothing different, no software changes etc.

Ok
Basics:

OS RHEL 3.3
Kernel 2.4.21-15.ELsmp
MySql 4.0.23
Tomcat 5.0.28
JDK 1.4.2-07
---------------

Here is the page a user gets when tomcat finally fails:


--------------

"top" when tomcat no longer serving ERRORED OUT...



Notice Low Memory.

--------------

"top" when catalina stopped


--------------

"top" when catalina restarted and running 5 minutes.


--------------

Here is the page user gets in browser when tomcat finally fails:


--------------

Here is a bit from this log:
/home/app/tomcat/logs/...host_log.txt


--------------

And from this log:



Then the stopping catalina and re starting.

--------------

Suggestions are MOST welcome
and TIA!

-Bob OConnor
Ben Souther
Sheriff

Joined: Dec 11, 2004
Posts: 13410

It sounds like your app is not closing connections or is not returning them to a connection pool (if you have one).
This will cause a memory leak but it looks like your crashing from maxing out the number of connections that the DB will give you first.

I'm guessing that someone tried fixing the problem on the old servers by increasing the number of allowed connections and/or the amount of intial and max memory settings in the JVM. This of course will only postpone the consequenses.


Java API J2EE API Servlet Spec JSP Spec How to ask a question... Simple Servlet Examples jsonf
Jeffrey Spaulding
Ranch Hand

Joined: Jan 15, 2004
Posts: 149
Yes Ben is right.

I have a server that had the same problem. If you have ample Memory and the load isnt too high you can get away with assigning lots of Memory to the JVM, increasing the number of Connections and restarting when nobody looks.

I have a server to manage with the same problem and that was my last resort since nobody would pay to debug the application that causes the problem.



Even wrote a watchdog that restarts by force when the crash is imminent (you can see that in memory usage -- UUUUH slimy hack)

J.
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12769
    
    5
Even wrote a watchdog that restarts by force when the crash is imminent (you can see that in memory usage -- UUUUH slimy hack)

J.

--------------------

The day HP shipped the first calculator with a [=] key, things started to become worse.

UUUH is right - how do you get the watchdog to do a clean shutdown and restart?
Incidently - I am with you on the HP calculator with an [=] key
Bill
Bob OConnor
Greenhorn

Joined: Mar 24, 2005
Posts: 2
Thanks, Ben, Jeffery and William.

Shortly after I wrote this post, the server stayed up and has now been up
20 hours. I had considered writing a cron job to stop catalina and then
a minute later restarting.... at least while I was asleep!

OTOH, could there be a "bad" page that uses up the connections?
Or
Is there a step I can use to look to see that all connections are
closed/available after a Stop?

TIA
-Bob O
Ben Souther
Sheriff

Joined: Dec 11, 2004
Posts: 13410

Unable to get connection, DataSource invalid: "org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool exhausted, cause: Timeout waiting for idle object"


I would take a look at some of the documentation for http://jakarta.apache.org/commons/dbcp.
That is what Tomcat is using.
I'm sure you can configure it and maybe up the logging a bit.
Tariq Dweik
Greenhorn

Joined: Aug 19, 2001
Posts: 20
Hi all,

I'm facing the same issue here, I've noticed that all of us r using tomcat 5.0.28, could it be ?

Thanks


Tariq Dweik
Jeffrey Spaulding
Ranch Hand

Joined: Jan 15, 2004
Posts: 149
Originally posted by William Brogden:

UUUH is right - how do you get the watchdog to do a clean shutdown and restart?
Incidently - I am with you on the HP calculator with an [=] key
Bill


Sorry wasn't on for some time.

Well [fat grin] the watchdog is a perl script that runs from a
cronjob. When the crash is on the horizon tomcat is still alive,
so i do a



It's that kind of slimy hack.

 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Can Not Keep Tomcat Up and running ~30 mins