aspose file tools*
The moose likes Tomcat and the fly likes Tomcat fails to restart with Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Soft Skills this week in the Jobs Discussion forum!
JavaRanch » Java Forums » Products » Tomcat
Bookmark "Tomcat fails to restart with "Address already in use" exception" Watch "Tomcat fails to restart with "Address already in use" exception" New topic
Author

Tomcat fails to restart with "Address already in use" exception

Alexey Andreev
Greenhorn

Joined: Jun 15, 2009
Posts: 4
Hi!

Our custom java web application is running on:

Dedicated Linux server, Tomcat 5.5.23.0, JVM 1.5.0_12

Tomcat run parameters:

java -server -Xms1000m -Xmx2000m -Xmn756m -XX:MaxPermSize=128m -Djava.awt.headless=true -verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -Xloggc:/usr/local/tomcat/logs/gc.log -XX:+UseParallelGC -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.util.logging.config.file=/usr/local/tomcat/conf/logging.properties -Djava.endorsed.dirs=/usr/local/tomcat/common/endorsed -classpath :/usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/commons-logging-api.jar -Dcatalina.base=/usr/local/tomcat -Dcatalina.home=/usr/local/tomcat -Djava.io.tmpdir=/usr/local/tomcat/temp org.apache.catalina.startup.Bootstrap start

Recently we started getting a strange problem while restarting the tomcat.

After running "service tomcat restart" (or stop-start it actually doesn't matter) the server fails to start and the following is written to catalina.out :



No other software is running on port 8080 on the server. And I even check that there is no active connection before restart, and still get the same.
If I do the second restart of tomcat after several minutes it goes fine and quick, but if I try to do it after an hour it almost everytime fails.

Any help is appreciated.

Thank you very much in advance.



Damayanti Samal
Greenhorn

Joined: Sep 09, 2008
Posts: 3
sometimes it found same , just you change the port number to some other port like 7777 and check .
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12835
    
    5
Exactly how are you starting and stopping Tomcat.

Exactly how are you checking the 8080 port for an existing user?

Bill
Kees Jan Koster
JavaMonitor Support
Rancher

Joined: Mar 31, 2009
Posts: 251
    
    5
Dear Alexey,

You can check who is listening using the commands in this article: http://java-monitor.com/forum/showthread.php?t=22

Make sure you check on the right interface on a multi-homed machine. Also, there may be background processes automatically starting Tomcat servers. Finally, it is very easy for developers to make a mistake so that Tomcat will not terminate properly. Thus, after stopping Tomcat, check that is dead before starting. So don't do "restart", but "stop, check, start".


Java-monitor, JVM monitoring made easy <- right here on Java Ranch
Alexey Andreev
Greenhorn

Joined: Jun 15, 2009
Posts: 4
Thanks for your replies.

Before restart I am stopping monitoring software which watches for tomcat process (MONIT). So it can not be started automatically.
I stop the tomcat. Check the process with "ps -ef | grep java". Check the 8080 port with "netstat -t -a -n -v | grep :8080"
Everything is clean and I do start. And get the exception in logs.

Must admit that often stop command takes a long time to complete if it can give you a clue.

I am using s standard tomcat service script for starts/stops.

here is an excerpt from it

START:




STOP:

Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 16303
    
  21

Tomcat shuts down in an orderly way. If one or more of the installed webapps isn't well-behaved, you'll have problems. One of the most common of these is webapps that run a scheduler service such as Quartz and don't shutdown the Quartz scheduler threads (in the servlet destroy() method). That will hang the Tomcat shutdown process forever - or until you forcibly terminate the JVM, since as long as even one thread is alive, the JVM cannot terminate.

Ordinarily the Tomcat shutdown process happens fairly quickly. The "Tomcat restart" process consists of a catalina stop, immediately followed by catalina start. Both of these operations are asynchronous - they don't wait for any sort of event to be posted before returning. So if shutdown takes too long, the catalina start may fail because the catalina stop is still running and hasn't yet released the server network ports.

The cure for that is to either make the webapp shut down faster, or to introduce a delay between stop and start.


Customer surveys are for companies who didn't pay proper attention to begin with.
Sherif Shehab
Ranch Hand

Joined: Mar 05, 2007
Posts: 483

Damayanti Samal wrote:sometimes it found same , just you change the port number to some other port like 7777 and check .


Right try to change the port to anything else rather 8080 and it will works fine


Thanks,
Sherif
Alexey Andreev
Greenhorn

Joined: Jun 15, 2009
Posts: 4
Sherif Shehab wrote:
Damayanti Samal wrote:sometimes it found same , just you change the port number to some other port like 7777 and check .


Right try to change the port to anything else rather 8080 and it will works fine


I will be surprised if that will help, but why not to try. I changed the port to 8084. I will report the situation with restarts in the next couple of days.
Alexey Andreev
Greenhorn

Joined: Jun 15, 2009
Posts: 4
Nope, all the same on another port (used 8084).

I also checked quartz and it shutdowns properly.

It's all a big mystery for me.

Maybe something connected directly with Linux OS? Sockets left in TIMED_WAIT state or any tricky things?
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 16303
    
  21

Alexey Andreev wrote:

Maybe something connected directly with Linux OS? Sockets left in TIMED_WAIT state or any tricky things?


I doubt it. I've worked with Tomcat since about 2.0. When the apps are all shut down, the rest of the server quickly follows. You might try the JMX console and see if there's something still happening. Or put the whole thing under a debugger and see what threads hang around the longest.

An easy way to tell if it's Tomcat or something in an app is to remove all the webapps and see if it still holds the port too long after shutdown.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Tomcat fails to restart with "Address already in use" exception