• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Tomcat 6 Manage Unstable - Deployments Require Server Bounce

 
Gregg Bolinger
GenRocket Founder
Ranch Hand
Posts: 15302
6
Chrome IntelliJ IDE Mac OS X
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm trying to automate some deployments to our test servers and I run into all sorts of different issues when using the Tomcat manager for deployment. JARs get corrupted, memory issues, etc. If I stop tomcat, copy in the WAR, and start tomcat, things work flawlessly.

Is this a known issue? Is there anything I can do to make deployments smoother without bouncing the server entirely?
 
Tim Holloway
Saloon Keeper
Pie
Posts: 18100
51
Android Eclipse IDE Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Well, first and foremost there's the dreaded PermGenSpace issue. Because Tomcat isn't cleaning up the old app sufficiently well, "orphan" memory in the PermGen region (which is fairly small) causes redeployments to fail.

Then there are the work and tempfiles, which contained old cached bits of stuff that needs to be flushed out. For example, if you remove a JSP definition in the new version of the WAR, that doesn't automatically remove its compiled form in the Tomcat work directory, which is why I delete the stuff under the work, temp and logs directories when I redeploy (clearing logs keeps me from reading old logs and getting false impressions).

Finally, there are problems with hot-deployments where servlet initialization isn't re-executed. I see this mostly in ORM-based apps where the old database pool config doesn't update.

You can reduce the problems by stopping the app, deploying and restarting it, but some problems are likely to remain. It may serve for emergency purposes, but the only surefire way to get a clean deploy is to stop the server, purge the work directories and restart. Fortunately, Tomcat comes up fast, although no all webapps do. If downtime is simply unacceptable, of course, you should be running a cluster anyway, and you can update/bounce the cluster nodes one at a time unless something really major changes.
 
Gregg Bolinger
GenRocket Founder
Ranch Hand
Posts: 15302
6
Chrome IntelliJ IDE Mac OS X
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Tim. It is more about continuous integration. We have Jenkins setup to build and do deployments. I'm using a simple script that uses curl to deploy the app via the Tomcat Manager. After I had posted this question I came across some information regarding always un-deploying first so I added that to my script. That has helped. But as you said, over time, it will fail again. How long has Tomcat been around?

The production deployments will be more manual and they are clustered so having one go down isn't a big deal.
 
Tim Holloway
Saloon Keeper
Pie
Posts: 18100
51
Android Eclipse IDE Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Actually, I never had issues with PermGen until about Tomcat5. I have some hope that Tomcat7 has cleaned it up a little, but most of my servers aren't at that level yet.
 
Gregg Bolinger
GenRocket Founder
Ranch Hand
Posts: 15302
6
Chrome IntelliJ IDE Mac OS X
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Tomcat5 was notoriously bad for PermGen'ing. I've heard that 7 is supposed to manage memory quite a bit better than even 6 does. I'm just not ready to pull the trigger on 7 yet. Not enough testing with it.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic