I'm trying to automate some deployments to our test servers and I run into all sorts of different issues when using the Tomcat manager for deployment. JARs get corrupted, memory issues, etc. If I stop tomcat, copy in the WAR, and start tomcat, things work flawlessly.
Is this a known issue? Is there anything I can do to make deployments smoother without bouncing the server entirely?
Well, first and foremost there's the dreaded PermGenSpace issue. Because Tomcat isn't cleaning up the old app sufficiently well, "orphan" memory in the PermGen region (which is fairly small) causes redeployments to fail.
Then there are the work and tempfiles, which contained old cached bits of stuff that needs to be flushed out. For example, if you remove a JSP definition in the new version of the WAR, that doesn't automatically remove its compiled form in the Tomcat work directory, which is why I delete the stuff under the work, temp and logs directories when I redeploy (clearing logs keeps me from reading old logs and getting false impressions).
Finally, there are problems with hot-deployments where servlet initialization isn't re-executed. I see this mostly in ORM-based apps where the old database pool config doesn't update.
You can reduce the problems by stopping the app, deploying and restarting it, but some problems are likely to remain. It may serve for emergency purposes, but the only surefire way to get a clean deploy is to stop the server, purge the work directories and restart. Fortunately, Tomcat comes up fast, although no all webapps do. If downtime is simply unacceptable, of course, you should be running a cluster anyway, and you can update/bounce the cluster nodes one at a time unless something really major changes.
Customer surveys are for companies who didn't pay proper attention to begin with.
Thanks Tim. It is more about continuous integration. We have Jenkins setup to build and do deployments. I'm using a simple script that uses curl to deploy the app via the Tomcat Manager. After I had posted this question I came across some information regarding always un-deploying first so I added that to my script. That has helped. But as you said, over time, it will fail again. How long has Tomcat been around?
The production deployments will be more manual and they are clustered so having one go down isn't a big deal.
Tomcat5 was notoriously bad for PermGen'ing. I've heard that 7 is supposed to manage memory quite a bit better than even 6 does. I'm just not ready to pull the trigger on 7 yet. Not enough testing with it.