I am implementing container-managed authentication for Tomcat 6. I am using a JDBCRealm, and so needed to put a mysql driver in <catalina home>\lib. This mysql-connector jar is version 5. However, one of my web apps is stuck using an older version of mysql-connector, which is in WEB-INF\lib for that web-app.
The problem I'm seeing now is that occasionally, Tomcat seems to choose one version of the mysql jar over the other, causing this web-app to break when it tries using the newer mysql jar (there are a variety of incompatibilities with this newer version that cannot be fixed at this time), OR the custom Authenticator will break since it needs the latest mysql jar.
I tried assigning the old server.loader property in Tomcat's catalina.properties file so that the mysql jar dependency for my JDBCRealm is isolated, Tomcat gave errors about not being able to load the Mysql classes. I am not sure, but it looks like the bootstrap loader is what loads the web-apps and their Realms (rather than the server loader), and that needs the mysql jar, which as far as I can tell, requires that everything be in <catalina home>\lib.
Is there any way to load the two different versions of the jar files and keep them separate?
It's very bad practice to include driver jars in webapps, and this is one reason why. What you are seeing is the kind of stuff that happens when some classes are being loaded from one location and some are loaded from another, resulting in an unholy mishmash where two or more discrete versions of a single class are in play, and which one a given function will end up with is more a game of roulette than anything else.
Unlike a certain other platform I could mention, both Java and the open-source databases take a lot of care in maintaining version inter-compatibility, although I can mention one notable offender at the binary (non-JDBC) client level. It's not MySQL, however, so that's not your problem. So in most cases, the problem would be cured by simply removing the MySQL driver from the WAR entirely.
From what you're saying, however, I understand that the newer MySQL driver does not function properly for this particular webapp. Perhaps, in part since it apparently wasn't designed according to best practices to begin with. Tomcat's class-loading rules are very explicit, so in normal cases, the WAR driver classes should take precedence within the webapp, regardelss of what other apps or the Realm are doing. However, if the webapp itself isn't well-designed, it may be making reference to resources outside of its expected boundaries.
Bottom line. Although people like to say "if it ain't broke, don't fix it", this is a naive thing to say in IT, where things get far more often broken from the outside in than vice versa. It's probably about time for that 150,000 mile overhaul on the offending webapp.
Customer surveys are for companies who didn't pay proper attention to begin with.