GeeCON Prague 2014*
The moose likes Tomcat and the fly likes Problem with JAVA_OPTS in Tomcat 5 on Linux Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


JavaRanch » Java Forums » Products » Tomcat
Bookmark "Problem with JAVA_OPTS in Tomcat 5 on Linux" Watch "Problem with JAVA_OPTS in Tomcat 5 on Linux" New topic
Author

Problem with JAVA_OPTS in Tomcat 5 on Linux

Jeff Heidman
Greenhorn

Joined: Nov 09, 2012
Posts: 4
I am hoping someone can help me on this one, it has me well and truly stumped.

So we have a web application that does some JAVA_OPTS config in the tomcat5.conf file. It originally looked like this, and worked just fine:


JAVA_OPTS="-server -Xms768M -Xmx1280M -Xloggc:/opt/prod/site/logs/gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:MaxPermSize=256M -XX:+UseParallelGC -XX:+UseParallelOldGC -Dfile.encoding=ISO-8859-1 -Djava.awt.headless=true -Dsun.rmi.dgc.server.gcInterval=3600000 -Dsun.rmi.dgc.client.gcInterval=3600000 -Djava.protocol.handler.pkgs=com.sun.net.ssl.internal.www.protocol"

OK, so we found a bug that required us to switch to a different TransformerFactory. So we had instruction to add the following to our JAVA_OPTS command line:

-Djavax.xml.transform.TransformerFactory=org.apache.xalan.processor.TransformerFactoryImpl

OK, so we add it to get this:


JAVA_OPTS="-server -Xms768M -Xmx1280M -Xloggc:/opt/prod/site/logs/gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:MaxPermSize=256M -XX:+UseParallelGC -XX:+UseParallelOldGC -Dfile.encoding=ISO-8859-1 -Djava.awt.headless=true -Dsun.rmi.dgc.server.gcInterval=3600000 -Dsun.rmi.dgc.client.gcInterval=3600000 -Djava.protocol.handler.pkgs=com.sun.net.ssl.internal.www.protocol -Djavax.xml.transform.TransformerFactory=org.apache.xalan.processor.TransformerFactoryImpl"

Sadly, this doesn't work for some reason. The problem still persists, and it doesn't appear that the addition "took".

(Side question: Is there some way to tell whether this addition gets loaded other than trying to reproduce the bug it is intended to address,, which is a pain in the ass? IE, is there some way to tell when something like this gets loaded?)

So someone suggests that we do it like this:


JAVA_OPTS="-server -Xms768M -Xmx1280M -Xloggc:/opt/aac2/site/logs/gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:MaxPermSize=256M -XX:+UseParallelGC -XX:+UseParallelOldGC -Dfile.encoding=ISO-8859-1 -Djava.awt.headless=true -Dsun.rmi.dgc.server.gcInterval=3600000 -Dsun.rmi.dgc.client.gcInterval=3600000 -Djava.protocol.handler.pkgs=com.sun.net.ssl.internal.www.protocol " -Djavax.xml.transform.TransformerFactory=org.apache.xalan.processor.TransformerFactoryImpl

Now that seems pretty weird to me, but ok - we give that a shot and hey! It works! There is much rejoicing.

Until we move it to production and put a load on the system, which promptly runs out of memory and crashed. Apparently moving it outside the quotation marks allowed the class to load, but for some reason munged the instructions, and now the JVM is not getting the memory it needs.

I am trying to understand this. I don't understand why it didn't work inside the quotation marks, and I don't understand how moving the command line outside allowed it to load, but somehow messed up the rest of it.

Anyone seen something like this before?

Jeff Heidman
Greenhorn

Joined: Nov 09, 2012
Posts: 4
Someone mentioned at work that for some reason the Transformer definition needs to come first in the -D list, so I changed this to look like this instead:

JAVA_OPTS="-server -Xms768M -Xmx1280M -Xloggc:/opt/prod/site/logs/gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:MaxPermSize=256M -XX:+UseParallelGC -XX:+UseParallelOldGC -Djavax.xml.transform.TransformerFactory=org.apache.xalan.processor.TransformerFactoryImpl -Dfile.encoding=ISO-8859-1 -Djava.awt.headless=true -Dsun.rmi.dgc.server.gcInterval=3600000 -Dsun.rmi.dgc.client.gcInterval=3600000 -Djava.protocol.handler.pkgs=com.sun.net.ssl.internal.www.protocol"

but that made no discernible difference.
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 16092
    
  21

I don't know that this "tomcat5.conf" file is, but the recommended place to define your JAVA_OPTS is in the TOMCAT_HOME/bin/setenv.sh file. You may need to create this file, since it's not supplied with Tomcat as distributed.

Modifying the catalina.sh or related files is not recommended.

Also, if you're going to put this much effort into it, I'd recommend moving up to at least Tomcat 6, since Tomcat 7 is the leading version right now and Tomcat 5 will probably hit end-of-life soon. It's already not the best release to get support on.


Customer surveys are for companies who didn't pay proper attention to begin with.
Jeff Heidman
Greenhorn

Joined: Nov 09, 2012
Posts: 4
Thanks for the response Tim.

Unfortunately, upgrading the tomcat version is simply not an option for the customer at this time.

And moving the JAVA_OPTS is probably not really feasible either - that is the properties architecture that the product team uses, and has used for some time without any problems, so while it is possible to make that change, it would take...a long time.

I just want to understand why the blasted thing won't pick it up when it is inside the quotes like that.
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 16092
    
  21

Being outside the mainstream isn't fatal, but it can be expensive. While I definitely don't recommend a panic upgrade on Tomcat, the old saying that "If it ain't broke, don't fix it" does not apply to information technology. The longer they wait, the more it's going to cost, if only because the number of people who understand what's going in Tomcat5 diminishes and if you don't understand what's happening on the old platform, you can't ensure that it still happens correctly on a newer one. Even in places like this forum, it has been getting harder and harder to get Tomcat5 help over the last 2 years.

Likewise, I promote standard configurations where possible, because the more deviations, the more chances that something will break. And considering how transient employment is these days, it's all too possible that the people who understand the customizations - and remember to apply them in cases such as server upgrades - won't be there at a critical time. The "interchangeable employee" conceit doesn't allow for domain knowledge, alas.

The main reason I brought up the setenv.sh thing was, in fact because one of the most common causes for 3-day head-scratching sessions for a 5-minute problem is that a new system is set up or an old one is upgraded and the extra localized tweaking doesn't get done properly (if at all).

Looking more closely at what your problem is about, however, I think that regardless of the above, you have a more fundamental problem.

Tomcat does not use xalan. Therefore you shouldn't be attempting to configure it at the JVM level. This should be done within whatever application(s) are using xalan.

Most likely, in fact, you're setting the JVM option and the webapps are overriding it on a per-application basis.
Jeff Heidman
Greenhorn

Joined: Nov 09, 2012
Posts: 4
Tim, I could not agree with you more - I am confident that once we figure this out, we will be kicking ourselves over the ridiculousness of it. It is just that kind of problem.

Unfortunately, our ability to control some of these things is very limited - our customer doesn't want to upgrade to our latest version, and hence we are stuck with Tomcat5. And the upgrade process for our application is pretty lengthy in any case.

When you say that xalan is not part of Tomcat, I am not sure what you mean by that (other than the obvious of course). My understanding of this -d directive is that it is basically telling the JVM "Hey, use this transformer instead of the default one you normally use", right? I certainly understand your point about how we are configuring this, but again, that is not something I have any control

How else would we go about accomplishing this other than a JAVA-OPTS directive?

Thanks again for all your help - it is very greatly appreciated. This problem has caused me no end of grief.
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 16092
    
  21

The "-D" option on the JVM command line defines a name/value property pair accessible via the System properties interface, Of all the places to set options in a webapp server, this is the LEAST desirable. In fact, it can be outright hazardous, since each webapp has a unique classpath and you can find yourself with missing/conflicting resource definitions at inconvenient times. A far better place to do this is in the META-INF/services/ of the WAR(s) of the application(s) that require XML transformation services. Or in the Context definition for the app, but that one requires application logic support.

Of course, depending on how antiquated the JVM and apps are, that particular feature may or may not work, just as the -D command-line option may not be picked up by the webapp.

 
GeeCON Prague 2014
 
subject: Problem with JAVA_OPTS in Tomcat 5 on Linux