OutOfMemory Error with MySQL 5 and / or Tomcat

[James Dekker]

Have a web app running inside Tomcat 5.5.28 with JDK 1.6.0_20 64 bit server on an Apple computer with a 3.06 GHz Intel Core 2 Duo processor and 8 GB of RAM.

My web app is connected to MySQL 5.4.3-beta.

For some odd reason, when I am doing certain things (as an end user) to my web app, I get the following output in the console:

Caused by: java.lang.OutOfMemoryError: Java heap space
	at com.mysql.jdbc.Buffer.getBytes(Buffer.java:198)
	at com.mysql.jdbc.Buffer.readLenByteArray(Buffer.java:318)
	at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1366)
	at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2333)
	at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:435)

Have already increased the heap via my CATALINA_OPTS inside my $CATALINA_HOME/bin/catalina.sh:

export CATALINA_OPTS="-Xms256m -Xmx1024m -XX:MaxPermSize=512m";

However, this OutOfMemoryError keeps recurring (and it has MySQL in its StackTrace).

* Is there a way to increase the Heap Size for MySQL?

* Am I setting the CATALINA_OPTS the correct way?

Thank you for taking the time to read this.

[SumitPal Pal]

Since you do not know - if the CATALINA_OPTS is indeed setting the max memory - here is a technique to find it out.


get the PID ( processID ) of the JVM running on your APPLE machine

Then run this command from command prompt

jinfo PID

This will show you the parameters passed to the JVM and here you can see if the CATALINA_OPTS is indeed passing the right max heap size to the JVM process.

If it is set to 1024M and you are still getting OutOfMemory - please make sure that your program is indeed not consuming that much memory

[James Dekker]

Thanks for responding, Sumit!

Here's what happened when I followed your advice:

jinfo 1571
Attaching to process ID 1571, please wait...
sun.jvm.hotspot.debugger.NoSuchSymbolException: Could not find symbol "gHotSpotVMTypeEntryTypeNameOffset" in any of the known library names (-)
at sun.jvm.hotspot.HotSpotTypeDataBase.lookupInProcess(HotSpotTypeDataBase.java:397)
at sun.jvm.hotspot.HotSpotTypeDataBase.getLongValueFromProcess(HotSpotTypeDataBase.java:378)
at sun.jvm.hotspot.HotSpotTypeDataBase.readVMTypes(HotSpotTypeDataBase.java:107)
at sun.jvm.hotspot.HotSpotTypeDataBase.<init>(HotSpotTypeDataBase.java:85)
at sun.jvm.hotspot.MacOSXTypeDataBase.<init>(MacOSXTypeDataBase.java:36)
at sun.jvm.hotspot.bugspot.BugSpotAgent.setupVM(BugSpotAgent.java:578)
at sun.jvm.hotspot.bugspot.BugSpotAgent.go(BugSpotAgent.java:499)
at sun.jvm.hotspot.bugspot.BugSpotAgent.attach(BugSpotAgent.java:337)
at sun.jvm.hotspot.tools.Tool.start(Tool.java:163)
at sun.jvm.hotspot.tools.JInfo.main(JInfo.java:128)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.tools.jinfo.JInfo.runTool(JInfo.java:79)
at sun.tools.jinfo.JInfo.main(JInfo.java:53)
Debugger attached successfully.
jinfo requires a java VM process/core!

Found the process # by issuing this from the command line:

ps -a | grep tomcat

Any other suggestions?

[SumitPal Pal]

try
ps -aedf | grep java

and you will see the parameters right there

[James Dekker]

Sumit,

Here you go:

ps -aedf | grep java

  501  1581     1   0   0:44.68 ttys000    2:08.33 /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home/bin/java
-Djava.util.logging.config.file=/Users/james/DevTools/Java/tomcat/apache-tomcat-5.5.28/conf/logging.properties 
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xms256m -Xmx1024m -XX:MaxPermSize=512m 
-Djava.endorsed.dirs=/Users/james/DevTools/Java/tomcat/apache-tomcat-5.5.28/common/endorsed -classpath /Users/james/DevTools/Java/tomcat/apache-tomcat-5.5.28/bin/bootstrap.jar 
-Dcatalina.base=/Users/james/DevTools/Java/tomcat/apache-tomcat-5.5.28 
-Dcatalina.home=/Users/james/DevTools/Java/tomcat/apache-tomcat-5.5.28 
-Djava.io.tmpdir=/Users/james/DevTools/Java/tomcat/apache-tomcat-5.5.28/temp org.apache.catalina.startup.Bootstrap start

[SumitPal Pal]

ok so the JVM does have 1GB of RAM.
Can You send the stack trace when you get this OutOfMemory error

[SumitPal Pal]

OK I do see the stack trace from your previous post.

What JDBC driver are you using to connect to MySQL.

# at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1366)
# at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2333)
# at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:435)

Can you post your code where you are reading the result.
What is the size of each row on an average when you are reading the data.
Is there some data in each row which when read into memory can exceed 1 GB of RAM.

[James Dekker]

Sumit,

I did sent it (see original posting)... What do you mean?

Thanks for all of your help...

[Jaikiran Pai]

Looks like some piece of code within your application is firing a query which loads a lot of data into memory. You'll have to narrow down the code which does that. If you have the entire exception stacktrace, then it will show you the relevant class and method which is firing the query.

[SumitPal Pal]

yes the gentleman in the last post mentions it clearly - there is some code that is loading too much object from the DB into memory and hence you are running out of memory.

2 ways to solve it

1. Allocate more memory to your java program ( 1GB is not sufficient it seems )
2. Change the logic in your program so that it does not read too much data