File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Performance and the fly likes Need Help Understanding OutofMemory Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Performance
Bookmark "Need Help Understanding OutofMemory" Watch "Need Help Understanding OutofMemory" New topic
Author

Need Help Understanding OutofMemory

William Martinez
Greenhorn

Joined: Jan 25, 2008
Posts: 5
Hello.

We are facing an OOM in production, that shows intermittently, with any load and in any function of the application.

I know that sounds not new to all. Anyway, We need advice on how to solve the problem. So, here is a list of questions.

1. Using a HeapDump and several analysis tools, we were not able to find any memory leak. We have enough memory and a -Xms512m -Xmx784m setting. Nevertheless, the dump file is just 115mb or less. Can we compare that? Is that size a clue we can use to determine the problem is somewhere else and not actually that the memory is being totally consumed?

2. Now, if the above holds true, and there is actually memory unused (wonder how can we totally be sure of that), how come the JVM throws the exception?

3. I read something about the different memory "sections" to hold object of different ages (GC generational model). Is if possible some of those sections are not growing although there is memory to use? How can we fine tune those settings, if there are settings for that?

4. Further more, we are working in this config:
Jboss 4.0.1sp1
JVM 1.4.2_12
Operating System W2K3 SP2
Could it be and older JMV-New O.S conflict problem?

The actual error is:
Unexpected Error; nested exception is:
java.lang.OutOfMemoryError
java.rmi.ServerError: Unexpected Error; nested exception is:
java.lang.OutOfMemoryError

5. Unexpected Errors. Should this mean an error captured at JVM? That is, the exception is not caught during normal execution like creating an object or going through a loop. Our guess it is thrown when actually executing the GC. How can we sure of that? If that is correct, does it mean is it a JVM problem?

Thanks for your help.
William.
Jim Yingst
Wanderer
Sheriff

Joined: Jan 30, 2000
Posts: 18671
For most of these, I don't know, really. But here are a couple possible answers:

2. It's possible for the JVM to throw this error even though there's a lot of unused memory if there was a particular line that required much more memory than was available. E.g.

This line might fail even though there's more than enough memory for everything else the application is doing. Of course something big like this is usually pretty obvious, so your problem is probably something else.

5. "Unexpected Error" was apparently something in the RMI code. It looks like the OOME occurred on the server, but you're looking at it from an RMI client. The RMI server caught the OOME, wrapped it with a ServerError, and serialized it to the RMI client. To know more about exactly what was happening - is there no stack trace? Sometimes with an OOME there is none, and it will say "stack trace unavailable". But otherwise, find the stack trace. If you don't see "unavailable" then probably some code is printing the error but omitting the trace, which is usually a mistake. The trace can help you answer questions about what's going on at the time of the error.

If the stack trace is unavailable, you can try putting a lot of logging statements in to the server code (possibly using AOP). The you can look at the server logs and see what the server was doing just prior to the OOME.


"I'm not back." - Bill Harding, Twister
anupa oru
Ranch Hand

Joined: Jan 15, 2005
Posts: 118
We had similar type of situation where memory was 50% use but ran into outofmemory. It turned out to be when JVM allocate memory to objects, short lived objects will go into YoungGeneration, long lived ones goes into oldGeneration. Even though memory was at 50% usage, we ran out of young generation. The way I have determined that was by dumping GC logging, as soon as we get OutOfMemory error we looked at gc.log and saw that we were at 100% of young generation(default is 64MB). You can have below setting run.bat if it's windows, run.sh if it's linux.

-verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -XX:+PrintTenuringDistribution -Xloggc:gc.log

This will create gc.log that would display that's being used at any given time, as soon as you get outofmemory error analyze the gc.log and see if you are running out of space in one of the generations.
useful links:
http://wiki.jboss.org/wiki/Wiki.jsp?page=TuneVMGarbageCollection

Thanks,
Anu
William Martinez
Greenhorn

Joined: Jan 25, 2008
Posts: 5
Thanks a lot for both answers.
We will try out your suggestions. Of course I cannot tell if they worked until the next OOM, which we cannot predict when will it occur.
Thanks!

William.
Raees Uzhunnan
Ranch Hand

Joined: Aug 15, 2002
Posts: 126
Looks like a jvm memory leak. Can you check the memory growth outside of jvm using tools like prstat on solaris ; top on linux or hp ?.

Thanks
Raees


Sun Certified Enterprise Architect
Java Technology Blog
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Need Help Understanding OutofMemory