Win a copy of Think Java: How to Think Like a Computer Scientist this week in the Java in General forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

JBoss 4.0.4 is slow down in every 20 days

 
Sky Loi
Ranch Hand
Posts: 65
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi, friends,

I have production env with JBoss 4.0.4 and was add ehcache in last release. BUt it was slow down and then shutdown due to the java heap out of memory in every 20 days after that and need reboot the jboss. So far, I have not found the root cause, just got some symptons: java heap usage is overflow, e.g. over 512 max to 6xx sometime, seems run out memory. But it is ok in the 19 days when I monitor the memory usage, usually have about 120 - 150M free memory left. Do you have any idea or experience on my case?

The specs:
JBoss 4.04
Java 1.5.0.07
server physical mem: 1G


Thanks
 
Peter Johnson
author
Bartender
Posts: 5852
7
Android Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The best thing to do is add -XX:+HeapDumpOnOutOfMemoryError to the JAVA_OPTS in the run script. Then when you get the OOME, the JVM will dump the heap contents to a file. You can open the file using a tool such as VisualVM. That might tell you what is using up the heap.

You should also monitor GC statistics. You can usually tell, by monitoring the heap usage after a major collection, whether the app has a leak - the tend line will have a positive slope.
For suggestions on how to gather GC data and analyze it, see these white papers:
Java Garbage Collection Statistical Analysis 101
Java Garbage Collection Performance Analysis 201
at http://www.cmg.org/cgi-bin/search.cgi?q=java+peter+johnson&x=30&y=10
 
Sky Loi
Ranch Hand
Posts: 65
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi, Peter,

Thanks for your great advice and direction. I will try this way to find the root cause.
 
Sky Loi
Ranch Hand
Posts: 65
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I just got the heap memory issue. The rest heap free size was just less than 1M (pre-allocated 512M). I got the thread dump at that time. The attachment is the thread dump logs. I did not see much exception. One thing I remembered I add new ehcache/jgroup components in the JBoss which seems has some conflict with the built-in jboss clustering components (the old jgroup libs). I am not sure it is the clue I could start with.

 
Sky Loi
Ranch Hand
Posts: 65
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
sorry, seems I get the thread dump instead of heap dump. I will check the heap dump way first.
 
Sky Loi
Ranch Hand
Posts: 65
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I used JProfiler to monitor the heap in JBoss server and found some suspicious java.lang.Object instances are increased gradually but cannot see what type exactly they are. I really assume it is the cause leading to 20-day heap memory leaking. Do we have any better idea how to get more into it? or any other way or tool?

Thanks
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic