aspose file tools*
The moose likes Performance and the fly likes VM thread and high CPU Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Performance
Bookmark "VM thread and high CPU" Watch "VM thread and high CPU" New topic
Author

VM thread and high CPU

Alex Armenteros
Ranch Hand

Joined: May 05, 2010
Posts: 73
I've got an application that works very relaxed most of the time having short (seconds) peaks of 10-20% CPU but... after working for about 5-6 hours its CPU usage starts to grow without control and if left alone the machine goes crazy and no other process work.

I've added a top -cH -n 1 -b -p PID >> topFile.out and a kill-3 pid to try to search the culprit and i've got this...

Other threads are at 0% CPU

top - 14:50:02 up 50 days, 15:24, 4 users, load average: 0.79, 0.70, 0.39
Tasks: 40 total, 0 running, 40 sleeping, 0 stopped, 0 zombie
Cpu(s): 4.0%us, 0.3%sy, 0.0%ni, 94.8%id, 0.2%wa, 0.0%hi, 0.6%si, 0.0%st
Mem: 8175332k total, 7552648k used, 622684k free, 468232k buffers
Swap: 4194264k total, 96k used, 4194168k free, 5738040k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
7219 root 16 0 918m 518m 9640 S 45.9 6.5 8:56.46 java -Xmx512M -jar
7235 root 15 0 918m 518m 9640 S 2.0 6.5 0:41.07 java -Xmx512M -jar
7209 root 18 0 918m 518m 9640 S 0.0 6.5 0:00.00 java -Xmx512M -jar

As you see no swap memory is used.

And kill -3 shows that the 45,9% CPU usage thread is this one (7219->1c33)

"VM Thread" prio=10 tid=0x0000000050ae0000 nid=0x1c33 runnable

Heap summary at the end of the stack


Heap
PSYoungGen total 26688K, used 21823K [0x00002aaac8980000, 0x00002aaaca530000, 0x00002aaad3420000)
eden space 25024K, 87% used [0x00002aaac8980000,0x00002aaac9ecfd08,0x00002aaaca1f0000)
from space 1664K, 0% used [0x00002aaaca390000,0x00002aaaca390000,0x00002aaaca530000)
to space 1664K, 0% used [0x00002aaaca1f0000,0x00002aaaca1f0000,0x00002aaaca390000)
PSOldGen total 349568K, used 349566K [0x00002aaab3420000, 0x00002aaac8980000, 0x00002aaac8980000)
object space 349568K, 99% used [0x00002aaab3420000,0x00002aaac897f918,0x00002aaac8980000)
PSPermGen total 31616K, used 31318K [0x00002aaaae020000, 0x00002aaaaff00000, 0x00002aaab3420000)
object space 31616K, 99% used [0x00002aaaae020000,0x00002aaaafeb5b30,0x00002aaaaff00000)

I've tried DaCapo benchmarks with all sorts of options with no noticeable results.

Tomorrow I'll try with -Xmx set to 2g. If you see something strange please tell me

Jayesh A Lalwani
Bartender

Joined: Jan 17, 2008
Posts: 2434
    
  28

You are running out of memory. VM thread idoes Garbage collection among other things. PSOldGen is at 99% which means practically it's full. The GC is spinning continuously and is using up CPU

Most probably you have a memory leak somewhere. Maybe references to some objects are not bening cleared. Get a heapdump when the problem happens again and look at it using a memory analysis tool. You should be able to find the problem pretty easily
Deepak Bala
Bartender

Joined: Feb 24, 2006
Posts: 6662
    
    5

Your permgen space is also at 99% capacity ? Are you sure the application does not spew out OutofMemoryErrors ? Perm gen is used to load class definitions among other permanent data.

What does the GC stat for the VM look like ? When the CPU is at 100%, which thread takes up more CPU time ?


SCJP 6 articles - SCJP 5/6 mock exams - More SCJP Mocks
Alex Armenteros
Ranch Hand

Joined: May 05, 2010
Posts: 73
That VM thread is the culprit always.

I'm searchig for Hibernate Memory Leak Issues to see if I got some clues.

But I got no info from my colleage today so i seems making the heap 4 times bigger worked fine
Deepak Bala
Bartender

Joined: Feb 24, 2006
Posts: 6662
    
    5

But I got no info from my colleage today so i seems making the heap 4 times bigger worked fine


For now

If it is a leak, you will eventually run out of memory. Most app servers allow you to monitor such parameters. Does your app server have such provisions ?
Vivek Punekar
Greenhorn

Joined: Aug 25, 2009
Posts: 12
Hi All,

Interesting and very relevant thread for the issue I am looking into.
In my case - there are two threads which are hogging CPU - The VM Thread and ConcurrentMarkSweep thread.
I can understand that ConcurrentMarkSweep thread is busy as less space is remaining in tenured gen. But, Jayesh Lalwani has commented that VM thread does GC.
I don't quite agree as in my case the GC thread is clearly labeled with name ConcurrentMarkSweep.

Now the query is - what could this VM thread be doing and how to find it?
Philippe Le Marchand
Greenhorn

Joined: Jan 22, 2013
Posts: 1
Just a post to give interesting point we found out facing the same issue:

If your PermGen is 99% you should maybe increase it to 128Mb for instance, as it takes 64Mb by default.
Because if you're running out of PermGen, it is possible (in a 64 bit Unix like server environment in our case) that the JVM starts using the swap instead of throwing PermGen error.
In this case, when your swap starts being full, the GC thread will use 100% of a CPU by constantly switching to the swap, making all I/O incredibly slow (like up to an hour to write a normal-sized file).

Hope it helps ;-)
Chris Hurst
Ranch Hand

Joined: Oct 26, 2003
Posts: 418
    
    2

If you are filling Perm Gen repeatedly you will be doing Full GC's.

Have you tried logging your GC and graphing the results with something like GC Viewer.


"Eagles may soar but weasels don't get sucked into jet engines" SCJP 1.6, SCWCD 1.4, SCJD 1.5,SCBCD 5
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: VM thread and high CPU