I wonder if anyone can give me a few pointers about something. I am supporting a servlet based application. I did not write it myself but do have access to the development team who did.
When the application is under reasonable load the JVM does an awful lot of minor GC - so much so that after a period of time the CPU can spend more time GC'ing than servicing requests. The following is a snippet from the app server's (JBoss AS 4.2.2 GA) stdout:
The way I'm reading this is that the system is doing a minor GC 3 times a second! and each minor GC is freeing up about 110MB. To come to this conslusion I've intepreted each line as:
[PSYoungGen: 112112K->2256K(113088K)] which means the young generation heap started at 112112K and ended at 2256K which means it released 109856K; and that the 113088K in brackets is the total heap size of the young generation. Hopefully thats the right way to interpret these logs.
Anyway, clearly its doing a lot of minor GC! I know I can tune the JVM and the way GC operates - heap sizes, ratios etc etc - but before I get into that I want to try to understand why all this memory becomes available for GC so frequently!
To produce the above I've artifically applied load with jmeter. My jmeter test is a 10 thread test running for 5 minutes. Each thread submits a single HTTP request to the servlet. The HTTP session timeout is 15 seconds. The above GC stats were produced with this jmeter test. The jmeter test ran for 5 minutes and in that time issued 5684 requests at an average throughput of 18.5 requests per second (I'm going to round that up to 20). I have taken a heap dump of the system under load and the development team can see each of the sessions and tell me that each session is using about 250K. So ...
In 1 second we make 20 requests. Each request results in a session of 250K. Each session lives for 15 seconds. Therefore session memory usage is 20 X 250K X 15 = 75000K
So, in any 15 second period, I can account for 75MB of memory.
I cant account for 110MB every second!!!
Where / how should I look to account for this massive amount of memory usage?
Any pointers would be very much appreciated;
Joined: Aug 18, 2004
In support of my posting, I've run the gc logs through GCViewer (http://www.tagtraum.com/gcviewer.html); and have attached the screenshot - might mean something to someone, but to my untrained eye 5.792GB/sec of garbage collection is going some!!!
Nathan Russell wrote:In support of my posting, I've run the gc logs through GCViewer (http://www.tagtraum.com/gcviewer.html); and have attached the screenshot - might mean something to someone, but to my untrained eye 5.792GB/sec of garbage collection is going some!!!
At the risk of stating the obvious, is the servlet creating a lot of short lived object instances while processing a request? Simple memory profiling will probably give you the answers you need.