aspose file tools*
The moose likes Performance and the fly likes Memory usage in JVM Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Performance
Bookmark "Memory usage in JVM" Watch "Memory usage in JVM" New topic
Author

Memory usage in JVM

Michal Glowacki
Ranch Hand

Joined: Mar 14, 2006
Posts: 114
hi,

not sure if I've chosen right forum. I have some web applications running on JBoss AS. Now I have connected to it's process with YourKit. When I go to memory, I see:

Class name: char[]
number of objects: 10,000,000
Shallow size 1,400,000,000 (43%)

And minior collection every 1 minute reducing this values to 2,000,000 and 400,000,000 accordingly.

I use a lot StringBuilder to generate some complex HTML code it would be difficult to make in JSP and put it as value of hutputText (with escape=false). Can it be the problem? Anyway right now minor collection happening every 1 minute but traffic is still very low. I have assigned 8gb to xmx and xms and I think it's too much (major collection takes around 8-10 seconds during heavy traffic hours every 20 mins).

Is the number of char[] classes a problem?

Regards,
Michal
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12788
    
    5
That does seem like an awful lot of char[] alright.

Exactly how are you creating a StringBuilder? According to the javadocs, the no args constructor gives only 16 characters - that initial char[] will be abandoned quickly if you are building big Strings. The String arg constructor starts with the string length + 16 - also likely to be abandoned early.

If this was my problem I would use the StringBuilder( someInt ) with a realistic estimate of the eventual length needed, thus avoiding all the intermediate char[] creation.

The automatic capacity increase just doubles the char[] size - see AbstractStringBuilder

Michal Glowacki
Ranch Hand

Joined: Mar 14, 2006
Posts: 114
I think I do it hard way, but it is difficult for me to change it as the size depends on the numbers of rows returned from database:

Pat Farrell
Rancher

Joined: Aug 11, 2007
Posts: 4658
    
    5

I'd handle this using the Google Guava collections methods and objects. With them, you dont' have to read all the rows from the DBMS into memory, you can use an Iterable and access each row inside the loop where you need the values. (You can do it without Guava, but it has so many other features that I normally start thinking of using it.) Guava is free, open source, available from Google, well supported, etc.

Its never a good idea to read in thousands of rows before you process the first one.
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12788
    
    5
If this was my problem I would use new StringBuilder( 10000 ) or maybe 20000 - more than needed probably but think of all the extra char[] that would not be created, and there would only be one char[] to be garbage collected.

A further GC consideration is that all those little char[] can leave your memory fragmented.

Just humor me, give it a try.

Bill
Martin Vajsar
Sheriff

Joined: Aug 22, 2010
Posts: 3610
    
  60

William Brogden wrote:A further GC consideration is that all those little char[] can leave your memory fragmented.

Not really. Modern garbage collectors actually compact the heap. (This makes memory allocation in Java lightning fast, no inspection of available memory linked-lists, just updates to the memory pointer. It can even make up for the performance overhead garbage collectors produce.)

If this was my problem I would use new StringBuilder( 10000 ) or maybe 20000 - more than needed probably but think of all the extra char[] that would not be created, and there would only be one char[] to be garbage collected.

I'd suggest to try to obtain a distribution of sizes your code produces in real environment. Then looking at that, you might come up with sensible initial capacity that would suffice in large number of cases.
Michal Glowacki
Ranch Hand

Joined: Mar 14, 2006
Posts: 114
I am still struggling with this problem. I have made some screenshots, when CPUs are working 50-90% and the traffic is low. JBoss process is more than 9GB at this moment. Here are my jvm settings:


set JAVA_OPTS=%JAVA_OPTS% -Xms8g -Xmx8g -XX:MaxPermSize=512m
set JAVA_OPTS=%JAVA_OPTS% -Dsun.rmi.dgc.client.gcInterval=691200000 -Dsun.rmi.dgc.server.gcInterval=691200000
set JAVA_OPTS=%JAVA_OPTS% -XX:+PrintGC -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -Xloggc:gc.log
set JAVA_OPTS=%JAVA_OPTS% -XX:LargePageSizeInBytes=5m -XX:ParallelGCThreads=20 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:ThreadStackSize=1024

I know you gave me some tips regarding StringBuilder initial capacity, however I just wanted to give you more input before I start over with major (in case of my project) changes.

Thanks,
Michal


[Thumbnail for 1.jpg]

[Thumbnail for 2.jpg]

[Thumbnail for 3.jpg]

Michal Glowacki
Ranch Hand

Joined: Mar 14, 2006
Posts: 114
and some more from memory section


[Thumbnail for 4.jpg]

[Thumbnail for 5.jpg]

[Thumbnail for 6.jpg]

Lingan Rajan
Ranch Hand

Joined: Jan 26, 2011
Posts: 30
Michal
I am not an expert with performance but here are my thouhghts.
The StringBuilder initialization should certainly help.
Apart from that the screen shots show lots of Object[] and String . Have you inspected the stack trace of those Objects ?
Also how about caching certain strings that are repeatedly used ? like the HTML elements .
Jayesh A Lalwani
Bartender

Joined: Jan 17, 2008
Posts: 2377
    
  28

I second the suggestion to set the initialCapacity on the StringBuilder. That should give you quite a boost.

Another suggestion:-
You might want to try pooling the StringBuilder so you don't create an instance for each request. This will reduce the number of char[] that the GC has to manage. Basically, the only time a char[] will be collected is when the StringBuilder is evicted from the pool, and if you tune the pool right, that should happen only when the load is low. Also, another advantage is that having a pool allows you to degrade gracefully. If the pool is maxed out, you can either make the request wait till instances are available or show a "Server busy" error message. Although either way is not the most desirable, it's much better than overallocation which is going to adversely affect all requests. Most probably you have a limit on JBoss's thread pool anyways. Depending on how much memory is used per request, you might be able to set the maxsize of StringBuilder pool = maxsize of JBoss thread pool

When you passivatethe StringBuilder the pool, call setLength(0). That should clear the array without deallocating it. setLength isn't cheap. So, there will be an overhead. You'll need to benchmark it to measure the impact.

ETA: Are you sure you can't change your implementation to build the HTML in JSP. I get the feeling that you are running into problems that JBoss has already solved. You might be reinventing the wheel a bit here.
Rojan punn
Greenhorn

Joined: Nov 29, 2011
Posts: 17

Hi,

I am not very sure about the all of the below ,

set JAVA_OPTS=%JAVA_OPTS% -Dsun.rmi.dgc.client.gcInterval=691200000 -Dsun.rmi.dgc.server.gcInterval=691200000
set JAVA_OPTS=%JAVA_OPTS% -XX:+PrintGC -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -Xloggc:gc.log
set JAVA_OPTS=%JAVA_OPTS% -XX:LargePageSizeInBytes=5m -XX:ParallelGCThreads=20 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:ThreadStackSize=1024

but try the blow java_opts
-XX:+UseParNewGC
-XX:+DisableExplicitGC

is the second JAVA_OPT is really required in your settings?

It is worth try romoving all your above java option and put the one i provided.

Are you using any loggers?
Are you updating db at any point in this process?

I think the large size of string can be a cause of the issue.Is there any way to build the string in smll chuncks?
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Memory usage in JVM