Get your CodeRanch badge!*
The moose likes Performance and the fly likes Application has all memory params set, Still Consumed Memory is way too high Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Java 8 in Action this week in the Java 8 forum!
JavaRanch » Java Forums » Java » Performance
Bookmark "Application has all memory params set, Still Consumed Memory is way too high" Watch "Application has all memory params set, Still Consumed Memory is way too high" New topic
Author

Application has all memory params set, Still Consumed Memory is way too high

Raaghu Kannan
Greenhorn

Joined: Aug 19, 2011
Posts: 8
Hi,

The basic problem is the java process is consuming way to higher memory, in spite of setting the memory parameters.
Following are the memory parameters which I am setting to launch the application

-DLOG_TO_STREAM=true -DLM_VLPR_SIZE=10 -Xms16m -Xmx64m -Xss128k -XX:PermSize=16m -XX:MaxPermSize=64m


Top command snapshot is below



I run the application on a 64 bit machine/java and I have around 12GB RAM

One more interesting observation, lets say the application is continuously doing some operation and memory consumption is around 3%. It will continue to consume 3% even when it is not processing.

This is a simple jms application in which the consumer keeps adding object to arraylist, when the arraylist size is 10 it produces that message to a destination. There is one thread running in parallel which checks if the arraylist has objects and if they are not sent out since 1 sec then it will call the same method which produces that message to a destination

In general once the arraylist is sent out I make the arraylist null / clear the collection and create a new arraylist object. Can arraylist cause such issue??



Appreciate any quick response.

Thanks and Regards
Raaghu.K
Jeff Verdegan
Bartender

Joined: Jan 03, 2004
Posts: 6109
    
    6

The -Xmx 64m that you set is the max heap that will be allocated for your app. Separately from that, the JVM itself needs memory, and I wouldn't be surprised if it needs more on a 64-bit system.

It will continue to consume 3% even when it is not processing.


That's not surprising. The JVM doesn't give memory back to the OS just because it does a GC. I think there are some circumstances where it does give a certain amount back, but you're best off just assuming that it never will.
Raaghu Kannan
Greenhorn

Joined: Aug 19, 2011
Posts: 8
Thanks for your response.

Is there any specific reason related to arraylist ??

Is there any other way where I strictly confine the process not to consumer more than 'certain' memory threshold

May be an OS parameter or anything at that level??

Regards
Raaghu.K
Winston Gutkowski
Bartender

Joined: Mar 17, 2011
Posts: 7081
    
  16

Raaghu Kannan wrote:Is there any specific reason related to arraylist ??

Not that I know of. The class has been around for a long time.
Is there any other way where I strictly confine the process not to consumer more than 'certain' memory threshold

Not, I suspect, without risking an OutOfMemoryError.

One thing I do notice is that your class will never exit unless interrupted. Is it possible that your creating a whole bunch of them and they're simply "hanging around"?
If it was me, I think I'd provide a mechanism to allow the class to terminate gracefully (eg, make it a java.io.Closeable, and add a close() method).

Winston


Isn't it funny how there's always time and money enough to do it WRONG?
Artlicles by Winston can be found here
Raaghu Kannan
Greenhorn

Joined: Aug 19, 2011
Posts: 8

Thanks for responding...

Not, I suspect, without risking an OutOfMemoryError.


In fact When I did profiling I saw the application is using 20 mb of heap and 12 mb of non-heap memory. I want to see out of memory

One thing I do notice is that your class will never exit unless interrupted. Is it possible that your creating a whole bunch of them and they're simply "hanging around"?
If it was me, I think I'd provide a mechanism to allow the class to terminate gracefully (eg, make it a java.io.Closeable, and add a close() method).


No I initiate this class only once at the start and leave it ... I want that while(true) to be running

Any other thoughts?

Regards
Raaghu.K
Winston Gutkowski
Bartender

Joined: Mar 17, 2011
Posts: 7081
    
  16

Raaghu Kannan wrote:In fact When I did profiling I saw the application is using 20 mb of heap and 12 mb of non-heap memory. I want to see out of memory

Fair enough. Those figures don't seem particularly high to me. Have you checked them against the figures when the class is simply loaded (ie, before it's started doing anything)?

No I initiate this class only once at the start and leave it ... I want that while(true) to be running
Any other thoughts?

Not at present. I would still provide some way to terminate it though. All it would take is something like:Just remember to close() it before you exit.

Winston
Raaghu Kannan
Greenhorn

Joined: Aug 19, 2011
Posts: 8
Have you checked them against the figures when the class is simply loaded (ie, before it's started doing anything)?


Yes, the memory consumption before or during any 'processing' remained almost same.

I like the idea of implementing closeable interface as it looks neat, I will put it. But we do not have soft exit , the application has to be killed when it is not required.

I am just reiterating the question, is there any possible way where we can put a cap on total memory consumption for a java process.

Thanks and Regards
Raaghu.K

Winston Gutkowski
Bartender

Joined: Mar 17, 2011
Posts: 7081
    
  16

Raaghu Kannan wrote:I am just reiterating the question, is there any possible way where we can put a cap on total memory consumption for a java process.

Yes, and probably several; particularly if your OS supports quotas, as most *n[iu]x'es do.

However, the fact is that your process seems to be initializing to that size. Again, there are probably java parameters that can restrict that too; but I've never tried it (I'm sure someone else has though). The JVM file size is around 10 meg though as I recall, so there may well be a lower limit. And as I say, your figures don't seem particularly terrible.
Also, Java object overheads are higher in a 64-bit JVM than they are in a 32-bit one.

But we do not have soft exit, the application has to be killed when it is not required.

Ugh. Couldn't you just provide a termination mechanism for it - eg, touching some sort of flag file?

Winston
Deepak Bala
Bartender

Joined: Feb 24, 2006
Posts: 6657
    
    5

I am just reiterating the question, is there any possible way where we can put a cap on total memory consumption for a java process.


The Xmx flag is the cap. A java process that tries to consume more memory will fail with an OutOfMemoryError. The native memory consumed through a JNI process is another problem entirely. I assume your scenario does not run into that possibility.


SCJP 6 articles - SCJP 5/6 mock exams - More SCJP Mocks
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Application has all memory params set, Still Consumed Memory is way too high
 
Similar Threads
Glassfish: How do I set endpointExceptionRedeliveryAttempts for a standalone JMS client?
read message from Topic
MDB deployment on JBoss
Why this compiles in Sun App Server but not JBOSS
ActiveMQ topic