JVM memory releasing after memory consuming task

[Khan Malamir]

Hi community,
I am trying to explore the memory behaviour of an application (standalone GUI) after customer complaing of great memory usage (application works on terminal server, so several terminal clients whth big memory consumption eat up all memory).

The memory dynamic is like this: there are memory consuming tasks, so there is a need for more available memory (i.e. Xmx128M) but in general there is no need for so much memory, but at most 40-50 MB instead. However after a heavy task the memory is 'filled up' and never freed, though not needed.

The test code below simulates user task that requires memory and after that continues in 'low-memory mode'. However I found out that I need to explicitly call System.gc() /a practice that is discouraged as I have read/ in order to restore the memory state to 'normal' state after the high-requirement task was completed. Strange is that if there is no allocation of bigger resources there are no garbage collections if there is no gc() call /as the second log shows/.

The question is what are the parameters to tell the JVM to behave so that the memory allocation(total seen in OS performance meters) follows at least roughly the real required memory?
I checked a recent thread but did not find the mentioned parameters.

Code below is the test code that produces the both logs below, with and without the System.gc() call on indicated line 45.

Thanks for your attention and your tips

Mike


TEST CODE: Performance.java

public class Performance{
  private static final int MEGABYTE = 1024*1024;
  
  public static void main(String[] args){
  reportMemory();
  new Thread(new MemoryReporter()).start();
    new Thread(new UserAction()).start();
  } 

private static void reportMemory(){
    Runtime r = Runtime.getRuntime();
    System.out.println(r.totalMemory()/(double)MEGABYTE + "(" + (int)(100*r.totalMemory()/r.maxMemory()) +")");
  }

  private static void sleep(int seconds){
    try{
      Thread.sleep(seconds*1000);
    }
    catch(Exception ex){
      ex.printStackTrace();
    }
  }
  
  
private static class MemoryReporter implements Runnable{
  public void run(){
    while(true){     
      reportMemory();
        sleep(1);
      }
    }
  }

  private static class UserAction implements Runnable{
    private static final int SIZE = 2*MEGABYTE;
    
    private int[] allocation;
    
    int i = 0;
public void run(){
      while(true){       
        if(i++ < 5){
          allocation = new int[SIZE];          
        }
        System.gc();//Line 45
        sleep(1);
      }
}
}
}

~:TEST CODE

Run with explicit System.gc() on line 45
-------------------------------------------

java -verbose:gc -Xms1M -Xmx20M -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 Performance
0.9375(4)
0.9375(4)
[GC 123K->84K(960K), 0.0032387 secs]
[Full GC 84K->84K(960K), 0.0065660 secs]
[Full GC 8276K->8276K(9156K), 0.0071989 secs]
9.734375(48)
[GC 8277K->8276K(9968K), 0.0006322 secs]
[Full GC 8276K->8276K(9968K), 0.0067376 secs]
[Full GC 16468K->8276K(18164K), 0.0153724 secs]
18.30078125(92)
[Full GC 16468K->8276K(18740K), 0.0142714 secs]
17.55078125(88)
[Full GC 16468K->8276K(17972K), 0.0144490 secs]
14.94921875(75)
[GC 8276K->8276K(15308K), 0.0001583 secs]
[Full GC 8276K->8276K(15308K), 0.0071805 secs]
[Full GC 16468K->8276K(19376K), 0.0144972 secs]
10.91796875(54)
[Full GC 8276K->8276K(11180K), 0.0069838 secs]
10.91796875(54)
[Full GC 8276K->8276K(11180K), 0.0068601 secs]
10.91796875(54)
[Full GC 8276K->8276K(11180K), 0.0053297 secs]


Run without explicit System.gc() on line 45
-------------------------------------------
java -verbose:gc -Xms1M -Xmx20M -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 Performance
0.9375(4)
0.9375(4)
[GC 123K->84K(960K), 0.0034072 secs]
[Full GC 84K->84K(960K), 0.0071337 secs]
8.94140625(44)
[GC 8277K->8276K(9156K), 0.0006473 secs]
[Full GC 8276K->8276K(9156K), 0.0068225 secs]
17.73828125(89)
[GC 16469K->16468K(18164K), 0.0002155 secs]
[Full GC 16468K->8276K(18164K), 0.0143771 secs]
18.30078125(92)
[GC 16469K->16468K(18740K), 0.0002150 secs]
[Full GC 16468K->8276K(18740K), 0.0157676 secs]
17.55078125(88)
[GC 16468K->16468K(17972K), 0.0002160 secs]
[Full GC 16468K->8276K(17972K), 0.0148848 secs]
19.625(98)
19.625(98)
19.625(98)
19.625(98)
19.625(98)
19.625(98)
19.625(98)
19.625(98)
19.625(98)
19.625(98)
19.625(98)
19.625(98)
19.625(98)
19.625(98)
19.625(98)
19.625(98)
19.625(98)

[Ernest Friedman-Hill]

Hi,

Welcome to JavaRanch!

The parameters mentioned in that thread let you tell Java to allocate a larger heap, to allocate the entire heap at once, or to distribute the garbage collector's sub-heaps in a specified way. There is no parameter that will change what you're observing here: as I said, the memory allocation from the OS can grow, but can never shrink. Free memory is returned to Java's internal heap, but never to the OS.

[Khan Malamir]

Thanks for replying.
For the first example/that calls gc()/ I observed the behaviour in Task Manager in Windows, it showed that the JVM was consuming about 22MB during heavy task and then it decreased to about 14MB, so there is a way to achieve the desired behaviour - small heap free ratios can be specified a thread to be launched to call System.gc() on regular intervals, say 5 seconds.

The point I don't like about this scheme is that it will be based on System.gc(), that is just a recommendation and can even be disabled with JVM parameters.

I wanted to come up with a solution that would be based on JVM parameters only and not require additional coding within the application.

In case the JRE version is important: application is run with Sun JRE 1.4.1/2 and soon will be tested for 1.5 compatibility.

Thanks again
Mike

[Stan James]

If you can run it in JDK 5 for a test, try the JConsole tool in the JDK bin directory. It's not as good as some other monitoring tools, but it's free and may show you something useful.

[Khan Malamir]

Thanks for the tip. I will try this tool.

[Rick O'Shay]

>> System.gc()

This can provide insight in to memory usage for debugging purposes. Aside from that it is precisely useless. In fact, in addition to accomplishing nothing it will probably degrade your system's performance. The JVM manages memory so all you need to do is not accumulate large amounts of memory by reducing class and static memory usage where possible.

[Jayesh Lalwani]

Originally posted by Ernest Friedman-Hill:
Hi,

The memory allocation from the OS can grow, but can never shrink. Free memory is returned to Java's internal heap, but never to the OS.

Is this true for all JVM's or just the Sun's JVM?

Wouldn't this be considered a "flaw" in java? If I have a memory intensive operation running 1% of the time, and normal operations take just 10% of peak memory, then Java would hold 90% of memory which would be unused for 99% of the time. Wouldn't it be better if Java just returns the memory back to the OS?

[William Brogden]

>> System.gc()

This can provide insight in to memory usage for debugging purposes. Aside from that it is precisely useless.

I disagree - if your application has just created a display that you know the user will take some time to read, it is an ideal time to call System.gc() with minimum impact on the perceived program speed.
Bill

[Gokul Somasundaram]

I disagree - if your application has just created a display that you know the user will take some time to read, it is an ideal time to call System.gc() with minimum impact on the perceived program speed.

There may be some other user who has hit the server and waiting for the response. I think calling system.gc() may affect his response time.

[Vinod John]

Originally posted by Jayesh Lalwani:

Wouldn't this be considered a "flaw" in java? If I have a memory intensive operation running 1% of the time, and normal operations take just 10% of peak memory, then Java would hold 90% of memory which would be unused for 99% of the time. Wouldn't it be better if Java just returns the memory back to the OS?

The maximum heap size that a JVM can consume is set by -Xmx parameter, so this
size is obviously not in control of the JVM. If we can make a set a optimal value for this parameter then, I guess the impact for the above scenario will be less.