Garbage Collection

[muthu muruges]

Hi,
Whether the objects removed from a vector will be gc'ed?.
I am having a vector which will store application specific alarm records. The vector grows slowly in normal working condition. In some error conditions the the vector will grow very fastly. The condition is that the vector can have maximum of 4000 records. If it reaches beyond the limit then the oldest records to be removed from the vector and to maintain the size to 4000. When I am removing the oldest records they are not reclaimed from the memory by the GC, eventhough I forced the GC. So the memory size continuosly increasing for the process (thru' ps -aux). How to make it gc'ed.

[Kaustubh Patil]

First and foremost.. nothing can force garbage collection. You can only indicate to the JVM to do GC, but still its just an indication.
You can probably set your objects as Soft References. This way you can be sure that those objects will be garbage collected before the JVM throws "OutOfMemory" error.
Let me know if you find a better way..
-Kaustubh.

[muthu muruges]

The application is running in java1.1 so I can't use SoftReference at all. I have put the verbosegc switch during the program startup. From that I am sure the GC got invoked when I call System.gc.

[Michael Morris]

Originally posted by m muruges:
The application is running in java1.1 so I can't use SoftReference at all. I have put the verbosegc switch during the program startup. From that I am sure the GC got invoked when I call System.gc.

Are you sure you don't have any dangling references? You may need to get a profiler and see if you can track down the source of the memory leak. Also, since you are using such an old version of java, you probably should check the
Bug Parade to see if there are any known issues for that version.

[Pavel Halas]

Is there any reason for using old java version?
Since 1.4 (HotSpot VM) version is garbage collecting certain. Use

System.runFinalization();
System.gc();

Full range information at Garbage collection

[Ernest Friedman-Hill]

So the memory size continuosly increasing for the process (thru' ps -aux).

This sentence is the key. You're assuming that when the objects are GC'd, the process size will shrink. The most common JVMs (Sun's, IBMs, others) never return memory to the OS. Once the Java heap grows it never shrinks. After GC, more memory is available to the Java process to be reused, but that memory isn't returned to the OS.

[muthu muruges]

So the process memory size will keep on increase?.
I gave -ms32m -mx128m during the program startup. The total memory size of the solaris box is 128MB. Whether it result in performance problem for other process?, Since the java process memory keep on increasing.
Could you please tell me the reason why it is keep on increasing if it is not a memory leak? any workaround to restrict this.
Thanks in advance.

[Ernest Friedman-Hill]

The -mx switch tells the JVM the largest size you will allow the Java heap to grow to. If you don't want it to get that big, then use a smaller argument!
Very loosely, the way a garbage collector works is that it lets garbage "pile up" until there's not enough memory left for the program to run; then it will put some effort into cleaning up the garbage and turning it back into free memory. If you tell the JVM that the heap can grow to 128M, then the garbage collector won't get really busy until, indeed, the heap has gotten that large. Getting more memory from the OS is easier than garbage collection.
If you make the heap limit smaller, then the garbage collector will work harder to make the program fit into this smaller heap size; the performance may even be -better- (as counterintuitive as that sounds, it really does happen.)

[muthu muruges]

If we reduce the mx size then the frequency of GC will be high right? Whether it may result in performance hit?
If the frequency of the GC increases then CPU usage % of the java process will be high. This may hit the other processes' performance, am I correct?

[Jim Yingst]

If we reduce the mx size then the frequency of GC will be high right? Whether it may result in performance hit?
If the frequency of the GC increases then CPU usage % of the java process will be high. This may hit the other processes' performance, am I correct?
Primarily, the choice is: would you rather have lots of small performance hits, or a small number of large performance hits? In many cases, I'd prefer the former - but it depends on your application. Also, be reducing -mx you're reducing memory usage, which reduces the chance that your machine will have to use virtual memory for some of its processes. Once you start using disk space instead of RAM, that's a big performance hit - avoid this. So a lean, mean JVM is often good for everyone.
[ July 16, 2003: Message edited by: Jim Yingst ]

[muthu muruges]

I am having small number of large performance hit. My application is running fine for the first week. At the end of the third week it uses 100% of the cpu time and more than 80% of the memory. Now the appln. will be very slow, not only that the other processes running in the solaris box also have performance hit.
Total memory of the solaris box is 128 mb. Can you please give your suggestion on this?

[Jim Yingst]

I gave -ms32m -mx128m during the program startup. The total memory size of the solaris box is 128MB.
This is bad. This means that the JVM can grow to use all the memory on the box. Well, the system will start using virtual memory before that happens - that's why things have gotten very slow for you. The system is spending all its time swapping them out of memory.
I would definitely reduce the -mx amount. Take a look at how much memory your other processes use when your java VM isn't running/ Make sure they have at least that much available even when the JVM is at maximum. If other processes need 30 MB, then set -mx to maybe 70-80 at most (leave some extra space).
In fact, cutting -mx down a lot more is probaly good. Either (a) the system will clear itself more quickly and easily, or (b) the JVM will throw OutOfMemroyError more promptly (indicating that you have a memory leak which GC cannot fix). This makes it much easier for you to study and debug the problem - you don't want to wait 3 weeks to see your program crash; you want to see it in as little time as possible. Then edit the code to see if you can fix the problem, and try again. A fast turnaround time is a good thing for debugging. Crash early, crash often. Of course, this assumes you're not currently running this thing in production. If you are - well, make a copy, give it very little memory, and experiment on that - and give the main program a large but reasonable amount of memory so it doesn't die too frequently while you try to fix the problem.
If you are getting OutOfMemoryErrors, well, you've got to look at where you're keeping references. Instance variables and class variables are good candidates, especially if they're Vectors. And if you're using 1.1 (ugh) then you may want to consult the bug reports for "memory leak" and any unusual classes you're using. I remember that there were several AWT components that seemed to have leaks in them. Consider getting a commercial memory tool like JProbe; could very well be worth the money. Good luck...

[muthu muruges]

Is there any open source profiler(for java 1.2.2 runs in solaris) available?
[ July 21, 2003: Message edited by: m muruges ]