garbage collection listener for cache cleanup

[peter coster]

I'm not really shure wheither this belongs to advanced but:

We have a webapplication that has a cache containing the data retreived from the backend. This cache has the normal "expiration" system, but will normally never actually clean up anything, this because the application should be able to run when the backend is down (amount of data is limited, so it's possible that everything will fit in the cache). But off course we do need a system that will clean up this cache when an outofmemoryerror is imminent. So I thought to implement a garbagecollect listener that will check the free memory after a gc occured and take action if required. I know this is possible with Jrockit, but we are using Sun's jvm. First question is, how would you do this?

But also:
- do you consider this appropriate ?
- alternatives ?

[Ernest Friedman-Hill]

The standard Java way to do this is for the cache to use WeakReferences -- see the class java.lang.ref.WeakReference. A WeakReference is a reference that is cleared as needed to make space.

[Peter Chase]

Sorry to question one of the experts, but that's not right, is it? A WeakReference does not wait until memory is due to run out. A WeakReference is usually cleared fairly promptly after all strong references are gone.

For caching, one normally wants a SoftReference. These are cleared later than WeakReference, and are usually held until memory is due to run out. They are still guaranteed to be cleared before OutOfMemoryError is thrown.

WeakReference and SoftReference are both examples of the general concept of a "weak reference".

[peter coster]

Thx. for the input so far. I've already looked into the SoftRefence.

One question tough. How will the HashMap's get method cope with equals() and hashCode methods ? because they would be referring SoftReference objects instead of my own CacheKey objects. Or should I just extend from SoftRefence and make my own version?

[peter coster]

I found examples on the Internet, but I don't really like em.
I have a possible solution, using this:

   
import java.lang.ref.SoftReference;
 
public class CacheKeyReference extends SoftReference {
 
    public CacheKeyReference(Object o) {
        super(o);
    }
 
    public boolean equals(Object o) {
        if (this.get() != null && o instanceof SoftReference)
            {
                return this.get().equals(((SoftReference) o).get());
            } else
            {
                return super.equals(o);
            }
    }
 
    public int hashCode() {
        if (this.get() != null)
            {
                return this.get().hashCode();
            } else
            {
                return super.hashCode();
            }
    }
}
 
  

Seems straight forward to me, am I missing something?
[ May 30, 2006: Message edited by: peter coster ]

[Paul Clapham]

Originally posted by peter coster:
How will the HashMap's get method cope with equals() and hashCode methods ? because they would be referring SoftReference objects instead of my own CacheKey objects. Or should I just extend from SoftRefence and make my own version?

Extend from SoftReference. Make two SoftReferences "equal" if their contained CacheKey objects are "equal" and make the hashCode of a SoftReference be the same as the hashCode of its contained CacheKey object.

Which is pretty much like the code you posted, now that I stare at it for long enough. I don't like it either, but I can say why. When the referenced object disappears and all you're left with is the "shell" SoftReference object, then the hashCode changes and the meaning of equals changes. I don't like that. You could deal with the hashCode problem by storing it in an instance variable the first time you calculate it from the referenced object. The equals problem I don't know -- maybe you should reload the referenced object if it isn't there.

For some reason I didn't have to deal with this when I used SoftReference objects. Maybe it was because I automatically reloaded a missing object if something wanted to look at it for any reason.

[Ernest Friedman-Hill]

Originally posted by Peter Chase:
Sorry to question one of the experts, but that's not right, is it?

Nope, sure isn't. Peter's quite right. I shouldn't be allowed to post before I've had my morning coffee.

[Andrew Trumper]

Originally posted by Paul Clapham:

Which is pretty much like the code you posted, now that I stare at it for long enough. I don't like it either, but I can say why. When the referenced object disappears and all you're left with is the "shell" SoftReference object, then the hashCode changes and the meaning of equals changes. I don't like that. You could deal with the hashCode problem by storing it in an instance variable the first time you calculate it from the referenced object. The equals problem I don't know -- maybe you should reload the referenced object if it isn't there.

He's also got this.get() check for null and another this.get() afterwards.. This can fail since object are not gc()ed from any specific thread and the second this.get() can return null!

The SoftReference should not be the key and should not need an equals/hashCode(). A key should be the key. Either that or use a WeakHashMap with SoftReferences for the values (but the object for the keys). That way you don't need to clean up the HashMap when the object goes bye-bye.

If you don't use a WeakHashMap you'll want to add something to the cached object's finalizer to remove the entry in the table. Just remember, the remove opperation has to be thread safe with respected to the rest of the cache.

[Paul Clapham]

Originally posted by peter coster:
edit: ignore please this was stupid

Well, you can delete a post, you know.

But I thought I would just throw this in: I have an application that uses SoftReference a lot; I wrap most of its objects in SoftReferences so that garbage collection can throw them out if it has space problems, but if I actually need the object (e.g. because it has to be displayed in the GUI) then I go out and fetch it again from the database if necessary.

As you're finding, it's hard to test this sort of thing. So what happened was I had a lot of objects loaded up, and the system started to thrash. There was a titanic battle between garbage collection, which needed the memory, and the GUI, which needed the objects. The hard drive churned ad infinitum but I never ran out of memory. So I was convinced that SoftReference worked just fine.

[Mr. C Lamont Gilbert]

peter, changing the hash of objects already mapped is a BAD idea and I think its against the hash contract.

I get the impression you were not using a Map at this point but a list or Set. Your objects will need to be mapped.

As for weakHashMap I have yet to figure out how this thing is supposed to do anything useful. I had to make my own cacheing map that I will be happy to share with anyone that wants it. Its actually not a cache though its a multiton. Should be easy to make into a cache. Well working with references is kind of complicated...I found an error every 6 months for about 3 years...

[peter coster]

The hash fullfills its contract now. And we have been using a hashmap from scratch ( a sync. FastHashMap ), The thing was performing good on our desktop, and has been running for close to a week in a test (with continuous put operations, over 10 billion now, with on average 650.000 elements in cache).

So I do agree that the hashCode method is a bit strange (still is), but it fullfills the contract now ( by storing the hashCode as an Integer (to know diff. between null && 0)), but the result is a high performance cache.
We're using jmx to notify the cache when it should perform a cleanup operation.

One final thing I would need to do is lock the get(...) methods while the cleanup is in progress. Easiest way to do would be to sync. both the cleanup and the get, but that would cause a performance hit, what I would like to do is lock the get only while a cleanup is in progress...

Using Java 1.4 this seems hard.


A bigger problem is that using Sun JVM on HP unix (bea weblogic 8), that these cached elements do seem to be thrown away when it's not required... Is this a bug?
[ June 07, 2006: Message edited by: peter coster ]

[Mr. C Lamont Gilbert]

I don't think you need to lock the get for the whole cleanup. Just lock for each remove operation. I assume remove is already locked just like get and put.

It seems convoluted to me to use JMX or something other than the standard facilities to provide memory sensitivity. Why dont you wrap your values in SoftReferences? This is much easier and should be much more elegant than trying to listen for low memory situation and then determine which elements to remove from cache.

For my cache, each cached object has an ID. The cached objects are mapped based on their IDs. You seem to be using the same object for key as for value. this is incorrect.

Also note that your map is holding a hard reference to the SoftReferences in either case. So you have to register each reference with a ReferenceQueue and when the reference is cleared, you must manually remove the SoftReference from the map. Just because the referrent has been cleared does not mean the map will automatically dump its key-value pair.


And unless you are trying to increase your job security by creating problems to solve, you best to dump the funky equals(). Once the object is put into the map the hash AND the equals methods must not change.
[ June 07, 2006: Message edited by: Mr. C Lamont Gilbert ]

[peter coster]

The equals is also corrected now, we are using softreferences, JMX is only used to trigger the cleanup of these softreferences.

[Mr. C Lamont Gilbert]

Originally posted by peter coster:
The equals is also corrected now, we are using softreferences, JMX is only used to trigger the cleanup of these softreferences.

A SoftReference cleans itself. Thats its whole purpose. There is a whole package 'java.lang.ref' for this type of stuff. Yet you use JMX. Its the proverbial axe to swat a fly from a friends forehead.

A soft reference will clean up its referrent when memory gets low, automatically. You do need to manage cleaning up the soft reference itself.

Also I think i mentioned that GC is tuneable and in turn it makes your cache tuneable.
[ June 10, 2006: Message edited by: Mr. C Lamont Gilbert ]

[peter coster]

Originally posted by Mr. C Lamont Gilbert:


A SoftReference cleans itself. Thats its whole purpose. There is a whole package 'java.lang.ref' for this type of stuff. Yet you use JMX. Its the proverbial axe to swat a fly from a friends forehead.

A soft reference will clean up its referrent when memory gets low, automatically. You do need to manage cleaning up the soft reference itself.

Also I think i mentioned that GC is tuneable and in turn it makes your cache tuneable.

[ June 10, 2006: Message edited by: Mr. C Lamont Gilbert ]

I use JMX to clean that referenceQueue, and to keep my cache sync. with that Queue.

What you suggested 2 posts back, is actually what we've been doing from the start. We did have one problem with it, when we started testing, which is that with the JVM set to client, Softreferences are cleaned to fast. Anyway, The thing works nice and fast now, so thank you all for the input.
[ June 23, 2006: Message edited by: peter coster ]

[Mr. C Lamont Gilbert]

In my setup I have two behaviors I can choose from. #1 is the queue is cleaned up on any action affecting the cache. #2 is I start a thread that pumps the queue whenever it gets a new reference.



#1 I mimmiced from weakHashMap but don't like it much since those references will stay forever if you never touch the cache again.

I also noticed that the JVM can collect Soft references in odd ways. But when I make the Heap large, generally they stuck around longer.

Glad its all working.

[Benjamin Ranck]

Commons.Collections has a reference map for exactly this type of thing:
http://jakarta.apache.org/commons/collections/api-release/org/apache/commons/collections/map/ReferenceMap.html

Cheers,
Benjamin