ObjectInputStream creating OutOfMemoryError

[Ryan Atwood]

Help. So I'm writing a network application that receives live streaming data from another server. The other server (not in my control) sends it via serialized objects and, I believe, my only way of reading these is to use the ObjectInputStream and ObjectOutputStream. My issue is that I'm getting a large number of objects over time and I notice two things
a) there seems to be some sort of bottleneck. the time it takes for the CPU to process an object takes longer.
b) i eventually hit the OOM error. increasing the memory for the application using Xmx only prolongs the problem.

my code looks something like this:

public static void A()
{
	while(true)
	{
		try
		{
			obj = inputStream.readObject());
			while (obj != null)
			{
				System.out.println("Memory: " + usedMemory());
				outputStream.reset();
				outputStream.flush();
				obj = null;
				obj = inputStream.readObject());
			}
		}
		catch (Exception e)
		{
			e.printStackTrace();
		}
	}
}

This is very simple code. The usedMemory() just uses the Runtime.getRuntime() class.The main part is inside the A() function where it does obj = inputStream.readObject(). If I run the code with that line commented out, I notice that my heap size always stays the same (pretty obvious). But if I keep that line, the memory size keeps increasing over time. I stripped the code from all the functionality linked to processing that object. Even then, the size keeps increasing.

This is the OOM memory I get:

    
Exception in thread "Thread-0" java.lang.OutOfMemoryError: Java heap space
        at java.io.ObjectInputStream$HandleTable.grow(ObjectInputStream.java:3420)
        at java.io.ObjectInputStream$HandleTable.assign(ObjectInputStream.java:3227)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1744)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
        at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1667)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1323)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
        at ABCD$1.run(ABCD.java:###)
        at java.lang.Thread.run(Thread.java:619)

I believe that the bottleneck I referred to earlier probably started happening when Java tries to re-size the HashTable (?).

People have suggested closing and re-opening the stream but I can't re-initialize it each time. I'll lose data and time.

Is there any way I can use a different I/O object that's much more efficient at this?

Thank you.

[Joe Ess]

#3 Why is OutOfMemoryError thrown after writing a large number of objects into an ObjectOutputStream?

The ObjectOutputStream maintains a table mapping objects written into the stream to a handle. The first time an object is written to a stream, its contents are written into the stream; subsequent writes of the object result in a handle to the object being written into the stream. This table maintains references to objects that might otherwise be unreachable by an application, thus, resulting in an unexpected situation of running out of memory. A call to the ObjectOutputStream.reset() method resets the object/handle table to its initial state, allowing all previously written objects to be elgible for garbage collection. See handle.

Java Serialization FAQ

[Ryan Atwood]

Thanks for the reply.

I do have the reset code at Line 11 and it doesn't seem to solve the increasing heap or the slow processing times.

[Ernest Friedman-Hill]

Is this your real code? You're resetting and flushing that ObjectOutputStream, but not actually using it for anything. You never write any data to it, and the objects are simply being discarded.

In any case, the stack trace shows an OutOfMemoryError while reading an object. Perhaps that object graph is simply too large to fit in memory. What do you know about the form of the data itself?

[Ryan Atwood]

No my actual code did do stuff with the object. What I'm saying is that, even with all that removed, and just setting it to null, the heap size just keeps increasing. Doing a System.gc() call doesn't seem to help either. If I can't cut down the heap size when I'm not even doing anything with the object, how can I expect to solve this problem after attaching functionality to it.

Thanks

[Joe Ess]

I tried something similar to your code, a server which writes a Date to a socket and a client which reads the date from the socket. I used the JConsole application included with the JDK to monitor the client. As you can see from this 20-minute run, there is no trend in heap consumption (keep in mind that the scale of the graph is .8 to 2 Mb, so we're talking about miniscule variations in heap)


The code for the client side looks much like your own:

import java.io.*;
import java.net.*;
import java.util.*;

public class TimeClient{
    
    public static void main(String args[]){
        try{
            Runtime r = Runtime.getRuntime();     
            Socket serverConnection = new Socket("localhost", 9999);
            ObjectInputStream ois = new ObjectInputStream(serverConnection.getInputStream());
            
            while(true){
                Object o = ois.readObject();
                System.out.println(o);
                long freeMem = r.freeMemory();            
                long totalMem = r.totalMemory(); 
                System.out.println("totalMem = "+totalMem+"  freeMem = "+freeMem  );
            }
            
        } catch (Exception e){
            e.printStackTrace();
        }
    }
}    

So there's got to be something else going on in your code. Can you show us a stand-alone example which demonstrates the behavior you are seeing?

[Ryan Atwood]

First I gotta say this forum is awesome. You guys are really helpful (especially Joe) and quick.

So I used the code above and changed the socket information to mine. Before my program got to this method, it initialized the socket, input/output streams and sent commands to the output stream for logging in.

public static void A()
    {
        Runnable handle = new Runnable()
        {
            public void run()
            {
                try
                {
                    Runtime r = Runtime.getRuntime();
                    while(true)
                    {
                        Object o = is.readObject();
                        System.out.println(o);
                        long freeMem = r.freeMemory();
                        long totalMem = r.totalMemory();
                        System.out.println("totalMem = "+totalMem+"  freeMem = "+freeMem  );
                    }

                }
                catch (Exception e)
                {
                    e.printStackTrace();
                }

            }
        };
        Thread r = new Thread(handle);
        r.start();
    }

I don't know how to use JConsole, but I saved the output of the free/total memory. This is the output I got after a few minutes (<5 minutes). The total memory keeps increasing over time. This screws my latency (can't see that in the output) because, I believe, JVM is trying to allocate more memory and storing/retrieving objects is taking a while.

Right now (10-15 mins later), total memory is at 33,415,168.

Thanks!

[Ryan Atwood]

And sure enough.. I got this a while later:

MyObject@11f0102
totalMem = 66650112  freeMem = 5439840
MyObject@7b51dc
totalMem = 66650112  freeMem = 5439840
Exception in thread "Thread-0" java.lang.OutOfMemoryError: Java heap space
        at java.io.ObjectInputStream$HandleTable.grow(ObjectInputStream.java:3420)
        at java.io.ObjectInputStream$HandleTable.assign(ObjectInputStream.java:3227)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1744)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
        at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1667)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1323)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
        at ABCD$1.run(ABCD.java:147)
        at java.lang.Thread.run(Thread.java:619)
BUILD SUCCESSFUL (total time: 46 minutes 40 seconds)

[Joe Ess]

ryan atw wrote:First I gotta say this forum is awesome. You guys are really helpful (especially Joe) and quick.

Thanks
Hey, how often do you call A()? Is there a chance that you are creating numerous instances and they're all hanging around, blocked on the socket waiting for input? That would account for the memory being used. Try adding an instance counter and printing creation and destruction of that anonymous class.

[Ryan Atwood]

I won't be able to run/test the code until next Monday (no streaming data until then) so I'll show you what my code looks like instead.

    public static void main(String[] args)
    {
        new ABCD();
    }

    public ABCD()
    {
        login();
        A();
    }

[Joe Ess]

You are leaving some code out, like initializing the Socket and ObjectOutputStream. I tried using your code for A() in my TimeClient posted above. The JConsole session for that looked exactly like the one I posted for my previous client. No memory leak.
You can dummy up a server to provide you with some data pretty easily. My server is all of 47 lines of code.

[Ryan Atwood]

    public void login()
    {
        try
        {
            socket = new Socket(host, port);
            out = new ObjectOutputStream(socket.getOutputStream());
            is = new ObjectInputStream(socket.getInputStream());

	    Object MyLoginObject = ....;
            out.writeObject(MyLoginObject);
            out.flush();
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }
    }

That's what I have in my login function. I can't reveal much code because I'm using a proprietary API from another vendor. My guess is that it's their server code that's the fault?

[Joe Ess]

ryan atw wrote: I can't reveal much code because I'm using a proprietary API from another vendor. My guess is that it's their server code that's the fault?

I can't say because I can't read the code and test it!
It would be a stretch to say that the server code is creating a memory leak in your client code.

[Ryan Atwood]

That's true.. but honestly that is my code. The parts I'm hiding are creating these custom objects, and sending them across the socket during login.

Thanks anyways.