java.lang.outofmemory

[Atul Mishra]

Hi all,

I am dealing with an out of memory exception while calling an Oracle stored procedure using Callable statement.

I have called 20000 objects with this procedure, procedure doesnt complain. Then when the objects are 24000 I got the error.

So what I did is, I created sublists with 1000 objects each and then for those 1000 objects, I call the stored procedure.

Still I deal with the same problem..

Any ideas will be appreciated.

Right now when I generate sublists, I am creating connection objects and then closing them also for each 1000 sets.

so its like one connection object will work for 1000 objects.
Callable statement is also registered once for every 1000 objects

csValues = conn.prepareCall("{ ? = call getvalues_proc(?,?,?,?,?,?,?) }");
csValues .registerOutParameter(1, oracle.jdbc.OracleTypes.CURSOR);

then before i stop working on these 1000 objects, I dispose the connection object and the csValues callablestatement also.

But still it fails saying Out of memory exception.

This is what it says exactly

Exception in thread "main" java.lang.OutOfMemoryError
at oracle.jdbc.driver.OracleStatement.prepareAccessors(OracleStatement.java:879)
at oracle.jdbc.driver.T4CResultSetAccessor.getCursor(T4CResultSetAccessor.java:315)
at oracle.jdbc.driver.OracleCallableStatement.getCursor(OracleCallableStatement.java:1659)

Please provide any thoughts

Now here is how I generate sublists.. I am thinking that I keep on creating list objects.. so even though Its working on 1000 objects at a time, the previously creating objects are still existing in memory.

Am I correct ?

this is a sample method I have which I use to create sublist

public void createBatch(List <TestObj> objs ){
		int size = objs.size();
		int i = 0;
		if(size > 1000){
						
		while(i < size){
		
		if((size-i) < 1000) {
			
			List<TestObj> batchObjs = objs .subList(i, size);
			processTestObj(batchObjs,conn);
		}else {
			
		List<TestObj> batchObjs = objs .subList(i, i+1000);  // my doubt is if the total number of items in list is huge, the number of lists getting created are becoming more. 
		log.info("generating sublist" + i);
		processTestObj(batchObjs,conn);

		
		}
		i = i+1000;
		}
		}else{
			processTestObj(batchObjs,conn);
		}
	}

Thanks
Atul


[NK: Added code tags]

[Atul Mishra]

Could any one please provide inputs ?

Thanks

[Satya Maheshwari]

Please use code tags for posting code.
Is it possible to profile your code to see, what is exactly eating up the memory?
Also, can you post the code in processTestObj?

[Peter Johnson]

Add the JVM option -XX:+DumpHeapOnOutOfMemoryError to your app's command line. The JVM will then create a *.hprof file in the current directory when it runs out of heap. Load that file into VisualVM (it comes with JDK6) and it will tell you what is in memory.

If you are running JDK 6, you can attach VisualVM to the JVM running your app and take intermittent memory snapshot and have VisualVM compare them to see what changed in the heap.

[kri shan]

Check your Stack and Heap sizes.

[Atul Mishra]

Hi all,

First-thanks for the replies:

1. satya - sorry I used "quotes".. will use code tags going ahead.
What I am thinking is since I am storing all objects in the collection, Collections[Lists] eat up memory.

Hence I am trying to see whether I can get chunks of resultset and processs the chunks rather than loading all 20K [or even 50K] to collection at once..

Do you agree ?

Thanks Peter and Kri Shan
Peter-we dont have 1.6 yet, still in 1.5
and also i dont think the company will allow modifying prod file to do it.
I will give a try

Thanks

[Peter Johnson]

we dont have 1.6 yet, still in 1.5

That is not a problem. I worked an OOME issue with one of our customers a few weeks back - they are using JDK 5. I had them set -XX:+DumpHeapOnOutOfMemoryError and then load the hprof file into the VisualVM installed on one of their laptops. You do not have to change the JDK for your production environment to make use of the features in VisualVM.

[Atul Mishra]

Peter,

I should just get VisualVM from Jdk 1.6 ?
Trying to download 1.6 and install it on my machine.
Then I will change the jvm option in the AIX machine and get the file.

Will let you know what I find out.

I did it, and added this to the shell script which calls the program
-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=file:///oom.hprof

but I dont see a file. Am i missing any ?


Thanks
Atul

[Atul Mishra]

I did the above and am getting three files only

heapdump.20091105.003746.1380464.phd
Snap0001.20091105.003746.1380464.trc
javacore.20091105.003746.1380464.txt

the other file didnt produce.

Please suggest as of what am I missing

Can i use the .phd file against Visual VM ?

Thanks

[Atul Mishra]

Tried using .phd file, didnt work
Somehow the .hprof file is not getting created

Here is my java home settings and classpath entries

-Dcom.ibm.oti.vm.bootstrap.library.path=/usr/java5_64/jre/bin
-Dsun.boot.library.path=/usr/java5_64/jre/bin
-Djava.library.path=/usr/java5_64/jre/bin:/usr/java5_64/jre/bin:/usr/java5_64/jre/bin/classic:/usr/java5_64/jre/bin:/appl/local/nsp/lib/oracle:/usr/java5_64/jre/bin/j9vm:/appl/local/nsp/lib/oracle:/usr/lib
-Djava.home=/usr/java5_64/jre

[Peter Johnson]

Oh, you are using AIX. You should always say that you are not using the Sun JDK when asking questions because that is what is assumed. The AIX JVM does thing differently than the Sun JVM. But look at the AIX JVM docs to see if there is any tool for analyzing the files that it produced.

[Atul Mishra]

Oh sorry..
Could anyone please suggest any ways to analyze this in AIX environment ?

Thanks

[kri shan]

Better contact IBM guys. We had similar kind of issue in AIX, IBM send the patch and AIX specific tuning parameters to us.