aspose file tools*
The moose likes Java in General and the fly likes How to get rid of heap OutofMemoryError Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Soft Skills this week in the Jobs Discussion forum!
JavaRanch » Java Forums » Java » Java in General
Bookmark "How to get rid of heap OutofMemoryError" Watch "How to get rid of heap OutofMemoryError" New topic
Author

How to get rid of heap OutofMemoryError

Vinny Menon
Ranch Hand

Joined: Jan 10, 2006
Posts: 62

Hi
I am posting after a long time..Hope everyone is fine
I am using Eclipse as my IDE.
In my application,i am creating an arraylist of my custom object(e.g CustomTransactionObject).The object has around 20 member fields.

I get around 57896 records from the database.each record represents one CustomTransactionObject.
i have to map those values to the CustomTransactionObject.



in this code..when the record count reaches around 22525 ..it gives me an OutofMemoryError.
I tried to change the settings of Eclipse by changing the vmargs parameter from 256M to 512M (ref eclipse.ini)

but still i got the same error..

can you please suggest me a solution or should i go for any other approach

cheers
vinnym


"feels good to be back after a long absence"


Regards,Vinny M.
proud Fan of European Champion CHELSEA FC
"If you don't see the bug where you're looking, perhaps you're looking in the wrong place" -James Gosling
Mike Noel
Ranch Hand

Joined: Dec 15, 2005
Posts: 108
When you switched to 512 did it still crash at about 22000 records? Also, how big are the fields in the class. In order to to run out of memory due to these objects it seems that each one would need to be around 10k in size. Is that about what you're dealing with?

If your records really are that big then you might want to think of a different algorithm for processing them. Even if you solve the memory problem for now your solution won't scale to more data.


Mike Noel
Rusty Shackleford
Ranch Hand

Joined: Jan 03, 2006
Posts: 490
You might want to think about an external ADT , like a B-tree


"Computer science is no more about computers than astronomy is about telescopes" - Edsger Dijkstra
Maulin Vasavada
Ranch Hand

Joined: Nov 04, 2001
Posts: 1873
Hi Rusty

This is little off the thread here but can you really give me example of real life use of B-tree in java code...? Somehow I never could hold of such thing. I know why one would want to use B-tree (for the same reason Vinny is facing OOM Error here) but never could fit to my applications so far

Thanks
Maulin
Naseem Khan
Ranch Hand

Joined: Apr 25, 2005
Posts: 809

Originally posted by Vinny Menon:
In my application,i am creating an arraylist of my custom object(e.g CustomTransactionObject).The object has around 20 member fields.

I get around 57896 records from the database.each record represents one CustomTransactionObject.
i have to map those values to the CustomTransactionObject.



Why you are retrieving that much record from database. Will you show 57,896 records to the end user at one time. obviously not.

You are getting OutOfMemoryError when only one thread is calling databse. What will happen when suppose 100 threads simultaneously call your dao component. You will have 57 lacs 89 thousand 600 instances in single jvm.

Better approach is retrieve only the required data. Say in first page if you want to show 10 records, retrieve 10 rows only. When user want to see more record say from 11-20, then make another call to database and retrieve 11-20 rows.

This approach is called pagination.

More on this check this thread...

Pagination


Naseem


Asking Smart Questions FAQ - How To Put Your Code In Code Tags
Vinny Menon
Ranch Hand

Joined: Jan 10, 2006
Posts: 62

Hi
thanks for your replies..
Why you are retrieving that much record from database. Will you show 57,896 records to the end user at one time. obviously not.

You are getting OutOfMemoryError when only one thread is calling databse. What will happen when suppose 100 threads simultaneously call your dao component. You will have 57 lacs 89 thousand 600 instances in single jvm.

Better approach is retrieve only the required data. Say in first page if you want to show 10 records, retrieve 10 rows only. When user want to see more record say from 11-20, then make another call to database and retrieve 11-20 rows.


naseem, i do not have to show these records on the front end to the user,if i had to show the records, then probably pagination is the best approach,as you have suggested.
I have implemented pagination in my earlier projects,which involved front end .

Instead i have to generate a report(essentially write to a file and provide the file to the user)..
Also i am using a single thread to retrieve data.Also at anytime only one thread will be active for this.

I am still pondering on how to solve this problem :roll:

Thanks
vinny m
Vj Kamath
Greenhorn

Joined: Nov 19, 2004
Posts: 15
Vinny wrote:
i do not have to show these records on the front end to the user,if i had to show the records, then probably pagination is the best approach,as you have suggested.
I have implemented pagination in my earlier projects,which involved front end .

Instead i have to generate a report(essentially write to a file and provide the file to the user)..
Also i am using a single thread to retrieve data.Also at anytime only one thread will be active for this.


You could use pagination for storing in the file as well. The concept is the same. You just retrieve a fixed set of records and write to the file(or report) and clearup the objects, then take the next set.

Advantage: You will used fixed amount of memory. Memory will not be dependant on the number of records you are retrieving

Disadvantage: You will have to make multiple calls to the database instead of one call. This may impact the response time for generating that report.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: How to get rid of heap OutofMemoryError