aspose file tools*
The moose likes JDBC and the fly likes java.lang.OutOfMemoryError Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Databases » JDBC
Bookmark "java.lang.OutOfMemoryError" Watch "java.lang.OutOfMemoryError" New topic
Author

java.lang.OutOfMemoryError

Khirod Patra
Greenhorn

Joined: Mar 27, 2008
Posts: 23
Hi,

OutOfMemorryError due to large number of Records near about 50,00000 fetching Using Hibernate.I used the Pagination technique to fetch records 5000 per Query. Also used

Runtime runtime=Runtime.getRuntime();
runtime.gc();

and

System.gc();
To flush the Heap of JVM, But after 1,17000 records , it shows The Above Error.Is there any Other way to Flush the Heap space Of JVM.
This Problem can be solved if I increase the JVM size explicitly, But I need to Flush the Heap space Instead of Increasing the Size Of JVM.


Regard
Khirod Patra
Paul Sturrock
Bartender

Joined: Apr 14, 2004
Posts: 10336

There is no way to flush the heap in Java. Even the steps you have tried very possibly do nothing (since you cannot explicitly envoke the garbage collector).

You can however tune how grabage collection works. See the Hotspot options.

Those aside, the safest thing to do is increase the size of the heap available to the JVM.


JavaRanch FAQ HowToAskQuestionsOnJavaRanch
Khirod Patra
Greenhorn

Joined: Mar 27, 2008
Posts: 23
Thanks for your Response, I increase the Heap size of JVM , But do not think that it is a permanent solution. In my problem I m using Hibernate , And I got one Solution, i.e By using Pagination. After using Pagination I got The Same Problem again, so I used to a session for 1000 record at a time and then close the Session. and repeat this in Loop.But I do not know Whether I m in correct Path ???

Sample Code:--


for(rowNo=0;rowNo<count;rowNo=rowNo+10000)
{
Session ses=HibernateUtil.getSessionFactory().getCurrentSession();
Transaction tr1=ses.beginTransaction();



list=ses.createQuery("from example6.Book b order by b.id asc").setFirstResult(rowNo).setMaxResults(10000).list();
Runtime runtime=Runtime.getRuntime();
runtime.gc();
for(int i=0;i<list.size();i++)
{

book=(example6.Book)list.get(i);
System.out.println(book.getId()+" "+book.getAuthor()+" "+book.getTitle());

}
list=null;
book=null;
ses.close();
System.gc();

}



Regards
Khirod Patra
Vaibhav Sharma
Greenhorn

Joined: Mar 28, 2008
Posts: 14
I am not so sure about the Hibernate,but i feel we can fix this problem easily in case of JDBC by setting the fetch size with setFetchSize() method of ResultSet.
I am also awaiting the response to this query.


SCJP 1.4 <br />SCWCD 5.0
Paul Sturrock
Bartender

Joined: Apr 14, 2004
Posts: 10336

Originally posted by Khirod Patra:
Thanks for your Response, I increase the Heap size of JVM , But do not think that it is a permanent solution. In my problem I m using Hibernate , And I got one Solution, i.e By using Pagination. After using Pagination I got The Same Problem again, so I used to a session for 1000 record at a time and then close the Session. and repeat this in Loop.


If you are running out of memory you have two choices:

1) increase the amount of memory available to your JVM. It could be that you program just needs more memory so this is a quick and easy solution. This is very probably the case if you are doing stuff with large data sets. If you find your program keeps on needing more and more memory you may have a memory leak, in which case you need to profile your application and see what is happening.
2) Decrease the amount of objects you maintian in memory. Since Hibernate's Session is a cache, pagenation is a good technique to do this. Also, since this is Hibernate, you can check what data you are bringing back. Have you mapped any associated objects to be lazy? Are youeagerly fetching stuff you don't need?


I would also add that using an ORM to process 50,00000 records is probably not a good choice of tools. Why such a large result set? Are you doing data migration work? If so, you might consider the import and export tools that come with your database. They may be better suited to the task.
[ June 23, 2008: Message edited by: Paul Sturrock ]
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: java.lang.OutOfMemoryError