Java Heap Space Error, OutofMemory Exception while writing large data to excel sheet.
Joined: Apr 09, 2009
I am getting Java Heap Space Error while writing large data from database to an excel sheet.
I dont want to use JVM -XMX options to increase memory.
Following are the details:
1) I am using org.apache.poi.hssf api for excel sheet writing.
2) JDK version 1.5
3) Tomcat 6.0
Code i have wriiten works well for around 23 thousand records, but it fails for more than 23K records.
@Paul Sturrock I think i should work on code, optimized enough to cope with default memory of JVM. Having good hope that some engineering can save me.! can you please go through the code and suggest me some good practises to handle memory usages.
@Ulf Dittmer May be POI can be the reason for outOfMemory, but can you please go through the code and suggest me some good practises to handle memory usages.
I havent run through your code but good design patterns most of the times should keep you from falling into such exceptions.
Did you run your code against a fairly smaller amount of data in your excel sheet. Use the Flyweight Pattern to create your objects if possible.
Author and all-around good cowpoke
Joined: Mar 22, 2000
How about writing the data as CSV lines and stream them to the user? Very low memory use and much faster than POI object creation. Excel should be happy to read CSV and build the spreadsheet.
There appears to be no real reason to build POI objects.
Prashant Kadake wrote:I think i should work on code, optimized enough to cope with default memory of JVM.
Are you're suggesting that it should be possible to modify any code whatsoever to run in any limited amount of memory? That may be true in theory but it may require a lot of work in real life. For example POI is written to keep its data in memory, so the more data you give it, the more memory it uses. It might be possible to modify POI to use temporary disk files (for example) as a backing store, but that would probably take an experienced programmer several months to do.
I don't see any mention of it in this thread, but have you analyzed the JRE's memory usage in detail. The details of this analysis would help identify where and if you have memory leaks in your code and where you can possibly rewrite the code to be more efficient.