aspose file tools*
The moose likes Servlets and the fly likes OutOfMemoryError: Java heap space error when exporting more than 65MB file    Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Servlets
Bookmark "OutOfMemoryError: Java heap space error when exporting more than 65MB file    " Watch "OutOfMemoryError: Java heap space error when exporting more than 65MB file    " New topic
Author

OutOfMemoryError: Java heap space error when exporting more than 65MB file

monica singh
Greenhorn

Joined: Jan 04, 2009
Posts: 26
Hi all,

My requirement is to export 65Mb file in a CSV format.There is no possibility for me to increase the heap size.
But ,I get the following error which is below the code while doing an export.
Whole 65mb ,I am putting into the response stream.I am not sure whether this is allowed or not.If not,please tell me the alternatives.

ServletOutputStream out = null;
final FacesContext faces = FacesContext.getCurrentInstance();
final HttpServletResponse response = (HttpServletResponse) faces
.getExternalContext().getResponse();
response.setContentType( "application/download" );
response.setHeader( ...................);
final byte[] byteArray = eachLine.toString().getBytes();
response.setContentLength( byteArray.length );
out.write(byteArray);

java.lang.OutOfMemoryError: Java heap space
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:95)
at org.apache.myfaces.webapp.filter.ExtensionsResponseWrapper$MyServletOutputStream.write(ExtensionsResponseWrapper.java:138)
at org.ajax4jsf.io.FastBufferOutputStream.writeTo(FastBufferOutputStream.java:151)
at org.ajax4jsf.framework.ajax.xmlfilter.FilterServletResponseWrapper.sendContent(FilterServletResponseWrapper.java:469)
at org.ajax4jsf.framework.ajax.xmlfilter.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:193)
at org.ajax4jsf.framework.ajax.xmlfilter.BaseFilter.doFilter(BaseFilter.java:220)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.myfaces.webapp.filter.ExtensionsFilter.doFilter(ExtensionsFilter.java:100)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.myfaces.webapp.filter.ExtensionsFilter.doFilter(ExtensionsFilter.java:147)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
David O'Meara
Rancher

Joined: Mar 06, 2001
Posts: 13459

It looks like byteArray contains the entire response and that you are trying to send the entire CSV response at once.
You may want to try wrapping out in a BufferedWriter (although it should already be, but a bit hard tell in your code)
and then write the response to the output stream in parts rather than all at once.
monica singh
Greenhorn

Joined: Jan 04, 2009
Posts: 26
How would I wrap a bufferedwriter here.Please help.thanks
David O'Meara
Rancher

Joined: Mar 06, 2001
Posts: 13459

You haven't shown the line of code where you assign a value to out, but typically it is easy as new BufferedWriter(out);
monica singh
Greenhorn

Joined: Jan 04, 2009
Posts: 26
Here is how m initializing out - out = response.getOutputStream();
I shal check now with buffered writer and see whether the problem exists
monica singh
Greenhorn

Joined: Jan 04, 2009
Posts: 26
I could see that new BufferedWritter constructor doesnot take out here which is a servletoutputstream.Above thing didnot work
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12769
    
    5
In the following code:


what is eachLine, and why are you creating an entirely new object from it with toString() when all you want is the bytes?

Does this CSV data exist in a file somewhere or is some other process creating it in memory?

Does your program create the CSV data? if not, what is the source?

Bill
monica singh
Greenhorn

Joined: Jan 04, 2009
Posts: 26
There is a file in some format in some location.
eachLine is the every line that I read from the buffered reader of that file.
I read the file line by line which is a string buffer which is converted to string and write it to csv.
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12769
    
    5
I read the file line by line which is a string buffer which is converted to string and write it to csv.


Why do you find it necessary to set the content length? Without that requirement, you could write to the response stream as each line is read.

It is still not clear - does the file exist in some other format that you have to convert line by line?

Bill
David O'Meara
Rancher

Joined: Mar 06, 2001
Posts: 13459

or read and return the source file length (again, only if you are not providing any conversion on the data)
Bauke Scholtz
Ranch Hand

Joined: Oct 08, 2006
Posts: 2458
Crossposted: http://forums.sun.com/thread.jspa?threadID=5378438&messageID=10668349#10668349
This is very rude. Please read this: http://faq.javaranch.com/java/BeForthrightWhenCrossPostingToOtherSites
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: OutOfMemoryError: Java heap space error when exporting more than 65MB file