OutOfMemoryError: Java heap space error when exporting more than 65MB file

Hi all,

My requirement is to export 65Mb file in a CSV format.There is no possibility for me to increase the heap size.
But ,I get the following error which is below the code while doing an export.
Whole 65mb ,I am putting into the response stream.I am not sure whether this is allowed or not.If not,please tell me the alternatives.

ServletOutputStream out = null;
final FacesContext faces = FacesContext.getCurrentInstance();
final HttpServletResponse response = (HttpServletResponse) faces
.getExternalContext().getResponse();
response.setContentType( "application/download" );
response.setHeader( ...................);
final byte[] byteArray = eachLine.toString().getBytes();
response.setContentLength( byteArray.length );
out.write(byteArray);

java.lang.OutOfMemoryError: Java heap space
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:95)
at org.apache.myfaces.webapp.filter.ExtensionsResponseWrapper$MyServletOutputStream.write(ExtensionsResponseWrapper.java:138)
at org.ajax4jsf.io.FastBufferOutputStream.writeTo(FastBufferOutputStream.java:151)
at org.ajax4jsf.framework.ajax.xmlfilter.FilterServletResponseWrapper.sendContent(FilterServletResponseWrapper.java:469)
at org.ajax4jsf.framework.ajax.xmlfilter.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:193)
at org.ajax4jsf.framework.ajax.xmlfilter.BaseFilter.doFilter(BaseFilter.java:220)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.myfaces.webapp.filter.ExtensionsFilter.doFilter(ExtensionsFilter.java:100)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.myfaces.webapp.filter.ExtensionsFilter.doFilter(ExtensionsFilter.java:147)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

It looks like byteArray contains the entire response and that you are trying to send the entire CSV response at once.
You may want to try wrapping out in a BufferedWriter (although it should already be, but a bit hard tell in your code)
and then write the response to the output stream in parts rather than all at once.

How would I wrap a bufferedwriter here.Please help.thanks

You haven't shown the line of code where you assign a value to out, but typically it is easy as new BufferedWriter(out);

Here is how m initializing out - out = response.getOutputStream();
I shal check now with buffered writer and see whether the problem exists

I could see that new BufferedWritter constructor doesnot take out here which is a servletoutputstream.Above thing didnot work

In the following code:

what is eachLine, and why are you creating an entirely new object from it with toString() when all you want is the bytes?

Does this CSV data exist in a file somewhere or is some other process creating it in memory?

Does your program create the CSV data? if not, what is the source?

Bill

There is a file in some format in some location.
eachLine is the every line that I read from the buffered reader of that file.
I read the file line by line which is a string buffer which is converted to string and write it to csv.

I read the file line by line which is a string buffer which is converted to string and write it to csv.

Why do you find it necessary to set the content length? Without that requirement, you could write to the response stream as each line is read.

It is still not clear - does the file exist in some other format that you have to convert line by line?

Bill

or read and return the source file length (again, only if you are not providing any conversion on the data)

Crossposted: http://forums.sun.com/thread.jspa?threadID=5378438&messageID=10668349#10668349
This is very rude. Please read this: http://faq.javaranch.com/java/BeForthrightWhenCrossPostingToOtherSites