Sending ZIP file over httpResponse

[trupti nigam]

Hello All,

My app needs to create a file from xml String and zip it and send the file over httpResponse to the user. The user will send the request through the URL.

I am able to create the file, ZIp it. But there is issue about sending the zipped file over HTTPResponse.
I can see the ZIP file getting created on the server. When I open it I see the contents. But when the end user recieves the ZIp file (the name etc looks good and correct) and opens it with WinRar, there is nothing inside it.
Below is the code that I use to Send zipped file over ServeletResponse. What am I doing wrong?

 public void sendFile(HttpServletResponse response,String fileName)
    {
       response.setContentType("application/zip");
       response.setContentLength(2048);
       response.setHeader("Content-Disposition","attachment;filename=\"" + fileName + "\"");
       try
       {

          ByteArrayOutputStream baos = new ByteArrayOutputStream();
          ZipOutputStream zos = new ZipOutputStream(baos);
          byte bytes[] = new byte[2048];

          FileInputStream fis = new FileInputStream(fileName);
          BufferedInputStream bis = new BufferedInputStream(fis);
          zos.putNextEntry(new ZipEntry(fileName));
          int bytesRead;
          while ((bytesRead = bis.read(bytes)) != -1)
          {
            zos.write(bytes, 0, bytesRead);
          }

           zos.closeEntry();
           bis.close();
           fis.close();

           zos.flush();
           baos.flush();
           zos.close();
           baos.close();

           ServletOutputStream op = response.getOutputStream();
           op.write(baos.toByteArray());
           op.flush();
           
       }catch(IOException ioe)
       {
           ioe.printStackTrace();
       }
    }

The code to Zip the file is

public String zipit(String fileName,String marketName) throws Exception
    {
         BufferedInputStream origin = null;
         String zipFile = createZipFileName(marketName);
         FileOutputStream dest = new FileOutputStream(zipFile);
         ZipOutputStream out = new ZipOutputStream(new BufferedOutputStream(dest));

         byte data[] = new byte[LENGTH_OF_ZIPDATA];
         // get a list of files from current directory
         File f = new File(fileName);
        
         FileInputStream fi = new FileInputStream(f);
         origin = new BufferedInputStream(fi, LENGTH_OF_ZIPDATA);
         ZipEntry entry = new ZipEntry(fileName);
         out.putNextEntry(entry);
         int count;
         while((count = origin.read(data, 0,LENGTH_OF_ZIPDATA)) != -1) {
         out.write(data, 0, count);
        }
        origin.close();
        out.close();

        return zipFile;
    }

[Richard Tookey]

It seems to me that you are zipping the file twice. If the file being read from
the stream created with

 FileInputStream fis = new FileInputStream(fileName);  
         

is already a zip file then why zip again in the Servlet?

P.S. Also, you should set the content length based on the length of the zip file and not a fixed value.

[trupti nigam]

I was able to solve the issue. Now there is only one issue. When I unzip the file and try to open the xml file in text pad, I see some unwnated chars at the beginning of the file. Due to this The file won't open in Explorer.
How can I avoid those chars?

thanks
Trupti

[Richard Tookey]

trupti nigam wrote:I was able to solve the issue.

Doesn't sound like it to me! If you are still zipping the file twice or you are still setting the content length to a hard coded value then you have not fixed anything.

Now there is only one issue. When I unzip the file and try to open the xml file in text pad, I see some unwnated chars at the beginning of the file. Due to this The file won't open in Explorer.
How can I avoid those chars?

thanks
Trupti

I still think you have the same problem in a different guise BUT I don't have a view of your latest code.

[trupti nigam]

Richard Tookey wrote:

trupti nigam wrote:I was able to solve the issue.

Doesn't sound like it to me! If you are still zipping the file twice or you are still setting the content length to a hard coded value then you have not fixed anything.

Now there is only one issue. When I unzip the file and try to open the xml file in text pad, I see some unwnated chars at the beginning of the file. Due to this The file won't open in Explorer.
How can I avoid those chars?

thanks
Trupti

I still think you have the same problem in a different guise BUT I don't have a view of your latest code.

I am not zipping the file twice..
Below is the code..

public void sendZipFile(HttpServletResponse response,String fileName,String marketName)
    {
       try
       {
          String zipFile = createZipFileName(marketName);
          ByteArrayOutputStream baos = new ByteArrayOutputStream();
          ZipOutputStream zos = new ZipOutputStream(baos);
          File file = new File(fileName);

          FileInputStream fis = new FileInputStream(fileName);
          BufferedInputStream bis = new BufferedInputStream(fis);
          byte bytes[] = new byte[(int)file.length()];
          zos.putNextEntry(new ZipEntry(fileName));
          int bytesRead;
          while ((bytesRead = bis.read(bytes)) != -1)
          {
            zos.write(bytes, 0, bytesRead);
          }

           zos.closeEntry();
           bis.close();
           fis.close();

           zos.flush();
           baos.flush();
           zos.close();
           baos.close();

           response.setContentType(ZIP_CONTENT_TYPE );
           response.setHeader("Content-Disposition","attachment;filename=\"" + zipFile + "\"");

           ServletOutputStream op = response.getOutputStream();
           op.write(baos.toByteArray());
           op.flush();
           
       }catch(IOException ioe)
       {
           ioe.printStackTrace();
       }
    }

This is working finr and the end user does get the zip file.

[Richard Tookey]

trupti nigam wrote:

This is working finr and the end user does get the zip file.

This does not stack up with your earlier statement "When I unzip the file and try to open the xml file in text pad, I see some unwnated chars at the beginning of the file" .

[Paul Clapham]

The originally-posted code takes a file named X and zips that into a zip archive. Then it sends that zip archive as the response, saying (via the Content-Distribution header) that its name is X.

This isn't consistent with X being named something.xml, because then the zip file arriving at the browser would be named something.xml instead of somethingelse.zip. However it is consistent with X being named something.zip, because then the zip file arriving at the browser would be named something.zip and everything would be normal.

So something is screwy here. Either you're zipping a zip file, as Richard suspects, or you're doing something else wrong. At any rate it would be extremely easy to look at this so-called unzipped XML file in a text editor and see if it's really that. Textpad isn't the only text editor in the world, or even on your computer.

[trupti nigam]

Paul Clapham wrote:The originally-posted code takes a file named X and zips that into a zip archive. Then it sends that zip archive as the response, saying (via the Content-Distribution header) that its name is X.

This isn't consistent with X being named something.xml, because then the zip file arriving at the browser would be named something.xml instead of somethingelse.zip. However it is consistent with X being named something.zip, because then the zip file arriving at the browser would be named something.zip and everything would be normal.

So something is screwy here. Either you're zipping a zip file, as Richard suspects, or you're doing something else wrong. At any rate it would be extremely easy to look at this so-called unzipped XML file in a text editor and see if it's really that. Textpad isn't the only text editor in the world, or even on your computer.

I am not using the code posted in the original post. Please check the latest code in my most recent reply. I see the text file getting created. The ZIP file gets created and sent to client. But when the client unzips it and opens it, I see few unwanted chars but the rest of the file looks good.

[Richard Tookey]

So what code generates the file opened by

     FileInputStream fis = new FileInputStream(fileName);  
     

P.S. You still don't set the content length. There is no need to close 'fis' since the close of 'bis' cascades to 'fis' . There is no need to allocate a buffer the size of the file; a fixed size of 16K odd will do. There is no need to buffer the zipped file in the ByteArrayOutputStream; you can write it directly to the ServletOutputStream.

[trupti nigam]

Richard Tookey wrote:So what code generates the file opened by

     FileInputStream fis = new FileInputStream(fileName);  
     

P.S. You still don't set the content length. There is no need to close 'fis' since the close of 'bis' cascades to 'fis' . There is no need to allocate a buffer the size of the file; a fixed size of 16K odd will do. There is no need to buffer the zipped file in the ByteArrayOutputStream; you can write it directly to the ServletOutputStream.

Below code generates the txt file that I zip in the code already mentioned.

 public ObjectOutputStream createFile(SecurityListImpl secResponse,String file)   throws IOException
    {
          ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(file));
          oos.writeObject(secResponse.toString());         
          oos.close();
          return oos;
    }

Also You mentioned that "There is no need to buffer the zipped file in the ByteArrayOutputStream; you can write it directly to the ServletOutputStream." How to I do that? Cause ServletOutPutStream.write method takes byte array.
Is this how you do it?

 zos.toString().getBytes().

[Richard Tookey]

Instead of

       ByteArrayOutputStream baos = new ByteArrayOutputStream();  
          ZipOutputStream zos = new ZipOutputStream(baos);  

you just use

      ServletOutputStream op = response.getOutputStream();  
  

and

         ZipOutputStream zos = new ZipOutputStream(op);  
      

and then do away with the ByteArrayOutputStream !

Note - the only way you are going to get more help is if you post a hex listing of the file before being zipped and ah hex listing of the file on the client after being unzipped.

[trupti nigam]

What is a hexListing of a file?

[Paul Clapham]

Actually the question to be answered is whether the unzipped version of the file is the same as the original file.

All you have said so far is that the unzipped version can't be loaded into Textpad when it's received. You haven't said that the original version can be loaded into Textpad. For all you know, the two files are identical and you are spending unnecessary time looking at your code.

[trupti nigam]

Here is new code after your suggestions. But this does not work... It downloads some file which can not be opened and is not a zip file.

 public void sendZipFile(HttpServletResponse response,String fileName,String marketName)
    {
       try
       {
          String zipFile = createZipFileName(marketName);
          ServletOutputStream op = response.getOutputStream();
          ZipOutputStream zos = new ZipOutputStream(op);
          File file = new File(fileName);

          FileInputStream fis = new FileInputStream(fileName);
          BufferedInputStream bis = new BufferedInputStream(fis);
          byte bytes[] = new byte[(int)file.length()];
          zos.putNextEntry(new ZipEntry(fileName));
          int bytesRead;
          while ((bytesRead = bis.read(bytes)) != -1)
          {
            zos.write(bytes, 0, bytesRead);
          }

           zos.closeEntry();
           bis.close();
           fis.close();

           zos.flush();
           zos.close();

           response.setContentType(ZIP_CONTENT_TYPE );
           response.setHeader("Content-Disposition","attachment;filename=\"" + zipFile + "\"");


           op.write(zos.toString().getBytes());
           op.flush();
           
       }catch(IOException ioe)
       {
           ioe.printStackTrace();
       }
    }

The website didn't allow me to attach file with .xml or .txt extention.

[trupti nigam]

Paul Clapham wrote:Actually the question to be answered is whether the unzipped version of the file is the same as the original file.

All you have said so far is that the unzipped version can't be loaded into Textpad when it's received. You haven't said that the original version can be loaded into Textpad. For all you know, the two files are identical and you are spending unnecessary time looking at your code.

The Zipped file has the original file contents but at the beginning of the file there are some unwanted chars that I see. I am able to open the file in Text PAd, word PAd , note pad etc. But I can not open it in explorer as there are unwanted chars and hence it becomes invalid XML file.

[Richard Tookey]

trupti nigam wrote:Here is new code after your suggestions.

Why on earth are you doing this???

           op.write(zos.toString().getBytes());  
           op.flush();  

I'm now betting that the bytes at the beggining of the file are the BOM (Byte Order Mark) probably introduced by Notepad or some other silly editor. You most definitely need to post the hex representation of the file.

[trupti nigam]

Richard Tookey wrote:

trupti nigam wrote:Here is new code after your suggestions.

Why on earth are you doing this???

           op.write(zos.toString().getBytes());  
           op.flush();  

I'm now betting that the bytes at the beggining of the file are the BOM (Byte Order Mark) probably introduced by Notepad or some other silly editor. You most definitely need to post the hex representation of the file.

So only flushing the servlet stream will work?Please be specific.

Also Here is what I see when I open it in text pad.

¬í | Tì<FIXML>
<SecList RspID="2" ReqID="1" LastFragment="Y">
<SecL Txt="Cocoa">

[Richard Tookey]

You just don't need those last three lines of code!! The zip content has been written already!

Who generated that XML and how? I'm betting it was generated using Notepad. If so then the first three bytes is a BOM and need to be removed before zipping. There is a class I wrote several year ago called BOMStripperiInputStream that can be obtained from http://code.google.com/p/train-graph/source/browse/trunk/src/org/paradise/etrc/data/BOMStripperInputStream.java?r=31 . If you use this to wrap the XML input stream before using it it will get rid of the BOM.

         FileInputStream fis = new FileInputStream(fileName);  
         BOMStripperInputStream bis = new BOMStripperInputStream(fis);
          BufferedInputStream bis = new BufferedInputStream(bis);  
 

P.S. Personally I would not include the BufferedInputStream in the chain since you are doing your own buffering. I would also change

       byte bytes[] = new byte[(int)file.length()];  

to

       byte bytes[] = new byte[4096];  

since, as I said before you don't need to create a buffer the length of the file; since this is a Servlet memory is likely to be at a premium.

[trupti nigam]

Richard Tookey wrote:You just don't need those last three lines of code!! The zip content has been written already!

Who generated that XML and how? I'm betting it was generated using Notepad. If so then the first three bytes is a BOM and need to be removed before zipping. There is a class I wrote several year ago called BOMStripperiInputStream that can be obtained from http://code.google.com/p/train-graph/source/browse/trunk/src/org/paradise/etrc/data/BOMStripperInputStream.java?r=31 . If you use this to wrap the XML input stream before using it it will get rid of the BOM.

In the earlier responses you asked me how am I generating the original file which I zip. Please see the code , how I am generating the original .xml file. This file when sent over servletOutput stream does not contain BOM. But when I zip it and send it to end user and the end user opens the file , I see BOM.

public ObjectOutputStream createFile(SecurityListImpl secResponse,String file)   throws IOException
    {
          ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(file));
          oos.writeObject(secResponse.toString());    
          oos.close();
          return oos;
    }

[trupti nigam]

Richard Tookey wrote:You just don't need those last three lines of code!! The zip content has been written already!

Ok I removed the last 3 lines of the code and it still does not work. Some random download file gets created which is in currupt format.

[Richard Tookey]

trupti nigam wrote:

Richard Tookey wrote:You just don't need those last three lines of code!! The zip content has been written already!

Who generated that XML and how? I'm betting it was generated using Notepad. If so then the first three bytes is a BOM and need to be removed before zipping. There is a class I wrote several year ago called BOMStripperiInputStream that can be obtained from http://code.google.com/p/train-graph/source/browse/trunk/src/org/paradise/etrc/data/BOMStripperInputStream.java?r=31 . If you use this to wrap the XML input stream before using it it will get rid of the BOM.

In the earlier responses you asked me how am I generating the original file which I zip. Please see the code , how I am generating the original .xml file. This file when sent over servletOutput stream does not contain BOM. But when I zip it and send it to end user and the end user opens the file , I see BOM.

public ObjectOutputStream createFile(SecurityListImpl secResponse,String file)   throws IOException
    {
          ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(file));
          oos.writeObject(secResponse.toString());    
          oos.close();
          return oos;
    }

Sorry I missed that !!! That code for generating the file is very very very very wrong. It generates a serialized Java object which is not not not not XML. You should just write the bytes of secResponse! Since I don't know what secResponse is I can't say more at the moment!

Looks like there is no need for the BOM stripper.

It occurs to me that if you are writing the file from your servlet then you don't need to. Just write the bytes of secResponse to the ZipOutputStream !

[trupti nigam]

Here are the final things.
your suggestion of "You should just write the bytes of secResponse! Since I don't know what secResponse is I can't say more at the moment! " does not work.
The BomStripper code does not work.
I tried using it but I still see those chars.

[Richard Tookey]

trupti nigam wrote:Here are the final things.
your suggestion of "You should just write the bytes of secResponse! Since I don't know what secResponse is I can't say more at the moment! " does not work.
The BomStripper code does not work.
I tried using it but I still see those chars.

I have already said that the BOM stripper will not work since the problem is in your file generation and your use of ObjectOutputStream to write the fle!

I still think you don't actually need to write the file in the first place but without a bigger view of your code I can't be certain. Does your servlet write the file before reading it back in to be zipped?

[trupti nigam]

Richard Tookey wrote:

trupti nigam wrote:Here are the final things.
your suggestion of "You should just write the bytes of secResponse! Since I don't know what secResponse is I can't say more at the moment! " does not work.
The BomStripper code does not work.
I tried using it but I still see those chars.

I have already said that the BOM stripper will not work since the problem is in your file generation and your use of ObjectOutputStream to write the fle!

I still think you don't actually need to write the file in the first place but without a bigger view of your code I can't be certain. Does your servlet write the file before reading it back in to be zipped?

Yes My servlet writes the response string (xml format string) to a file before reading it back to be zipped.

Here is the code

String file = createFileName(marketName);
ObjectOutputStream oos = fileHelper.createFile(secResponse,file);
fileHelper.sendZipFile(httpServletResponse,file,marketName);



public ObjectOutputStream createFile(SecurityListImpl secResponse,String file)   throws IOException
    {
          ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(file));
          oos.writeObject(secResponse.toString());
          oos.close();
          return oos;
    }


public void sendZipFile(HttpServletResponse response,String fileName,String marketName)
    {
       try
       {
          String zipFile = createZipFileName(marketName);
          ServletOutputStream op = response.getOutputStream();
          ByteArrayOutputStream baos = new ByteArrayOutputStream();
          ZipOutputStream zos = new ZipOutputStream(baos);

          File file = new File(fileName);

          FileInputStream fis = new FileInputStream(fileName);         
          BufferedInputStream bis = new BufferedInputStream(fis );

          byte bytes[] = new byte[(int)file.length()];
          zos.putNextEntry(new ZipEntry(fileName));
          int bytesRead;
          while ((bytesRead = bis.read(bytes)) > 0)
          {
            zos.write(bytes, 0, bytesRead);
          }

           zos.closeEntry();
           bis.close();
           fis.close();

           zos.flush();
           baos.flush();
           zos.close();
           baos.close();

           response.setContentType(ZIP_CONTENT_TYPE );
//           response.setContentLength(LENGTH_OF_ZIPDATA);
           response.setHeader("Content-Disposition","attachment;filename=\"" + zipFile + "\"");

           op.write(baos.toByteArray());

           op.flush();
           
       }catch(IOException ioe)
       {
           ioe.printStackTrace();
       }
    }

private String createFileName(String marketName)
    {
        SimpleDateFormat dateFormat  = new SimpleDateFormat("yyyyMMddhhmmss");
        dateFormat.format(new Date());
        String thisFile = new String(marketName.concat(dateFormat.format(new Date())) + ".xml");
        return thisFile;
    }

This code creates xyz.xml and then creates xyz.zip but when I unzip the file , it has BOM chars.

[Richard Tookey]

It is not a BOM, it is a result of using the ObjectOutputStream . ObjectOutputStream is inappropriate. You don't need to write the file so createFile() is not needed. In fact the whole can be pretty much reduced to

public void sendZipFile(HttpServletResponse response,SecurityListImpl secResponse,String marketName)  
{  
       String zipFile = createZipFileName(marketName);  
       ServletOutputStream op = response.getOutputStream();  
       ZipOutputStream zos = new ZipOutputStream(op);  
       zos.putNextEntry(new ZipEntry(fileName));  
       zos.write(secResponse.toString().getBytes(Charset.forName("utf-8"));
       zos.closeEntry();  
       zos.close();  

       response.setContentType(ZIP_CONTENT_TYPE );  
       response.setHeader("Content-Disposition","attachment;filename=\"" + zipFile + "\"");  
}  

This is untested by cannot be far from working.

[trupti nigam]

Thanks for all the code. It worked with little modification in the above code.

public void sendZippedFile(HttpServletResponse response,SecurityListImpl secResponse,String marketName,String fileName) throws Exception
    {
           String zipFile = createZipFileName(marketName);
           ServletOutputStream op = response.getOutputStream();
           ByteArrayOutputStream baos = new ByteArrayOutputStream();
           ZipOutputStream zos = new ZipOutputStream(baos);
           zos.putNextEntry(new ZipEntry(fileName));
           zos.write(secResponse.toString().getBytes(Charset.forName("utf-8")));
           zos.closeEntry();
           zos.close();

           response.setContentType(ZIP_CONTENT_TYPE );
           response.setHeader("Content-Disposition","attachment;filename=\"" + zipFile + "\"");

           op.write(baos.toByteArray());
           op.flush();

    }

with your code it was creating the file "download" with no extenstion. When I renamed the file to "download.zip" i was able to open it like regualr text file. But with above modification I am able to get the file with "xyz.zip" format and the inside file does not have BOM.

thanks a lot for your consistent help!

[Richard Tookey]

trupti nigam wrote:Thanks for all the code. It worked with little modification in the above code.

public void sendZippedFile(HttpServletResponse response,SecurityListImpl secResponse,String marketName,String fileName) throws Exception
    {
           String zipFile = createZipFileName(marketName);
           ServletOutputStream op = response.getOutputStream();
           ByteArrayOutputStream baos = new ByteArrayOutputStream();
           ZipOutputStream zos = new ZipOutputStream(baos);
           zos.putNextEntry(new ZipEntry(fileName));
           zos.write(secResponse.toString().getBytes(Charset.forName("utf-8")));
           zos.closeEntry();
           zos.close();

           response.setContentType(ZIP_CONTENT_TYPE );
           response.setHeader("Content-Disposition","attachment;filename=\"" + zipFile + "\"");

           op.write(baos.toByteArray());
           op.flush();

    }

with your code it was creating the file "download" with no extenstion. When I renamed the file to "download.zip" i was able to open it like regualr text file. But with above modification I am able to get the file with "xyz.zip" format and the inside file does not have BOM.

thanks a lot for your consistent help!

Sorry but the ByteArrayOutputStream is redundant! In my code just change the order writing the headers and writing the data!

[Sathesh Sivashanmugam]

Hi,

You are hard coding the content length to 2048 disregarding the original file length. So the browser will download only the first 2048 bytes of the zip file. Try removing that line and try again.