• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Paul Clapham
  • Ron McLeod
  • Jeanne Boyarsky
  • Tim Cooke
Sheriffs:
  • Liutauras Vilda
  • paul wheaton
  • Henry Wong
Saloon Keepers:
  • Tim Moores
  • Tim Holloway
  • Stephan van Hulst
  • Carey Brown
  • Frits Walraven
Bartenders:
  • Piet Souris
  • Himai Minh

Split and Merge large zip files

 
Greenhorn
Posts: 26
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello

Can anyone tel me how can i split and then merge large zip files using java?

Thank you!
 
Sheriff
Posts: 3036
12
Mac IntelliJ IDE Python VI Editor Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
By split you mean unzip, and by merge you mean zip everything into one archive? Check into the java.util.zip package.
 
Vinod Borole
Greenhorn
Posts: 26
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I have a .NET application that reads a 1GB size zip file and breaks it into 5 parts of each 200MB, one by one each part is sent to my java application in a multipart request. My java applications downloads each part and store in a location. Now as i have all the parts, i need to merge them together into a 1GB zip file as it was original.
I am not finding a way how to merge them, Can you please suggest?

Note: The .NET application was throwing Out of memory exception when it tried sending a 1GB zip directly, the maximum it could send was 200MB.
 
Greg Charles
Sheriff
Posts: 3036
12
Mac IntelliJ IDE Python VI Editor Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Ah, so it's slicing apart a large file into smaller chunks. In that case, it doesn't really matter that it was a zip file to start with. You need a FileInputStream to read the five files in order, and a FileOutputStream to output them to a single reassembled file. For each file, read some bytes into a byte array (4096 is probably a good size to use) then write that byte array to the assembled file. When you run out of bytes in one file, open the next one and keep going.
 
Sheriff
Posts: 22650
126
Eclipse IDE Spring VI Editor Chrome Java Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
That will work if the file is split in the simple "let's cut here and continue with the next one" way. Other splitting / merging methods may put some metadata in the split files.
 
Marshal
Posts: 27214
87
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Greg Charles wrote:For each file, read some bytes into a byte array (4096 is probably a good size to use) then write that byte array to the assembled file. When you run out of bytes in one file, open the next one and keep going.

A SequenceInputStream might help with this process.
 
Vinod Borole
Greenhorn
Posts: 26
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I have tried a sample project where I have broken down 67MB zip file into 2 parts of one of 50MB and the other of 17MB. It's taking me more than 30mins to merge them together with the below code.
the zip file after merging that i get is of perfect size, but when i extract it its saying corrupted.

Can any tel me what am i doing wrong? Also, if anyone can suggest me fastest way of doing the same work, as its taking long time to merge with this code.



 
Greg Charles
Sheriff
Posts: 3036
12
Mac IntelliJ IDE Python VI Editor Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Don't use readLine(). That's for reading text, and may be changing the line endings. You don't need anything fancy. Even the DataInput/OutputStreams seem unnecessary, and you will do your own buffering so the the Buffered Streams are overkill as well. Create an array of bytes, read bytes from the input into the array, from the return value figure out how many bytes you read, and write them to the output. It shouldn't take anywhere near that long for 67 megs.
 
permaculture is a more symbiotic relationship with nature so I can be even lazier. Read tiny ad:
free, earth-friendly heat - a kickstarter for putting coin in your pocket while saving the earth
https://coderanch.com/t/751654/free-earth-friendly-heat-kickstarter
reply
    Bookmark Topic Watch Topic
  • New Topic