• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Problem with ZipInputStream

 
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I am trying to read a zip file having size approximately equal to 300 mb , which when unzipped comes to 4.5 gb

while executing i am getting below error:

Caused by: java.util.zip.ZipException: invalid entry size (expected 4294967295 but got 5230434248 bytes)
at java.util.zip.ZipInputStream.readEnd(ZipInputStream.java:376)
at java.util.zip.ZipInputStream.read(ZipInputStream.java:148)




The file consist of approximately 9993030 records.

Below is the code snippet that i am running






The above code reads 9993023 lines of data after that it ends with the error message invalid entry size (expected 4294967295 but got 5230434248 bytes)

Unzipped the file and then read it using java was able to read all the records.

ZipInputStream internally use inflater class. After reading 9993023 line on next read the Inflater class read method returns 0 as read bytes because of which the EOF is set and we get the above error.

Ideally Inflater should return some bytes as still the records exists.

Is there and issue with the Inflater class of java and why is it not able to read the rest of the records.


Any help appreciated!!



 
Sheriff
Posts: 22783
131
Eclipse IDE Spring VI Editor Chrome Java Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Bhavik Shahm wrote:[b]Caused by: java.util.zip.ZipException: invalid entry size (expected 4294967295 but got 5230434248 bytes)


4294967295 is 2^32 - 1. In other words, at 4GB the ZIP file fails. If I recall correctly, ZipFile and ZipInputStream were limited to this size up to Java 6. If I also recall correctly, this limited was removed in Java 7.

From Apache Commons Compress:

The traditional ZIP format is limited to archive sizes of four gibibyte (actually 232 - 1 bytes ≈ 4.3 GB) and 65635 entries, where each individual entry is limited to four gibibyte as well.


Besides the file size, the number of entries you have also exceeds the limits.

So you have two choices: upgrade to Java 7, or switch to Apache Commons Compress 1.3 or higher.
 
Bhavik Shahm
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks for the reply.

The same thing when i am trying with below code snippet it is working

ZipFile zip=new ZipFile(new File("Y2012071.EDI.zip"));

Enumeration enum1=zip.entries();

while(enum1.hasMoreElements()){

ZipEntry ze=(ZipEntry)enum1.nextElement();

BufferedReader br = new BufferedReader(
new InputStreamReader(zip.getInputStream(ze)));
String line;
int linNum = 1;
while ((line = br.readLine()) != null) {
System.out.println(linNum);
linNum++;

}


Both getting InputStream from ZipFile or creating new ZipInputStream uses Inflater class for decompressing the data.

But i am getting error when i use ZipInputStream and not in case when i get inputstream using ZipFile.getInputStream.

Moreover it is reading the records and giving error before 15 records to EOF.

Will also check apache commons compress api

 
Rob Spoor
Sheriff
Posts: 22783
131
Eclipse IDE Spring VI Editor Chrome Java Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Also, make sure the ZIP file is not corrupt by testing it in another application, like WinZip or 7-Zip.
 
Consider Paul's rocket mass heater.
reply
    Bookmark Topic Watch Topic
  • New Topic