• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

2GB file size limitation for transferTo() in jre 1.5.0_17 in linux and jre1.4.0_18 in Solaris

 
Ranch Hand
Posts: 43
Eclipse IDE Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi All,

We are facing an issue regarding large file transfer (around 4GB) through transferTo() method of FileChannel class in jre 1.5.0_17 in linux and jre1.4.0_18 in Solaris.
It seems that there is a file size limit of 2GB and after that rest of the files getting truncated and we are receiving only 1.99GB files at the destination location.

Any idea why this is happening and how we can resolve the issue ?

Thanks,
Joydeep
 
Bartender
Posts: 4179
22
IntelliJ IDE Python Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
What is the format for the drive being copied to? Some formats (FAT32 for example) have a file size limit of around that size.
 
Joydeep Ghatak
Ranch Hand
Posts: 43
Eclipse IDE Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The destination drive format type is ext4
 
Joydeep Ghatak
Ranch Hand
Posts: 43
Eclipse IDE Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Also we have tried with jre1.6.0_20 (64 bit Linux version), but received the same result (1.99GB destination file size)unfortunately.
Not sure if that is the JRE issue or not .
 
Sheriff
Posts: 22781
131
Eclipse IDE Spring VI Editor Chrome Java Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
How are you using the transferTo method? Like most read/write operations, transferTo doesn't have to do everything in one try. It may need multiple tries. That's why you do something like this in a loop:
 
Joydeep Ghatak
Ranch Hand
Posts: 43
Eclipse IDE Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Rob !!! Thanks Steve !!!

Thanks for the reply,

Well the usage of transferTo() is direct here and I am working on a legacy code, which is something like,


Do you think this is the root of the issue ?

Thanks,
Joydeep
 
Master Rancher
Posts: 4796
72
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
It's definitely worth putting the transferTo() call into a loop, as Rob suggested. Nothing in the docs indicates that the call is guaranteed to transfer all available bytes, and they do provide the return value that tells you how many bytes they did transfer. So you need to check that value and use it, as Rob showed.

It's not a very user-friendly API. They should really provide a method that simply does what is requested, and either blocks until done, or throws an error if it can't. Because that's what many users expect. But it's not what the docs say, and it's not what they did.
 
Joydeep Ghatak
Ranch Hand
Posts: 43
Eclipse IDE Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks everyone for your kind input.
Will try to resolve the issue and update this thread.

Thanks,
Joydeep
 
They worship nothing. They say it's because nothing is worth fighting for. Like this tiny ad:
a bit of art, as a gift, the permaculture playing cards
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic