Two Laptop Bag*
The moose likes Java in General and the fly likes Best way to read large file into memory Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Java in General
Bookmark "Best way to read large file into memory" Watch "Best way to read large file into memory" New topic
Author

Best way to read large file into memory

Buzz Andersen
Ranch Hand

Joined: Aug 28, 2001
Posts: 54
This is a question that has really been giving me fits: I am writing a program that I want to use to transfer arbitrarily large files using SOAP. Because I am using SOAP (as opposed to sockets or something lower level like that), I really need to read the entire file into memory and send it as a SOAP parameter (rather than buffering it and sending a little at a time).
The most obvious solution, of course, is to read the entire file into a byte array. But here's the tricky part: I haven't been able to confirm this for sure, but it seems that Java arrays cannot be longer than Integer.MAX_VALUE elements. How, then, do I read in a file with, say, 155,000 bytes?
The only other solution I can think of is to break the bytes up into pages of, say, 1024 bytes each, and store each page in an array. Can anyone comment on whether or not this is a good idea or bad idea? Any better ideas?
Thanks much...
Buzz Andersen
David Weitzman
Ranch Hand

Joined: Jul 27, 2001
Posts: 1365
The int max around 2 billion. That'll hold 155,000 bytes easy. Do you mean 155 gigs?
[ May 07, 2002: Message edited by: David Weitzman ]
Buzz Andersen
Ranch Hand

Joined: Aug 28, 2001
Posts: 54
Oh--you're right. Boy is my face red: I was mixing up the maximum values of "int" with "short" :-(.
I guess what this means is that my problem (a java.lang.OutOfMemoryError) must originate somewhere within Apache SOAP...
Thanks for your help!
Buzz
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Best way to read large file into memory
 
Similar Threads
Convert image to raw binary data
Specific Attachment Problem...
Reading a large text file
When to use setBytes() and setBinaryStream()
extracting bytes from byte array