• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Read/Write

 
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Is there anyway to read and write the file without loading the file into memory. I have to read and append the data to a large file (more than 2gb). When I try to use BufferedReader/BufferedWriter it loads the file into the memory (I believe) and I am facing some out of memory error. Please advice.

Thanks
Siva.
 
Bartender
Posts: 2700
IntelliJ IDE Opera
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I believe that this can be done with a RandomAccessFile.
 
Sheriff
Posts: 22783
131
Eclipse IDE Spring VI Editor Chrome Java Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Siva Kannan wrote:When I try to use BufferedReader/BufferedWriter it loads the file into the memory (I believe)


No it doesn't. It stores a little portion, but definitely not the entire file. Are you sure it's not you that stores the entire file? You shouldn't, as you will only need a few (usually one) lines at a time.

Wouter: RandomAccessFile can be used for binary files, but for text files it's not really good (ASCII files can be handled but I wouldn't go any further).
 
Siva Kannan
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks for the reply Rob and Wouter.

I have a small clarification in the below code Rob.

Consider I have a 2gb text file and I need to append something at the end of the file.

FileOutputStream fis = new FileOutputStream(file);
OutputStreamWriter osw = new OutputStreamWriter(fis);
BufferedWriter bw = new BufferedWriter(osw);
bw.append("Lots there to learn");
bw.close();

For the above code when I create file output stream, don't the JVM load the whole file into the memory atleast temporarily? Please correct me if i am wrong.

Thanks Again.
 
Rob Spoor
Sheriff
Posts: 22783
131
Eclipse IDE Spring VI Editor Chrome Java Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
FileOutputStream and FileWriter have constructors that allow you to append to the file instead of overwriting it.
 
This one time, at bandcamp, I had relations with a tiny ad.
a bit of art, as a gift, that will fit in a stocking
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic