• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Paul Clapham
  • Devaka Cooray
  • Liutauras Vilda
Sheriffs:
  • Jeanne Boyarsky
  • paul wheaton
  • Henry Wong
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Tim Moores
  • Carey Brown
  • Mikalai Zaikin
Bartenders:
  • Lou Hamers
  • Piet Souris
  • Frits Walraven

Read/Write

 
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Is there anyway to read and write the file without loading the file into memory. I have to read and append the data to a large file (more than 2gb). When I try to use BufferedReader/BufferedWriter it loads the file into the memory (I believe) and I am facing some out of memory error. Please advice.

Thanks
Siva.
 
Bartender
Posts: 2700
IntelliJ IDE Opera
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I believe that this can be done with a RandomAccessFile.
 
Sheriff
Posts: 22796
131
Eclipse IDE Spring Chrome Java Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Siva Kannan wrote:When I try to use BufferedReader/BufferedWriter it loads the file into the memory (I believe)


No it doesn't. It stores a little portion, but definitely not the entire file. Are you sure it's not you that stores the entire file? You shouldn't, as you will only need a few (usually one) lines at a time.

Wouter: RandomAccessFile can be used for binary files, but for text files it's not really good (ASCII files can be handled but I wouldn't go any further).
 
Siva Kannan
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks for the reply Rob and Wouter.

I have a small clarification in the below code Rob.

Consider I have a 2gb text file and I need to append something at the end of the file.

FileOutputStream fis = new FileOutputStream(file);
OutputStreamWriter osw = new OutputStreamWriter(fis);
BufferedWriter bw = new BufferedWriter(osw);
bw.append("Lots there to learn");
bw.close();

For the above code when I create file output stream, don't the JVM load the whole file into the memory atleast temporarily? Please correct me if i am wrong.

Thanks Again.
 
Rob Spoor
Sheriff
Posts: 22796
131
Eclipse IDE Spring Chrome Java Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
FileOutputStream and FileWriter have constructors that allow you to append to the file instead of overwriting it.
 
Doody calls. I would really rather that it didn't. Comfort me wise and sterile tiny ad:
We need your help - Coderanch server fundraiser
https://coderanch.com/wiki/782867/Coderanch-server-fundraiser
reply
    Bookmark Topic Watch Topic
  • New Topic