• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Tim Cooke
  • Liutauras Vilda
  • Jeanne Boyarsky
  • paul wheaton
Sheriffs:
  • Ron McLeod
  • Devaka Cooray
  • Henry Wong
Saloon Keepers:
  • Tim Holloway
  • Stephan van Hulst
  • Carey Brown
  • Tim Moores
  • Mikalai Zaikin
Bartenders:
  • Frits Walraven

Reading large txt file size> 300mb

 
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
how to read very large file and store into array size>300 mb
 
Bartender
Posts: 6109
6
Android IntelliJ IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi, and welcome to the Ranch!

You'd use the same exact code as you would to read and store a smaller file.

But why do you need to store the whole file in memory? And why do you want to use an array?
 
Manishkumar Patel
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Jeff Verdegan wrote:Hi, and welcome to the Ranch!

You'd use the same exact code as you would to read and store a smaller file.

But why do you need to store the whole file in memory? And why do you want to use an array?




i want to read large file and spilt into smaller file. suppose file having size 300 mb,i want to split into 30 file having each 10 mb size
 
Saloon Keeper
Posts: 7645
178
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
So you don't need to keep 300MB in memory, just 30MB, correct? That's a small amount of memory for a modern computer.
 
Jeff Verdegan
Bartender
Posts: 6109
6
Android IntelliJ IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Moores wrote:So you don't need to keep 300MB in memory, just 30MB, correct? That's a small amount of memory for a modern computer.



Not even that much. Just a 1-byte buffer would work, although it'd be horribly inefficient. A buffer of 8k-64k would be fine, and he probably wouldn't get any performance gain beyond that, but I don't see any particular reason not to just go with the full 30 MB.
 
A timing clock, fuse wire, high explosives and a tiny ad:
Gift giving made easy with the permaculture playing cards
https://coderanch.com/t/777758/Gift-giving-easy-permaculture-playing
reply
    Bookmark Topic Watch Topic
  • New Topic