• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Reading a large text file

 
Ranch Hand
Posts: 46
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,
I want to read from a .htm file (around 800KB, 18000 lines) and store the text in a String object. The code follows:


This works for smaller files, but for this file it is not working.
Please help.

Thanks,
Venu Chakravorty.
 
Marshal
Posts: 79178
377
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
How do you know it is not working? Maybe it is working, only taking a long time. You appear to be using the + operator on Strings several million times. That can have a very severe performance overhead; try the append() method of a StringBuilder instead.
 
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I can't see any reason to read line by line.

If this was my problem, I would get the file length, create a char[] the right size and use the InputStreamReader method to read straight to the char[].

The char[] could be used to construct a String or you could fiddle with it in the char[] form.

Bill
 
Ranch Hand
Posts: 781
Netbeans IDE Ubuntu Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

William Brogden wrote:I can't see any reason to read line by line.



++


If this was my problem, I would get the file length, create a char[] the right size and use the InputStreamReader method to read straight to the char[].



Not really viable unless one know the relationship between the number of bytes and the number of chars. If you absolutely have to do this, and one rarely has to, and if you you not willing to use "apache.commons.io" then I think the easiest way to to this is to just read all the bytes into an array using DataInputStream#readFully(). One get the file length from the File object. One then converts the byte array to a String specifying the character encoding to use.

Note that a file can be longer than 2 GByte but a byte array is limited to 2 GBytes so even if one had enough memory one can't read guarantee to read the whole of a file into memory.
 
Venu Chakravorty
Ranch Hand
Posts: 46
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks a lot.
 
Venu Chakravorty
Ranch Hand
Posts: 46
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks a lot.
 
reply
    Bookmark Topic Watch Topic
  • New Topic