aspose file tools*
The moose likes Java in General and the fly likes managing big data in string Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of EJB 3 in Action this week in the EJB and other Java EE Technologies forum!
JavaRanch » Java Forums » Java » Java in General
Bookmark "managing big data in string" Watch "managing big data in string" New topic
Author

managing big data in string

Nischal Topno
Ranch Hand

Joined: Nov 24, 2001
Posts: 45
how to manage big string data. if i use the String object i run out of memory.

Through the program i'm trying to search for a given word in a given file. The logic i've used is read the file as InputStream and covert the bytes read to String. In this string do and indexOf() of the word i'm searching.

This works fine for small files, but if the file is big it runs out of memory. i am using a P3 555 Mhz with 256 mb RAM and it runs out of memory for a 20 mb file.

Any smart way to achieve the goal?

if i've to manipulate huge chunks of String data what is the way out?

Nischal
Merrill Higginson
Ranch Hand

Joined: Feb 15, 2005
Posts: 4864
You could use a java.io.BufferedReader to read the file. It has a readLine() method that reads a line at a time into a String variable. You could then search the variable for the word, note whether it is found or not, and then read the next line into the same String variable.

This way, you're only processing one line at a time, and don't need to store the entire file in memory while you're processing it.
[ March 18, 2005: Message edited by: Merrill Higginson ]

Merrill
Consultant, Sima Solutions
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: managing big data in string
 
Similar Threads
Read a 5.6 mb text file
Total memory won't reduce after freeing memory allocated for String?
Any optimization solution?
Help!! - java.lang.OutOfMemoryError
Tomcat & Memory