aspose file tools*
The moose likes I/O and Streams and the fly likes want to search a string within a file size 2GB Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Spring in Action this week in the Spring forum!
JavaRanch » Java Forums » Java » I/O and Streams
Bookmark "want to search a string within a file size 2GB" Watch "want to search a string within a file size 2GB" New topic
Author

want to search a string within a file size 2GB

Nagendra Ramu
Greenhorn

Joined: Jul 20, 2006
Posts: 3
hi,
I have three files called first_err.log, first.log, first.out. In all the three files i want to get the exception names like NullPointerException, IndexOutofBounds, etc and count the total number of occurancy for each exceptions and put in a new file called Excepiton.txt.
We should use java version 1.4.x only.

please help in this. Thank u
Joe Ess
Bartender

Joined: Oct 29, 2001
Posts: 8927
    
    9

Welcome to the JavaRanch, Nagendra.
What in particular do you have a question about? If you are unfamiliar with working with files, The Java Tutorial has an introduction to Java IO here. If you want to get fancy, there's a number of string searching algorithms to consider once you've figured out the reading and writing part of your program.


"blabbing like a narcissistic fool with a superiority complex" ~ N.A.
[How To Ask Questions On JavaRanch]
Nagendra Ramu
Greenhorn

Joined: Jul 20, 2006
Posts: 3
hi
i want the code related to my question which i had mentioned earlier.
presently i am running batch file to search the exceptions , it takes lot of time. so, please give me some code related to this.
thank u
Joe Ess
Bartender

Joined: Oct 29, 2001
Posts: 8927
    
    9

Originally posted by Nagendra Ramu:

i want the code related to my question which i had mentioned earlier.


Ah, then you want to hire a programmer.
You see, we're all volunteers here, and while we love to help people, we insist that you Show Some Effort.
Joe Danahoo
Greenhorn

Joined: Jul 20, 2006
Posts: 1
If you haven't already, no need to hire a programmer, unless of course you do want it done professionally.

If you could give me a list of all the expceptions you want to parse from the files, i could code up a draft and see if it fits your needs...

Edit:

Forgive me if it doesnt work, and let me know, ill fix it.
Also, i am a beginner, just learning, so sorry if its not top-notch code



[ July 21, 2006: Message edited by: Joe Danahoo ]
[ July 21, 2006: Message edited by: Joe Danahoo ]
Scott Cook
Greenhorn

Joined: Aug 01, 2006
Posts: 14
Your code looks great, Joe, but perhaps you didn't notice that he said his files were 2G in size? BufferedReader will output each line of text to a String. The problem with that is that it goes into the String Pool, where it can never be deleted (except by exiting the program). That means 2G of memory will be taken up (minus the occasional line duplicates), just by reading in the files!

I mainly mention this because I'm having the same problem (trying to read-in and parse HUGE text files). A BufferedReader that returned type StringBuilder instead of String would be great, but I don't see it in the API.

Anyone have any more ideas?

Thanks!
Scott
[ August 02, 2006: Message edited by: Scott Cook ]
Joe Ess
Bartender

Joined: Oct 29, 2001
Posts: 8927
    
    9

Originally posted by Scott Cook:
The problem with that is that it goes into the String Pool, where it can never be deleted (except by exiting the program).


welcome to the JavaRanch, Scott
If that were true, I don't think any java program would be able to run for any amount of time without running out of memory.
Strings which are hardcoded in your program go into a literal pool and are never destroyed. Strings which are created in the course of a program's execution are elegible for garbage collection just like every other object.
[ August 02, 2006: Message edited by: Joe Ess ]
Scott Cook
Greenhorn

Joined: Aug 01, 2006
Posts: 14
Strings which are created in the course of a program's execution are elegible for garbage collection


Thanks for clearing that up! I read about String Immutability in the appendix of Head First Java and got really nervous. It sounded like a HUGE memory leak to me!(not literally, but has the same effect)

Now I feel much safer using BufferedReader.
Appu Chan
Greenhorn

Joined: Aug 29, 2002
Posts: 28
Seen another thread on this...
searching large file
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: want to search a string within a file size 2GB