This week's book giveaway is in the OO, Patterns, UML and Refactoring forum. We're giving away four copies of Refactoring for Software Design Smells: Managing Technical Debt and have Girish Suryanarayana, Ganesh Samarthyam & Tushar Sharma on-line! See this thread for details.
Try opening the file in a text editor. Are the 100000 lines actually on separate lines? Because if they all appear on one very long line, readLine() will try to read the whole file into memory at once. Which may be what's happening to you here.
Joined: May 29, 2008
Thanks a lot Mike for your reply. What you said was correct, while entering i dint use the newline. But now its fine. I have one more thing to ask you. Can you please say something about file encryption. Like i have said that the file with 100,0000 lines is a text file i want to encrypt that file to hide data and i use a particular key then only i should be able to view the data. If this is possible then how can i do it.
Thanks and Regards
Joined: Mar 05, 2008
Well, I'm sure it's possible, but offhand I don't know the best way to do it. It's probably better if you address this question to everyone, not just me.
One problem is that most encryption schemes basically create binary files, which means the idea of reading a single line will become difficult, as newline characters may appear in the encrypted data, but that has nothing to do with where the line breaks are in the unencrypted data. Considering the size of your data, you may want to split the contents up into a set of smaller files, each small enough to load completely into memory without an OutOfMemoryError. For example, you could have 10000 files of 1000 lines each, each encrypted individually. (I just made those numbers up - you'd have to experiment to see what works for you.) And you need to think about what types of operations you need to be able to perform. Do you want to be able to look up individual records, without necessarily loading every record in the file? In that case you're probably better off with many small files. Or maybe using a database, with each record stored in a BLOB. The best answer will probably depend on how you need to be able to use the data; I'm just tossing out some ideas about things you might want to think about. And I bet there are others here with more experience in this particular area. I hope that helps.