This week's book giveaway is in the Servlets forum.
We're giving away four copies of Murach's Java Servlets and JSP and have Joel Murach on-line!
See this thread for details.
The moose likes I/O and Streams and the fly likes Writing data into *.txt file using java Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Murach's Java Servlets and JSP this week in the Servlets forum!
JavaRanch » Java Forums » Java » I/O and Streams
Bookmark "Writing data into *.txt file using java" Watch "Writing data into *.txt file using java" New topic
Author

Writing data into *.txt file using java

Maria Smith
Ranch Hand

Joined: Apr 01, 2004
Posts: 40
Hi I am new to java. What is the efficient way of writing data into *.txt file using java
I need to dump data from one of the oracle table to *.txt file using java.
Right now I am using
File outputFile = new File(fileName);
PrintWriter out =new PrintWriter(new BufferedWriter (new FileWriter(outputFile)));
using out.write method to write the data into text file.

Gurus please let me know if there is any other way to acheive same result much faster.
Because right now it takes about 30 minutes to create file for about 300,000 records.

Thanks,
Maria
Ernest Friedman-Hill
author and iconoclast
Marshal

Joined: Jul 08, 2003
Posts: 24183
    
  34

If you're just using out.write(), then you don't need the PrintWriter; but that's probably not the issue.

Usually the major performance issues in this kind of code come from the unintentional creation of large numbers of little Strings. We'd have to see some of the actual writing code to understand where the problem might lie there.

There's also the simple issue that it might take 30 minutes to store that much information on your system -- if the disk is a remote share of some kind, for example.


[Jess in Action][AskingGoodQuestions]
Maria Smith
Ranch Hand

Joined: Apr 01, 2004
Posts: 40
I just get the data like this from the result set and dump it in a file

while(rs.next()
{
out.write(rs.getString(1).trim()+',');
out.write(rs.getString(2).trim()+',');
out.write(rs.getString(3).trim()+',');
out.write(rs.getString(4).trim()+',');
out.write(rs.getString(5).trim()+',');
out.write('\n');
out.flush();
}
Thanks
Ernest Friedman-Hill
author and iconoclast
Marshal

Joined: Jul 08, 2003
Posts: 24183
    
  34

OK, well: each "trim()" call creates a String, and each "+" creates a String and a StringBuilder. Also, flush() is a relatively expensive call. Replicating "trim()" without creating a String would be messy, but you might see a noticeable difference just by doing this:

while(rs.next()
{
out.write(rs.getString(1).trim());
out.write(',');
out.write(rs.getString(2).trim());
out.write(',');
out.write(rs.getString(3).trim());
out.write(',');
out.write(rs.getString(4).trim());
out.write(',');
out.write(rs.getString(5).trim());
out.write('\n');
}
out.flush();

On the other hand, it may be that file I/O isn't the limiting factor here -- database communication may be.
Jim Yingst
Wanderer
Sheriff

Joined: Jan 30, 2000
Posts: 18671
If that doesn't work, then as an experiment, see how long it takes just to iterate through the ResultSet, without writing anything:

I suspect you may find that it takes just as long - which would indicate that the Java code isn't the problem; it's the connection to the database. If that's the case, then you'll have to look elsewhere for improvements. Maybe the SQL query can be written more efficently, or maybe you can relocate the code to run on the same machine as the DB (assuming it's not doing that already). Or maybe there are advanced DB tuning options. Anyway, find out where the problem really is before you spend too much time trying to fix it.
[ January 22, 2007: Message edited by: Jim Yingst ]

"I'm not back." - Bill Harding, Twister
Maria Smith
Ranch Hand

Joined: Apr 01, 2004
Posts: 40
Thank you so much for your reply. I am going to change the code to to see if i get any better performance.
But in general for a query about million rows I get result set back by 12 minutes. Then it takes another 50 to 60 minutes to dump the data in a file.

Maria
 
It is sorta covered in the JavaRanch Style Guide.
 
subject: Writing data into *.txt file using java
 
Similar Threads
Question about Tree Node --Word Tree
Writing to a file
Need help outputting to a text file...
Writing an Empty Line in a File....
Reading from file