File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Writing data into *.txt file using java

 
Maria Smith
Ranch Hand
Posts: 40
  • 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi I am new to java. What is the efficient way of writing data into *.txt file using java
I need to dump data from one of the oracle table to *.txt file using java.
Right now I am using
File outputFile = new File(fileName);
PrintWriter out =new PrintWriter(new BufferedWriter (new FileWriter(outputFile)));
using out.write method to write the data into text file.

Gurus please let me know if there is any other way to acheive same result much faster.
Because right now it takes about 30 minutes to create file for about 300,000 records.

Thanks,
Maria
 
Ernest Friedman-Hill
author and iconoclast
Marshal
Pie
Posts: 24204
34
Chrome Eclipse IDE Mac OS X
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If you're just using out.write(), then you don't need the PrintWriter; but that's probably not the issue.

Usually the major performance issues in this kind of code come from the unintentional creation of large numbers of little Strings. We'd have to see some of the actual writing code to understand where the problem might lie there.

There's also the simple issue that it might take 30 minutes to store that much information on your system -- if the disk is a remote share of some kind, for example.
 
Maria Smith
Ranch Hand
Posts: 40
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I just get the data like this from the result set and dump it in a file

while(rs.next()
{
out.write(rs.getString(1).trim()+',');
out.write(rs.getString(2).trim()+',');
out.write(rs.getString(3).trim()+',');
out.write(rs.getString(4).trim()+',');
out.write(rs.getString(5).trim()+',');
out.write('\n');
out.flush();
}
Thanks
 
Ernest Friedman-Hill
author and iconoclast
Marshal
Pie
Posts: 24204
34
Chrome Eclipse IDE Mac OS X
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
OK, well: each "trim()" call creates a String, and each "+" creates a String and a StringBuilder. Also, flush() is a relatively expensive call. Replicating "trim()" without creating a String would be messy, but you might see a noticeable difference just by doing this:

while(rs.next()
{
out.write(rs.getString(1).trim());
out.write(',');
out.write(rs.getString(2).trim());
out.write(',');
out.write(rs.getString(3).trim());
out.write(',');
out.write(rs.getString(4).trim());
out.write(',');
out.write(rs.getString(5).trim());
out.write('\n');
}
out.flush();

On the other hand, it may be that file I/O isn't the limiting factor here -- database communication may be.
 
Jim Yingst
Wanderer
Sheriff
Posts: 18671
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If that doesn't work, then as an experiment, see how long it takes just to iterate through the ResultSet, without writing anything:

I suspect you may find that it takes just as long - which would indicate that the Java code isn't the problem; it's the connection to the database. If that's the case, then you'll have to look elsewhere for improvements. Maybe the SQL query can be written more efficently, or maybe you can relocate the code to run on the same machine as the DB (assuming it's not doing that already). Or maybe there are advanced DB tuning options. Anyway, find out where the problem really is before you spend too much time trying to fix it.
[ January 22, 2007: Message edited by: Jim Yingst ]
 
Maria Smith
Ranch Hand
Posts: 40
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you so much for your reply. I am going to change the code to to see if i get any better performance.
But in general for a query about million rows I get result set back by 12 minutes. Then it takes another 50 to 60 minutes to dump the data in a file.

Maria
 
Consider Paul's rocket mass heater.
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic