files using threads

[Anil kumar]

we are extracting data from the database and then it is written into 10 different flat files. the data is very huge and it is taking hours to write one file, in this way it takes many hours to write data into all 10 files.

so i am thinking to use threads while using the write file to reduce number of hours it is taking. but in threads it writes to only one file at a time, and behaving as normal program writing one file at a time.

can some one suggest solution for this problem, or there any ways to deal with this.

[Joe Ess]

Welcome to the JavaRanch, Anand. Please change your displayed name to conform to our
naming policy. In short, your name must be a first name and last name, separated by a space and not obviously fake.
Threads are useful for doing several things at once. They do not make doing an intensive process faster. There's several things that can impact the performance of database access and file IO. What strategy will get you the greatest payoff depends on your particular situation. There's no one answer. You need to benchmark the current performance, then try different approaches to see what works. Start with the IO chapter of Java Platform Performance. After you tune the file side, read through Best Practices to improve JDBC Performance. Again, there's multiple things to try and measure. Good luck!

[Anil kumar]

Thanks Joe for Ur suggestion,

I want to use threads so that I can write to several files at a time, is it possible to write number of file at a time using threads.

[David Harkness]

Yes, using Java's original I/O you can write to multiple files simultaneously by using multiple threads. With NIO, you can accomplish the same thing with a single thread.

And to echo Joe's point, make sure you're using buffered I/O.

What did you try (show some code) and how did it fail to work? If you understand creating and using multiple threads and writing to files, combining the two is pretty straight-forward. Where are you getting stuck?