karthick sambanghi wrote:I want to extract XML datatype from oracle database & write it to a file.
Well, that right there strikes me as a complete waste of time, but there's no accounting for some clients.
the issue is it is huge volume database. At present i am extracting 1 million records & writing it to a file within 17 minutes time.
But the client requirement is to extract the data & write it to a file within 5 minutes for 2 million records....
First off: is this going out to one file or many? If only one, then threading is unlikely to make much difference.
I have created one Manager thread & worker thread. The input size of thread is "10" for 1 million records so that it will create some "100" jobs with batchsize=10000
Is there any other way to increase my performance ?
It sounds to me like you're diving into the mechanics of how you're going to do this before you've actually worked out
what you need to do.
Where is the delay? Is it getting this stuff
from the database? Or is it writing it out? Have you, for example, timed a
single Thread doing each thing in isolation? You could time the latter by just bashing out 2 million arbitrary lines of roughly the right size to a file (or many files, if that's what you're going to need), and forget about the db altogether. I wouldn't be at all surprised if you might be looking at a couple of minutes right there.
And, like I say, what you do will be very dependant on whether you're writing out to a single file or many of them - and indeed, whether you're getting all this stuff from a single database record or many of them.
Oh, and one last thing: I hope you're buffering your output, because that alone could have a major bearing on how long it takes.
Winston