I hope I'm posting in the right place. Since I can't tell whether my problem is in my use of ResultSet or with the OutputStream, I'm posting in general.
This output of this code is running very slowly. There are about 10,000 records for the period in question that my ResultSet object is returning. It is then formatting and writing the results to a BufferedWriter "out".
When this code is first executed, the time it takes the cursor to move to the next row in the while loop is less than a second. It then slows down considerably as it continues to run. I just timed it at record 122 (of 10,000) and it took more than 15 seconds for the cursor to move to the next row. Thanks for helping me figure out what I'm doing wrong.
I'd be a little surprised if it's the IO, but you're flushing between each chunk, which may somewhat defeat the purpose of using a buffered writer (I'm not sure of the internals off the top of my head).
Thanks! Earnest, you are correct. It was the SENSITIVE flag. I changed it to INSENSITIVE and the report was done in about 15 seconds. 15 seconds versus a day to get through 1,000 records is pretty good.
Actually you would be still better off using a FORWARD_ONLY result set, since that's what your program actually does. It never does any scrolling (reading back old records again) so there's no point in saying it does. If you say you want a scrolling result set, the JDBC driver has to make every single record available in case you might go back and ask for it again. Whereas if you ask for a forward-only result set, the JDBC driver can happily discard a record the instant you ask for the next one. Much easier on the memory if you do that, then.
In fact that's the case for about 99% of JDBC applications in real life, I would guess, so I don't know why the tutorials persist in starting with anything but FORWARD_ONLY examples.