This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes JDBC and the fly likes How to handle huge data using Resultset Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Databases » JDBC
Bookmark "How to handle huge data using Resultset" Watch "How to handle huge data using Resultset" New topic
Author

How to handle huge data using Resultset

yadagiri Sreepadi
Greenhorn

Joined: Jul 11, 2006
Posts: 1
Hi,
Iam developing an application where in I need to read the data from a Table and I had to populate it in the Excel sheet.
In a single stretch Iam fetching some where around 60,000 records from the table. The ResultSet is holding that much of data and then Iam writing the result set in to a File.

Is it the right way to handle the data or is there any optimal way to do this....

Please advice me....
Stan James
(instanceof Sidekick)
Ranch Hand

Joined: Jan 29, 2003
Posts: 8791
Hi, welcome to the ranch!

A ResultSet is allowed to be lazy. (I once read that they are all lazy when they can be, but can't confirm that myself.) That is, it can pull some subset of rows from the database into its own memory. When you use next() enough times to get to the end of that chunk, it gets more. That means it doesn't have to hold the entire result in memory at one time, designed just for situations like yours.

If you don't hold it all in memory either you may well come out ok. Sounds like you're writing to a file and not filling a collection with data. Have you tried it on large results and run into trouble? Or are you more concerned about scaling up in the future?

You can defeat this lazy scheme by specifying a bi-directional cursor. Make sure your default cursor is forward reading only.


A good question is never answered. It is not a bolt to be tightened into place but a seed to be planted and to bear more seed toward the hope of greening the landscape of the idea. John Ciardi
 
It is sorta covered in the JavaRanch Style Guide.
 
subject: How to handle huge data using Resultset
 
Similar Threads
ResultSet needs connection?
JDBC and non-indexed table
Null error
getting re-read error
Can i filter Result set