aspose file tools*
The moose likes JDBC and the fly likes How to use streaming when query the database opposed to retrieve all data at once in resultset. Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Databases » JDBC
Bookmark "How to use streaming when query the database opposed to retrieve all data at once in resultset." Watch "How to use streaming when query the database opposed to retrieve all data at once in resultset." New topic
Author

How to use streaming when query the database opposed to retrieve all data at once in resultset.

Mohit Kumar Tayal
Greenhorn

Joined: Aug 27, 2008
Posts: 29
Hi all,

I am working with a core java application in which i am query a mysql database to retrieve the data from a table which have nearly 5000000 records when i am using the statemnt like this:

ResultSet rs = pstm.executeQuery(select_query);

where pstm is an object of prepared statement and select query is the query to retrieve the data from the database.

I am getting outofmemory exception due to limited heap size.

So i want to retrieve the result in a stream so i tried to set

pstm.setFetchSize(Integer.MIN_VALUE)

but as this method says it is for defining the number of rows to be retrieved but i want to define the number of megabytes to be retrieved because it may be my one row have vaery large data so it may be the scene that only two rows create the same problem.

So how to handle this type of scene.

Please help me on this.

Thank You.
Jeanne Boyarsky
internet detective
Marshal

Joined: May 26, 2003
Posts: 30309
    
150

Mohit,
Most people use paging to specify the number of records. What type of field do you have that returns such large data? A BLOB/CLOB? If so, maybe you want to get all the other fields and the very large field one at a time?

What are you doing with megabytes of data from a single row multiple times?


[Blog] [JavaRanch FAQ] [How To Ask Questions The Smart Way] [Book Promos]
Blogging on Certs: SCEA Part 1, Part 2 & 3, Core Spring 3, OCAJP, OCPJP beta, TOGAF part 1 and part 2
 
Consider Paul's rocket mass heater.
 
subject: How to use streaming when query the database opposed to retrieve all data at once in resultset.