Win a copy of Mesos in Action this week in the Cloud/Virtualizaton forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

How to use streaming when query the database opposed to retrieve all data at once in resultset.

 
Mohit Kumar Tayal
Greenhorn
Posts: 29
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi all,

I am working with a core java application in which i am query a mysql database to retrieve the data from a table which have nearly 5000000 records when i am using the statemnt like this:

ResultSet rs = pstm.executeQuery(select_query);

where pstm is an object of prepared statement and select query is the query to retrieve the data from the database.

I am getting outofmemory exception due to limited heap size.

So i want to retrieve the result in a stream so i tried to set

pstm.setFetchSize(Integer.MIN_VALUE)

but as this method says it is for defining the number of rows to be retrieved but i want to define the number of megabytes to be retrieved because it may be my one row have vaery large data so it may be the scene that only two rows create the same problem.

So how to handle this type of scene.

Please help me on this.

Thank You.
 
Jeanne Boyarsky
author & internet detective
Marshal
Posts: 34422
347
Eclipse IDE Java VI Editor
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Mohit,
Most people use paging to specify the number of records. What type of field do you have that returns such large data? A BLOB/CLOB? If so, maybe you want to get all the other fields and the very large field one at a time?

What are you doing with megabytes of data from a single row multiple times?
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic