Win a copy of Think Java: How to Think Like a Computer Scientist this week in the Java in General forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

java.lang.OutOfMemory error while fetching data from large tables

 
chaitanya Komaravalli
Greenhorn
Posts: 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,
i am trying to fetch data using "executeQuery()" into a ResultSet from the database. But since the data in that table is large, i am recieving "java.lang.OutOfMemory" Error. So, to resolve that, i have used "setMaxRows()" for my statement object. This resolved the error but i don't recieve the entire data. If i call "executeQuery()" again, i recieve the same data. I don't even know a filtering criterion where by i can filter the data for each "executeQuery()"..
How can i resolve this problem

Thanx in advance
--Chaitanya
 
Dana Hanna
Ranch Hand
Posts: 227
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If the table is so large - what do you expect your program to do with it? What is the purpose of this app?
 
Pravin Panicker
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You are selecting huge amount of data at one go. So the exception is expected. But will you be using all of them?
you could resort to some sort of pagination , if your application permits, and fetch
fewer records as and when needed.
 
Jamie Robertson
Ranch Hand
Posts: 1879
MySQL Database Suse
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Pravin Panicker:
You are selecting huge amount of data at one go. So the exception is expected. But will you be using all of them?
you could resort to some sort of pagination , if your application permits, and fetch
fewer records as and when needed.

I iterate through hundreds of thousands of rows without any problems, even when my code was very...ummm...inefficient ( new to programming ). Iterating through a resultset itself is not grounds for getting out of memory errors. The problem is when you accidentally hold that record in memory. You can avoid holding the data in memory by using a NON-scrollable resultset, not adding each row to a collection, and not using a rowset. If you are holding the records in memory ( intentional or unintentional ) then you can expect at some point for your computer or JVM to run out of memory.
Jamie
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic