This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes JDBC and the fly likes OutOfMemoryError while accessing Postgresql via JDBC Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Databases » JDBC
Bookmark "OutOfMemoryError while accessing Postgresql via JDBC" Watch "OutOfMemoryError while accessing Postgresql via JDBC" New topic
Author

OutOfMemoryError while accessing Postgresql via JDBC

Alexander Koenig
Greenhorn

Joined: Feb 05, 2009
Posts: 5
Hi all,

I'm relatively new to Java and have the following problem with accessing a Postgresql database.

I connect to the database and do a select, then iterate over the result set and build objects from the data. This works without a problem when the select returns only a few rows, but when I get a really large result (ca. 380,000 rows) I always get an OutOfMemoryError.

I googled some and found that I should increase the heap size, but either this does not help or I am doing it wrong.

The relevant part of my code looks like this:



Regarding the heap size: I am using NetBeans 6.5 under SuSE Linux. I edited the netbeans.conf so that the memory related default options look like this:

-J-Xss2m -J-Xms256m -J-XX:PermSize=128m -J-XX:MaxPermSize=1024m

What am I doing wrong? There has to be an easy way to do this.


[Sorry, if this appears twice, I had some problems submitting it.]
Sagar Rohankar
Ranch Hand

Joined: Feb 19, 2008
Posts: 2902
    
    1

Is it a web application you are working upon, If yes then increases the memory of servlet container(server).

Many times , its happens that, we kept some very important resources open, like database connection, some stream, etc. My advise ll be, just recheck whether you have closed connection properly or put close connection in "finally" block.

Nb6.5 provides profiler , which helps you to get the picture of which objects are occupying more memory and opened.

Lastly, you can search this forum for same problem.


[LEARNING bLOG] | [Freelance Web Designer] | [and "Rohan" is part of my surname]
Alexander Koenig
Greenhorn

Joined: Feb 05, 2009
Posts: 5
Sagar Rohankar wrote:Is it a web application you are working upon, If yes then increases the memory of servlet container(server).


No, just a simple stand alone app.

Many times , its happens that, we kept some very important resources open, like database connection, some stream, etc. My advise ll be, just recheck whether you have closed connection properly or put close connection in "finally" block.


I don't think that i forgot to close anything. The problem is that I don't even get past the executeQuery before the error.

Nb6.5 provides profiler , which helps you to get the picture of which objects are occupying more memory and opened.


I got the profiler working once (somehow it doesn't work anymore) and saw that a huge amount of byte objects were created (about 1.5 million, if I remember correctly). I assume that these belong to the result set.

Alexander Koenig
Greenhorn

Joined: Feb 05, 2009
Posts: 5
Sagar Rohankar wrote:Nb6.5 provides profiler , which helps you to get the picture of which objects are occupying more memory and opened.


I found the profiler output and these are the crucial elements, I think:

Class Name--------------- Instances ------------------ Size
byte[] ------------------------ 1418758 (85%)---------- 44438400 (82%)
byte[][] ------------------------ 236151 (14%) ------------7556876 (14%)
Sagar Rohankar
Ranch Hand

Joined: Feb 19, 2008
Posts: 2902
    
    1

Alexander Koenig wrote:
Regarding the heap size: I am using NetBeans 6.5 under SuSE Linux. I edited the netbeans.conf so that the memory related default options look like this:

-J-Xss2m -J-Xms256m -J-XX:PermSize=128m -J-XX:MaxPermSize=1024m


One doubt, the starting memory is 256 MB, that's OK, but what about the 'maximum' memory assigned, I thinks it's JVM option is , like this.

-Xmx 512m
Alexander Koenig
Greenhorn

Joined: Feb 05, 2009
Posts: 5
Sagar Rohankar wrote:
One doubt, the starting memory is 256 MB, that's OK, but what about the 'maximum' memory assigned, I thinks it's JVM option is , like this.

-Xmx 512m



Well, when I add this to the netbeans.conf netbeans_default_options
-J-Xmx=512m
NetBeans won't start anymore.
Sagar Rohankar
Ranch Hand

Joined: Feb 19, 2008
Posts: 2902
    
    1

Alexander Koenig wrote:
Sagar Rohankar wrote:
One doubt, the starting memory is 256 MB, that's OK, but what about the 'maximum' memory assigned, I thinks it's JVM option is , like this.

-Xmx 512m



Well, when I add this to the netbeans.conf netbeans_default_options
-J-Xmx=512m
NetBeans won't start anymore.


I suggest max memory from this wiki entry. Just check out the log files of NB for the reason behind crash.
Alexander Koenig
Greenhorn

Joined: Feb 05, 2009
Posts: 5
Sagar Rohankar wrote:
Alexander Koenig wrote:
Sagar Rohankar wrote:
One doubt, the starting memory is 256 MB, that's OK, but what about the 'maximum' memory assigned, I thinks it's JVM option is , like this.

-Xmx 512m



Well, when I add this to the netbeans.conf netbeans_default_options

NetBeans won't start anymore.


I suggest max memory from this wiki entry. Just check out the log files of NB for the reason behind crash.


Okay. I wrote -J-Xmx=512m , when I should have been writing -J-Xmx512m .
I increased it to 800m which wasn't enough. But when I try to set it higher, the ide refuses to start again. I think there is some kind of connection between the settings for Xmx and XX:MaxPermSize. Is there some site where I can look these up?

Also: I couldn't find an error log for netbeans, only a generic messages.log which does not log anything if netbeans doesn't start.

I get a nagging feeling that I won't be able to increase the heap size enough to be able to perform my select. Isn't there some way around this problem? I did something similar in Perl and didn't get any problems on my machine. Is there really that much overhead when using Java?
 
 
subject: OutOfMemoryError while accessing Postgresql via JDBC
 
Similar Threads
sql insert statement giving sqlcode: -206 sqlstate: 42703
values are repeated in pdf
Retrieve all table names from a DB for a user
Inserting single column values of database table into multiple columns of jtable and do comparison
Using ResultSet