• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

OutOfMemoryError while accessing Postgresql via JDBC

 
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi all,

I'm relatively new to Java and have the following problem with accessing a Postgresql database.

I connect to the database and do a select, then iterate over the result set and build objects from the data. This works without a problem when the select returns only a few rows, but when I get a really large result (ca. 380,000 rows) I always get an OutOfMemoryError.

I googled some and found that I should increase the heap size, but either this does not help or I am doing it wrong.

The relevant part of my code looks like this:



Regarding the heap size: I am using NetBeans 6.5 under SuSE Linux. I edited the netbeans.conf so that the memory related default options look like this:

-J-Xss2m -J-Xms256m -J-XX:PermSize=128m -J-XX:MaxPermSize=1024m

What am I doing wrong? There has to be an easy way to do this.


[Sorry, if this appears twice, I had some problems submitting it.]
 
Ranch Hand
Posts: 2908
1
Spring Java Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Is it a web application you are working upon, If yes then increases the memory of servlet container(server).

Many times , its happens that, we kept some very important resources open, like database connection, some stream, etc. My advise ll be, just recheck whether you have closed connection properly or put close connection in "finally" block.

Nb6.5 provides profiler , which helps you to get the picture of which objects are occupying more memory and opened.

Lastly, you can search this forum for same problem.
 
Alexander Koenig
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Sagar Rohankar wrote:Is it a web application you are working upon, If yes then increases the memory of servlet container(server).



No, just a simple stand alone app.

Many times , its happens that, we kept some very important resources open, like database connection, some stream, etc. My advise ll be, just recheck whether you have closed connection properly or put close connection in "finally" block.



I don't think that i forgot to close anything. The problem is that I don't even get past the executeQuery before the error.

Nb6.5 provides profiler , which helps you to get the picture of which objects are occupying more memory and opened.



I got the profiler working once (somehow it doesn't work anymore) and saw that a huge amount of byte objects were created (about 1.5 million, if I remember correctly). I assume that these belong to the result set.

 
Alexander Koenig
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Sagar Rohankar wrote:Nb6.5 provides profiler , which helps you to get the picture of which objects are occupying more memory and opened.



I found the profiler output and these are the crucial elements, I think:

Class Name--------------- Instances ------------------ Size
byte[] ------------------------ 1418758 (85%)---------- 44438400 (82%)
byte[][] ------------------------ 236151 (14%) ------------7556876 (14%)
 
Sagar Rohankar
Ranch Hand
Posts: 2908
1
Spring Java Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Alexander Koenig wrote:
Regarding the heap size: I am using NetBeans 6.5 under SuSE Linux. I edited the netbeans.conf so that the memory related default options look like this:

-J-Xss2m -J-Xms256m -J-XX:PermSize=128m -J-XX:MaxPermSize=1024m



One doubt, the starting memory is 256 MB, that's OK, but what about the 'maximum' memory assigned, I thinks it's JVM option is , like this.

-Xmx 512m
 
Alexander Koenig
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Sagar Rohankar wrote:
One doubt, the starting memory is 256 MB, that's OK, but what about the 'maximum' memory assigned, I thinks it's JVM option is , like this.

-Xmx 512m




Well, when I add this to the netbeans.conf netbeans_default_options
-J-Xmx=512m
NetBeans won't start anymore.
 
Sagar Rohankar
Ranch Hand
Posts: 2908
1
Spring Java Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Alexander Koenig wrote:

Sagar Rohankar wrote:
One doubt, the starting memory is 256 MB, that's OK, but what about the 'maximum' memory assigned, I thinks it's JVM option is , like this.

-Xmx 512m




Well, when I add this to the netbeans.conf netbeans_default_options
-J-Xmx=512m
NetBeans won't start anymore.



I suggest max memory from this wiki entry. Just check out the log files of NB for the reason behind crash.
 
Alexander Koenig
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Sagar Rohankar wrote:

Alexander Koenig wrote:

Sagar Rohankar wrote:
One doubt, the starting memory is 256 MB, that's OK, but what about the 'maximum' memory assigned, I thinks it's JVM option is , like this.

-Xmx 512m




Well, when I add this to the netbeans.conf netbeans_default_options

NetBeans won't start anymore.



I suggest max memory from this wiki entry. Just check out the log files of NB for the reason behind crash.



Okay. I wrote -J-Xmx=512m , when I should have been writing -J-Xmx512m .
I increased it to 800m which wasn't enough. But when I try to set it higher, the ide refuses to start again. I think there is some kind of connection between the settings for Xmx and XX:MaxPermSize. Is there some site where I can look these up?

Also: I couldn't find an error log for netbeans, only a generic messages.log which does not log anything if netbeans doesn't start.

I get a nagging feeling that I won't be able to increase the heap size enough to be able to perform my select. Isn't there some way around this problem? I did something similar in Perl and didn't get any problems on my machine. Is there really that much overhead when using Java?
 
Consider Paul's rocket mass heater.
reply
    Bookmark Topic Watch Topic
  • New Topic