This week's book giveaway is in the Servlets forum.
We're giving away four copies of Murach's Java Servlets and JSP and have Joel Murach on-line!
See this thread for details.
The moose likes Java in General and the fly likes How Do I Find a Memory Leak? Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Murach's Java Servlets and JSP this week in the Servlets forum!
JavaRanch » Java Forums » Java » Java in General
Bookmark "How Do I Find a Memory Leak?" Watch "How Do I Find a Memory Leak?" New topic
Author

How Do I Find a Memory Leak?

Rachel Kozlowski
Ranch Hand

Joined: Jan 14, 2009
Posts: 40
I am getting the following:
java.lang.OutOfMemoryError: Java heap space

What should I look for? The error is not specific.
fred rosenberger
lowercase baba
Bartender

Joined: Oct 02, 2003
Posts: 11150
    
  16

you need to look for someplace where you are creating objects and never freeing them. For example, if you have a recursive algorithm, you might never be reaching your end-condition.

Are you processing an obscene amount of data?

Something you can try is putting println() statements in your code to see what it's doing. maybe there's a loop somewhere that's not exiting?

Or, there are memory profilers out there you can try. I've never used on, so can't give any specific advice, but it will monitor your memory usage and maybe clue you in to where all your memory is going.


There are only two hard things in computer science: cache invalidation, naming things, and off-by-one errors
Rachel Kozlowski
Ranch Hand

Joined: Jan 14, 2009
Posts: 40
I am processing an obscene amount of data. "Yes it is in the range of pornographic amounts..."

I have not seen this problem up until I began processing this large amount of data. I am currently setting my variables to null at certain points of my code and removing object declarations from inside For-Loops. I will let you know how I do.
Joe Ess
Bartender

Joined: Oct 29, 2001
Posts: 8836
    
    7

You can try increasing the heap size with the -Xmx command-line argument described here.
If that doesn't do the trick, you may have to moderate how much data you are processing.
There is still a chance that you have a real memory leak, and in that case, things get very difficult to figure out. Try searching for "outofmemoryerror" in our Performance Forum. There are many posts there with articles and tools to help (like this one).


"blabbing like a narcissistic fool with a superiority complex" ~ N.A.
[How To Ask Questions On JavaRanch]
Campbell Ritchie
Sheriff

Joined: Oct 13, 2005
Posts: 37900
    
  22
You have got some useful answers (thank you Fred and Joe) but that question is too difficult for us beginners Moving it.
Rachel Kozlowski
Ranch Hand

Joined: Jan 14, 2009
Posts: 40
Will it still be answered if it is moved? The last time my topic was moved, there was no more help.
Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18541
    
    8

Rachel Kozlowski wrote:I am currently setting my variables to null at certain points of my code and removing object declarations from inside For-Loops.
But this sort of thing usually amounts to rearranging the deck chairs on the Titanic. More likely your problem is that you are creating a large number of objects and adding them to some kind of collection. Or generating enormous strings. Or something else which involves storing all your data in memory.

If you don't see anything like that in your code, then you can run a profiler on your code and see which objects you're storing a large number of.
Rachel Kozlowski
Ranch Hand

Joined: Jan 14, 2009
Posts: 40
Ok, you are right it did not work to set objects to null. (I am a beginner, so bear with me.)

How does one run a profiler? I am using eclipse as my development too. I need step by step instructions of how to determine what is holding on to the large amount of data.
fred rosenberger
lowercase baba
Bartender

Joined: Oct 02, 2003
Posts: 11150
    
  16

How long is your code? if it's not too long, you can post it and it may be obvious to someone - but don't expect anyone to go through hundreds of lines of code. If it's under a hundred or so lines, someone may look at it.

Can you give us an idea what exactly your doing? a high-level overview of your algorithm? The more you can tell us, the more focused advice folks may be able to give you.
Rachel Kozlowski
Ranch Hand

Joined: Jan 14, 2009
Posts: 40
Fred, you are very sweet, but unfortunately my code is long and rambling. I would never expect you to look through that. I will get back with you on the logic.
Rachel Kozlowski
Ranch Hand

Joined: Jan 14, 2009
Posts: 40
I am attempting to populate a report using a classes from Jasper Reports (3rd party software class).
The reportParametersMap is a Hashmap that is populated with data returned from a database:
and the reportDesignObj is basically used to format the Report, by putting a Title on the page and organize the report data on the report and then put footers at the bottom of the page.

The last line of code that the debugger hits before it goes off into the memory leak is :


The area of code that it is found in below is small. Do you see anything obvious?

Lucas Franceschi
Ranch Hand

Joined: Nov 10, 2008
Posts: 106

you know, i may be talking non-sense, but there's a "runtime" object, that represents the current runtime of your process, this object have the "gc()" method, gc stands for garbage collector.

i'm not sure if java collects garbage for itself or if you need to call the gc() method, but it'll be a good practice to take a deep look at this.

also, make sure your variables are on the lesser scope that you can let them, because you would end up with a variable defined since the code start and only used inside one try, what would make that variable to be garbage outside the try, so define it only on the try.

again, i'm not completely sure about this, its just tips, things for you to give a look around the internet


regards


Lucas Franceschi
Software Developer for SGI Sistemas, lukas1596@gmail.com
fred rosenberger
lowercase baba
Bartender

Joined: Oct 02, 2003
Posts: 11150
    
  16

you do not need to call the garbage collection method - and I believe doing so can actually slow down your program.

Have you tried increasing your heap size? On the command line, you can type

java -X

and get the non-standard flags. You'll see

-Xms<size> set initial Java heap size
-Xmx<size> set maximum Java heap size

Try making both of these large... I don't know what the default is, or how much memory your box has, but this may help. Maybe something like

java -Xmx512m -Xms512m

if you have that much memory. This may also just be a band aid... if your data size keeps increasing, you'll eventually hit this limit and be back at square 1. Of course, this may have repercussions on other applications on the box, so take that under advisement.

Campbell Ritchie
Sheriff

Joined: Oct 13, 2005
Posts: 37900
    
  22
There is usually no need to call the System.gc() or similar methods. Java will run garbage collection automatically.

And you still seem to be getting answers.
Lucas Franceschi
Ranch Hand

Joined: Nov 10, 2008
Posts: 106

aha! you're working with jasper too!


look at this:
i am working with jasper reports,
today I got the same problem you got, out of memory.

solution?


maybe you'll need to import some of these:


virtualization is a jasper prevention for your (and mine) case, out of memory.
that's because jasper builds the report on memory, then puts it into a file.

with virtualization, you tell jasper to create some temporary files, that would hold parts of the report, 'till its completed, and merges into one only report.
the "50" in the virtualizer, is the max number of virtual files you want to be crated.

with this property, you can give goodbye for your problem

so, its jasper's problem, not your code's.

(if you specified that you work with jasper in the first post, i could have helped you since the start.)

Lucas Franceschi
Ranch Hand

Joined: Nov 10, 2008
Posts: 106

you know, i'm actually testing this property now, and its taking times like 2, or 3 hours to generate my "memory breaking" report, so that if you can test this property there, it would be a great help.

so i insist to say that i know it works, but I never tested it, i'm testing and its taking long time for me to get any result, i'm generating giant, ultrageous big reports, one for each result on database, that means kinda 100000 giant reports being generated once, only to make sure it will not be out of memory, under any circunstances.

also, the last line you execute before the memory leak will be the line that calls the report filling, because the memory leak occurs inside jasper's engine, and only the property I've told you can be a hope of avoiding it.
fred rosenberger
lowercase baba
Bartender

Joined: Oct 02, 2003
Posts: 11150
    
  16

Lucas Franceschi wrote:...to make sure it will not be out of memory, under any circunstances.

That's a pretty bold statement. Reminds me of the time some guy allegedly said "Why would anyone EVER need more than 640k of memory?"
Lucas Franceschi
Ranch Hand

Joined: Nov 10, 2008
Posts: 106

fred rosenberger wrote:
Lucas Franceschi wrote:...to make sure it will not be out of memory, under any circunstances.

That's a pretty bold statement. Reminds me of the time some guy allegedly said "Why would anyone EVER need more than 640k of memory?"

actually, the idea is:
i'm useng a method that prevents java to get out of memory, what should I do to test it?
push it to the limits! i mean, its clear that generating almost millions of reports, one bigger than another, will leak the memory, so if I dont get a "out of memory" exception, the property works fine.

its about testing, because the costumer will not have the conscience to know that the program "crashed" because of the "memory leak", no, what will it say? "your program crashed."

thats what i'm trying to prevent, i'm pushing jasper to its limits, so that i can make sure that anything the user does will not be mentioned as a "crash in my program."

we need to remember, when programming for sell, that our users may be an expert about computer, and may be a complete ignorant about computer, that's why we need a lot of testing.


in about 10 years, we will repeat your quote, but like this "Why would anyone EVER need more than 100Gb of memory?"
or "can you believe that some years ago, people needed to use a strange device, they called mouse, because they could not touch the screen? weird isn't it?"


I think my sons will get curious when i start to talk to them how computers needed to have keyboards in the "old" times.
Rachel Kozlowski
Ranch Hand

Joined: Jan 14, 2009
Posts: 40
look at this:
i am working with jasper reports,
today I got the same problem you got, out of memory.

solution?
view plaincopy to clipboardprint?
JRFileVirtualizer virtualizador = new JRFileVirtualizer(50);
reportParametersMap.put(JRParameter.REPORT_VIRTUALIZER, virtualizador);
JRFileVirtualizer virtualizador = new JRFileVirtualizer(50);
reportParametersMap.put(JRParameter.REPORT_VIRTUALIZER, virtualizador);


Lucas, I am still having the same problem. Where in the code would I put this?
Lucas Franceschi
Ranch Hand

Joined: Nov 10, 2008
Posts: 106

Rachel Kozlowski wrote:Lucas, I am still having the same problem. Where in the code would I put this?


well, you need to put it on your parameters map, in the same place where you define the other parameters.

try making some tests, take a look at jasper's documentation and try to make it work, but dont forget i'm having the same problem, so, post here if you get any results.


see you tomorrow guys! i'm going home.
Rachel Kozlowski
Ranch Hand

Joined: Jan 14, 2009
Posts: 40
I did increase the heap size yesterday. It is at 1024.

If I run the complied code outside of eclipse it takes a number of minutes for the page to display, but when I run it in eclipse I get the error. Regardless, I need to solve this issue before I get even more data.
Lucas Franceschi
Ranch Hand

Joined: Nov 10, 2008
Posts: 106

so, setting the property didn't work?
i dont know what to do, it does not work here too.

i'm totally lost now, increasing java heap space would not help, because i would need to change jhs in everyone that uses my program.

I dont know what to do, sorry rachel.
Rachel Kozlowski
Ranch Hand

Joined: Jan 14, 2009
Posts: 40
I may be doing something incorrect. As another person from this forum noted Jasper Reports put out demo code and I looked at it and the syntax is slightly different from yours.
The snippet from the sample is different. Notice that they have it writing to a tmp file and also notice that the virtualizer is included in the fillReport syntax. I am going to try this next. You should also give it a shot.



The entire Sample is below:
Lucas Franceschi
Ranch Hand

Joined: Nov 10, 2008
Posts: 106

Rachel Kozlowski wrote:I may be doing something incorrect. As another person from this forum noted Jasper Reports put out demo code and I looked at it and the syntax is slightly different from yours.
The snippet from the sample is different. Notice that they have it writing to a tmp file and also notice that the virtualizer is included in the fillReport syntax. I am going to try this next. You should also give it a shot.



The entire Sample is below:
...


well, what happens is that the virtualizer will be passed, in this snipped that you have pasted, by the only parameter in the report, that means, the "fillreport" method needs filename, datasource, and parameters. cirtualizer will be a parameter in this case.

also, have you already tested it? if so, what was the results?

it have been some time since the last time I looked at this, so maybe you have already found a solution for this case, if so, please tell us.

try looking for JRFileVirtualizer on google, find the javadoc, and take a look at the constructors, then you'll note that the second argument is optional, and it will be on the local directory if nothing is passed.

hope you have already solved it, but anyway if you don't, try looking a bit more.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: How Do I Find a Memory Leak?
 
Similar Threads
Cant use object from another class
Posting to bump up my count...
Text box value is not being completely filled with request attribute
WA #1.....word association
The x900 effect ?