How Do I Find a Memory Leak?

[Rachel Kozlowski]

I am getting the following:
java.lang.OutOfMemoryError: Java heap space

What should I look for? The error is not specific.

[fred rosenberger]

you need to look for someplace where you are creating objects and never freeing them. For example, if you have a recursive algorithm, you might never be reaching your end-condition.

Are you processing an obscene amount of data?

Something you can try is putting println() statements in your code to see what it's doing. maybe there's a loop somewhere that's not exiting?

Or, there are memory profilers out there you can try. I've never used on, so can't give any specific advice, but it will monitor your memory usage and maybe clue you in to where all your memory is going.

[Rachel Kozlowski]

I am processing an obscene amount of data. "Yes it is in the range of pornographic amounts..."

I have not seen this problem up until I began processing this large amount of data. I am currently setting my variables to null at certain points of my code and removing object declarations from inside For-Loops. I will let you know how I do.

[Joe Ess]

You can try increasing the heap size with the -Xmx command-line argument described here.
If that doesn't do the trick, you may have to moderate how much data you are processing.
There is still a chance that you have a real memory leak, and in that case, things get very difficult to figure out. Try searching for "outofmemoryerror" in our Performance Forum. There are many posts there with articles and tools to help (like this one).

[Campbell Ritchie]

You have got some useful answers (thank you Fred and Joe) but that question is too difficult for us beginners Moving it.

[Rachel Kozlowski]

Will it still be answered if it is moved? The last time my topic was moved, there was no more help.

[Paul Clapham]

Rachel Kozlowski wrote:I am currently setting my variables to null at certain points of my code and removing object declarations from inside For-Loops.

But this sort of thing usually amounts to rearranging the deck chairs on the Titanic. More likely your problem is that you are creating a large number of objects and adding them to some kind of collection. Or generating enormous strings. Or something else which involves storing all your data in memory.

If you don't see anything like that in your code, then you can run a profiler on your code and see which objects you're storing a large number of.

[Rachel Kozlowski]

Ok, you are right it did not work to set objects to null. (I am a beginner, so bear with me.)

How does one run a profiler? I am using eclipse as my development too. I need step by step instructions of how to determine what is holding on to the large amount of data.

[fred rosenberger]

How long is your code? if it's not too long, you can post it and it may be obvious to someone - but don't expect anyone to go through hundreds of lines of code. If it's under a hundred or so lines, someone may look at it.

Can you give us an idea what exactly your doing? a high-level overview of your algorithm? The more you can tell us, the more focused advice folks may be able to give you.

[Rachel Kozlowski]

Fred, you are very sweet, but unfortunately my code is long and rambling. I would never expect you to look through that. I will get back with you on the logic.

[Rachel Kozlowski]

I am attempting to populate a report using a classes from Jasper Reports (3rd party software class).
The reportParametersMap is a Hashmap that is populated with data returned from a database:

  HashMap<String, Object>  reportParametersMap = new HashMap<String, Object>();
reportParametersMap.put("reportingParameterObject", reportingParameterObject);
  

and the reportDesignObj is basically used to format the Report, by putting a Title on the page and organize the report data on the report and then put footers at the bottom of the page.

The last line of code that the debugger hits before it goes off into the memory leak is :

  reportPrintObj = JasperFillManager.fillReport(reportObj, reportParametersMap, new JREmptyDataSource());  

The area of code that it is found in below is small. Do you see anything obvious?

 
public static JasperPrint compileNFillReportData(JasperDesign reportDesignObj, HashMap<String, Object> reportParametersMap) {

        JasperPrint reportPrintObj = null;

        try {
            int nWidth = 0;
            if (reportDesignObj.getPageHeight() > reportDesignObj.getPageWidth()){
            	nWidth = reportDesignObj.getPageHeight() + 5;
            	reportDesignObj.setPageWidth(nWidth);
            }

            //Compile the report
            JasperReport reportObj = JasperCompileManager.compileReport(reportDesignObj);

            //Fill the report
            reportPrintObj = JasperFillManager.fillReport(reportObj, reportParametersMap, new JREmptyDataSource());


        } catch (Exception e) {
            logger.error("There was an error trying to compile and fill the report ...", e);
        }

        return reportPrintObj;
    } 
  

[Lucas Franceschi]

you know, i may be talking non-sense, but there's a "runtime" object, that represents the current runtime of your process, this object have the "gc()" method, gc stands for garbage collector.

i'm not sure if java collects garbage for itself or if you need to call the gc() method, but it'll be a good practice to take a deep look at this.

also, make sure your variables are on the lesser scope that you can let them, because you would end up with a variable defined since the code start and only used inside one try, what would make that variable to be garbage outside the try, so define it only on the try.

again, i'm not completely sure about this, its just tips, things for you to give a look around the internet


regards

[fred rosenberger]

you do not need to call the garbage collection method - and I believe doing so can actually slow down your program.

Have you tried increasing your heap size? On the command line, you can type

java -X

and get the non-standard flags. You'll see

-Xms<size> set initial Java heap size
-Xmx<size> set maximum Java heap size

Try making both of these large... I don't know what the default is, or how much memory your box has, but this may help. Maybe something like

java -Xmx512m -Xms512m

if you have that much memory. This may also just be a band aid... if your data size keeps increasing, you'll eventually hit this limit and be back at square 1. Of course, this may have repercussions on other applications on the box, so take that under advisement.

[Campbell Ritchie]

There is usually no need to call the System.gc() or similar methods. Java will run garbage collection automatically.

And you still seem to be getting answers.

[Lucas Franceschi]

aha! you're working with jasper too!


look at this:
i am working with jasper reports,
today I got the same problem you got, out of memory.

solution?

JRFileVirtualizer virtualizador = new JRFileVirtualizer(50);		
reportParametersMap.put(JRParameter.REPORT_VIRTUALIZER, virtualizador);

maybe you'll need to import some of these:

import net.sf.jasperreports.engine.*;
import net.sf.jasperreports.engine.util.JRLoader;
import net.sf.jasperreports.engine.query.*;
import net.sf.jasperreports.engine.fill.*;
import net.sf.jasperreports.engine.util.*;
import net.sf.jasperreports.engine.data.*;

virtualization is a jasper prevention for your (and mine) case, out of memory.
that's because jasper builds the report on memory, then puts it into a file.

with virtualization, you tell jasper to create some temporary files, that would hold parts of the report, 'till its completed, and merges into one only report.
the "50" in the virtualizer, is the max number of virtual files you want to be crated.

with this property, you can give goodbye for your problem

so, its jasper's problem, not your code's.

(if you specified that you work with jasper in the first post, i could have helped you since the start.)

[Lucas Franceschi]

you know, i'm actually testing this property now, and its taking times like 2, or 3 hours to generate my "memory breaking" report, so that if you can test this property there, it would be a great help.

so i insist to say that i know it works, but I never tested it, i'm testing and its taking long time for me to get any result, i'm generating giant, ultrageous big reports, one for each result on database, that means kinda 100000 giant reports being generated once, only to make sure it will not be out of memory, under any circunstances.

also, the last line you execute before the memory leak will be the line that calls the report filling, because the memory leak occurs inside jasper's engine, and only the property I've told you can be a hope of avoiding it.

[fred rosenberger]

Lucas Franceschi wrote:...to make sure it will not be out of memory, under any circunstances.

That's a pretty bold statement. Reminds me of the time some guy allegedly said "Why would anyone EVER need more than 640k of memory?"

[Lucas Franceschi]

fred rosenberger wrote:

Lucas Franceschi wrote:...to make sure it will not be out of memory, under any circunstances.

That's a pretty bold statement. Reminds me of the time some guy allegedly said "Why would anyone EVER need more than 640k of memory?"

actually, the idea is:
i'm useng a method that prevents java to get out of memory, what should I do to test it?
push it to the limits! i mean, its clear that generating almost millions of reports, one bigger than another, will leak the memory, so if I dont get a "out of memory" exception, the property works fine.

its about testing, because the costumer will not have the conscience to know that the program "crashed" because of the "memory leak", no, what will it say? "your program crashed."

thats what i'm trying to prevent, i'm pushing jasper to its limits, so that i can make sure that anything the user does will not be mentioned as a "crash in my program."

we need to remember, when programming for sell, that our users may be an expert about computer, and may be a complete ignorant about computer, that's why we need a lot of testing.


in about 10 years, we will repeat your quote, but like this "Why would anyone EVER need more than 100Gb of memory?"
or "can you believe that some years ago, people needed to use a strange device, they called mouse, because they could not touch the screen? weird isn't it?"


I think my sons will get curious when i start to talk to them how computers needed to have keyboards in the "old" times.

[Rachel Kozlowski]

look at this:
i am working with jasper reports,
today I got the same problem you got, out of memory.

solution?
view plaincopy to clipboardprint?
JRFileVirtualizer virtualizador = new JRFileVirtualizer(50);
reportParametersMap.put(JRParameter.REPORT_VIRTUALIZER, virtualizador);
JRFileVirtualizer virtualizador = new JRFileVirtualizer(50);
reportParametersMap.put(JRParameter.REPORT_VIRTUALIZER, virtualizador);

Lucas, I am still having the same problem. Where in the code would I put this?

[Lucas Franceschi]

Rachel Kozlowski wrote:Lucas, I am still having the same problem. Where in the code would I put this?

well, you need to put it on your parameters map, in the same place where you define the other parameters.

try making some tests, take a look at jasper's documentation and try to make it work, but dont forget i'm having the same problem, so, post here if you get any results.


see you tomorrow guys! i'm going home.

[Rachel Kozlowski]

I did increase the heap size yesterday. It is at 1024.

If I run the complied code outside of eclipse it takes a number of minutes for the page to display, but when I run it in eclipse I get the error. Regardless, I need to solve this issue before I get even more data.

[Lucas Franceschi]

so, setting the property didn't work?
i dont know what to do, it does not work here too.

i'm totally lost now, increasing java heap space would not help, because i would need to change jhs in everyone that uses my program.

I dont know what to do, sorry rachel.

[Rachel Kozlowski]

I may be doing something incorrect. As another person from this forum noted Jasper Reports put out demo code and I looked at it and the syntax is slightly different from yours.
The snippet from the sample is different. Notice that they have it writing to a tmp file and also notice that the virtualizer is included in the fillReport syntax. I am going to try this next. You should also give it a shot.

   
JRFileVirtualizer virtualizer = new JRFileVirtualizer(2, "tmp");
			
			// filling the report
			JasperPrint jasperPrint = fillReport(fileName, ds, virtualizer);
 

The entire Sample is below:

   
import java.util.HashMap;
import java.util.Map;

import net.sf.jasperreports.engine.JRDataSource;
import net.sf.jasperreports.engine.JREmptyDataSource;
import net.sf.jasperreports.engine.JRException;
import net.sf.jasperreports.engine.JRExporterParameter;
import net.sf.jasperreports.engine.JRParameter;
import net.sf.jasperreports.engine.JasperExportManager;
import net.sf.jasperreports.engine.JasperFillManager;
import net.sf.jasperreports.engine.JasperPrint;
import net.sf.jasperreports.engine.JasperPrintManager;
import net.sf.jasperreports.engine.export.JRCsvExporter;
import net.sf.jasperreports.engine.fill.JRFileVirtualizer;
import net.sf.jasperreports.view.JasperViewer;

/**
 * @author Teodor Danciu ([e-mail])
 * @version $Id: VirtualizerApp.java 1229 2006-04-19 13:27:35 +0300 (Wed, 19 Apr 2006) teodord $
 */
public class VirtualizerApp
{

	/**
	 * 
	 */
	private static final String TASK_PRINT = "print";

	private static final String TASK_PDF = "pdf";
	
	private static final String TASK_XML = "xml";

	private static final String TASK_XML_EMBED = "xmlEmbed";

	private static final String TASK_HTML = "html";

	private static final String TASK_CSV = "csv";
	
	private static final String TASK_VIEW = "view";
	
	private static final String TASK_EXPORT = "export";

	/**
	 * 
	 */
	public static void main(String[] args)
	{
		String fileName = null;
		String outFileName = null;
		String taskName = null;
		

		if (args.length == 0)
		{
			usage();
			return;
		}

		int k = 0;
		while (args.length > k)
		{
			if (args[k].startsWith("-T"))
				taskName = args[k].substring(2);
			else if (args[k].startsWith("-F"))
				fileName = args[k].substring(2);
			else if (args[k].startsWith("-O"))
				outFileName = args[k].substring(2);

			k++;
		}

		try
		{
			// Virtualization works only with in memory JasperPrint objects.
			// All the operations will first fill the report and then export
			// the filled object.
			
			// creating the data source
			JRDataSource ds = new JREmptyDataSource(1000);
			
			// creating the virtualizer
			JRFileVirtualizer virtualizer = new JRFileVirtualizer(2, "tmp");
			
			// filling the report
			JasperPrint jasperPrint = fillReport(fileName, ds, virtualizer);


			if (TASK_PRINT.equals(taskName))
			{
				JasperPrintManager.printReport(jasperPrint, true);
			}
			else if (TASK_PDF.equals(taskName))
			{
				exportPDF(outFileName, jasperPrint);
			}
			else if (TASK_XML.equals(taskName))
			{
				exportXML(outFileName, jasperPrint, false);
			}
			else if (TASK_XML_EMBED.equals(taskName))
			{
				exportXML(outFileName, jasperPrint, true);
			}
			else if (TASK_HTML.equals(taskName))
			{
				exportHTML(outFileName, jasperPrint);
			}
			else if (TASK_CSV.equals(taskName))
			{
				exportCSV(outFileName, jasperPrint);
			}		
			else if (TASK_EXPORT.equals(taskName))
			{
				exportPDF(outFileName + ".pdf", jasperPrint);
				exportXML(outFileName + ".jrpxml", jasperPrint, false);
				exportHTML(outFileName + ".html", jasperPrint);
				exportCSV(outFileName + ".csv", jasperPrint);
				
				// manually cleaning up
				virtualizer.cleanup();
			}
			else if (TASK_VIEW.equals(taskName))
			{
				JasperViewer.viewReport(jasperPrint, true);
			}
			else
			{
				usage();
				System.exit(0);
			}
		}
		catch (JRException e)
		{
			e.printStackTrace();
			System.exit(1);
		}
		catch (Exception e)
		{
			e.printStackTrace();
			System.exit(1);
		}
	}

	private static void exportCSV(String outFileName, JasperPrint jasperPrint) throws JRException
	{
		long start = System.currentTimeMillis();
		JRCsvExporter exporter = new JRCsvExporter();

		exporter.setParameter(JRExporterParameter.JASPER_PRINT, jasperPrint);
		exporter.setParameter(JRExporterParameter.OUTPUT_FILE_NAME, outFileName);

		exporter.exportReport();

		System.err.println("CSV creation time : " + (System.currentTimeMillis() - start));
	}

	private static void exportHTML(String outFileName, JasperPrint jasperPrint) throws JRException
	{
		long start = System.currentTimeMillis();
		JasperExportManager.exportReportToHtmlFile(jasperPrint, outFileName);
		System.err.println("HTML creation time : " + (System.currentTimeMillis() - start));
	}

	private static void exportXML(String outFileName, JasperPrint jasperPrint, boolean embedded) throws JRException
	{
		long start = System.currentTimeMillis();
		JasperExportManager.exportReportToXmlFile(jasperPrint, outFileName, embedded);
		System.err.println("XML creation time : " + (System.currentTimeMillis() - start));
	}

	private static void exportPDF(String outFileName, JasperPrint jasperPrint) throws JRException
	{
		long start = System.currentTimeMillis();
		JasperExportManager.exportReportToPdfFile(jasperPrint, outFileName);
		System.err.println("PDF creation time : " + (System.currentTimeMillis() - start));
	}
	


	private static JasperPrint fillReport(String fileName, JRDataSource dataSource, JRFileVirtualizer virtualizer) throws JRException
	{
		long start = System.currentTimeMillis();

		// Preparing parameters
		Map parameters = new HashMap();
		parameters.put(JRParameter.REPORT_VIRTUALIZER, virtualizer);

		JasperPrint jasperPrint = JasperFillManager.fillReport(fileName, parameters, dataSource);
		
		virtualizer.setReadOnly(true);

		System.err.println("Filling time : " + (System.currentTimeMillis() - start));
		return jasperPrint;
	}

	/**
	 * 
	 */
	private static void usage()
	{
		System.out.println("VirtualizerApp usage:");
		System.out.println("\tjava VirtualizerApp -Ttask -Ffile");
		System.out.println("\tTasks : print | pdf | xml | xmlEmbed | html | csv | export | view");
	}

}

 

[Lucas Franceschi]

Rachel Kozlowski wrote:I may be doing something incorrect. As another person from this forum noted Jasper Reports put out demo code and I looked at it and the syntax is slightly different from yours.
The snippet from the sample is different. Notice that they have it writing to a tmp file and also notice that the virtualizer is included in the fillReport syntax. I am going to try this next. You should also give it a shot.

   
JRFileVirtualizer virtualizer = new JRFileVirtualizer(2, "tmp");
			
			// filling the report
			JasperPrint jasperPrint = fillReport(fileName, ds, virtualizer);
 

The entire Sample is below:
...

well, what happens is that the virtualizer will be passed, in this snipped that you have pasted, by the only parameter in the report, that means, the "fillreport" method needs filename, datasource, and parameters. cirtualizer will be a parameter in this case.

also, have you already tested it? if so, what was the results?

it have been some time since the last time I looked at this, so maybe you have already found a solution for this case, if so, please tell us.

try looking for JRFileVirtualizer on google, find the javadoc, and take a look at the constructors, then you'll note that the second argument is optional, and it will be on the local directory if nothing is passed.

hope you have already solved it, but anyway if you don't, try looking a bit more.