I think I have a memory leak: 33 megs of char arrays.. what to do?

[Marvin Diggler]

Hi,
Ran JProfiler no my program to see why I keep getting an OutofMemory error. It appears that I have a lot of char[]'s.. according to the program it gets up to 33 megs and just keeps going until I recieve the error.

I'm confused though, because I can see anywhere I use a char[]. In a couple place, I used charAt() and substring(), but those variables are reused so the reference should be destroyed with time the varible is reassigned.

If I run my program with 3 log files (its a web log scanner), it gets up to 71 megs. However, if I just load my program and have it load the data (just a serialized object), the program only takes about 30 megs of ram. I am assuming the memory leak is not happening with the object that stores the data, it must be somewhere else in the program. Is this a good assumption?

Thanks!

[Ernest Friedman-Hill]

Hi Marvin,

Welcome to JavaRanch!

Remember that Strings and StringBuffers contain char[]'s internally, and it's likely those char[]s that are giving you trouble. Do you have a List or Map full of Strings or StringBuffers? Or are you using StringTokenizer, which will keep such a collection internally? Are you keeping more data that you need? Or perhaps you're simply loading all of a log file into one humungous String and then parsing it?

[Marvin Diggler]

Hi,
Yes, I have TreeMaps filled with Strings. I have a lot(probably thousands) of TreeMaps actually because I have to show parent to child relationships and the number of hits for that relationship.

I am just baffled, because why if I save the data, and then reload it, the program is only using 31 megs as compared to the 71 it uses when it originally scans the files.

If its a problem with my maps, wouldnt the program take 71 megs when I load the data (from a serialized object)?

[Marvin Diggler]

also, I do use a string tokenizer for the log file. It just does one line at a time and then I reinit the string tokenizer for the next line.

[Ernest Friedman-Hill]

Depending on how a String is constructed, it might contain a larger char[] than is needed. Perhaps when the data is loaded in via serialization, the Strings are built to use exactly the correct size of char[].

[Marvin Diggler]

Is there anything I can do in my program to make sure the correct size char[] is used?

[Ernest Friedman-Hill]

Depends on how you parse the logs. What are you doing now?

[Marvin Diggler]

Currently, each line of the log is read with the StringTokenizer. I then call substring on the URLs (page and it referrer) to remove http://. Then the the URLs are stored in a HashMap. Do I need to call some type of resize method along the way?