File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Performance and the fly likes Observing heap space error when viewing 90MB file from UI after 4 times. Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Performance
Bookmark "Observing heap space error when viewing 90MB file from UI after 4 times." Watch "Observing heap space error when viewing 90MB file from UI after 4 times." New topic
Author

Observing heap space error when viewing 90MB file from UI after 4 times.

monica singh
Greenhorn

Joined: Jan 04, 2009
Posts: 26
My requirement is "To display some log statements on the UI by fetching it from a location"
Say,if i m having log file size of 96 MB,when I am trying to open with a CSV file on the UI for the 5th time,
I get a heap space error.
Can anyone specify an optimized way of doing file operations,here reading and displaying on the UI.

Basically,I wanted to know how would I clean up the objects that have got created for the first time when I say
'View' to display the logs on the UI.Second time or the next time when I click on the button to view logs
should not face any heap error problems.
Please help me out in this as this is a critical problem I am facing and not able to proceed further.

Thanks.
Joe Ess
Bartender

Joined: Oct 29, 2001
Posts: 8924
    
    9

There are many ways to get an OutOfMemoryError. Our performance FAQ has some links on how to Investigate OutOfMemoryError.
From your description, you are probably accumulating data in some data structure.


"blabbing like a narcissistic fool with a superiority complex" ~ N.A.
[How To Ask Questions On JavaRanch]
monica singh
Greenhorn

Joined: Jan 04, 2009
Posts: 26
Hi Joe,
I am not able to point out where exactly in my applciation is causing a heap error.
In this application.,I am having say a 62 MB file,which contains some lakhs of lines,I have to read this from a location and display in the UI as CSV file.
I am reading as strings ,all these lines.

Tried with Jprofiler.Installation was not successful as my linux box jboss doesnt come up after that.
Any thoughts.please share.

Thanks.
Anurag Blore
Ranch Hand

Joined: Jan 15, 2003
Posts: 74
I think you shoud use StringBuilder inplace of String to read these lines. Can you share the piece of code or may be outline of code how you are trying to read and display this file content.
Are you displaying whole content in oneshot to user(UI ?) how user gets benifited by seeing file with one lakh record?

I am asking these question to think about a proper solution for your problem.
monica singh
Greenhorn

Joined: Jan 04, 2009
Posts: 26
My problem briefly:

There is an Arraylist of bean objects which I would display when I click on a button from UI.

First,I would get a list of applicable files that I had to read from.There could be a single
file or multiple gz files to read from.

Then we would go for putting all the records in the arraylist.

For each of the files in the fileList,I would see the size ,if it is 5MB ,I would give the output
in a CSV file window to open/save.
I would start reading my file now.
Before that,I would see wheteher it is a normal file or archived zip file.
There are only some specific files I have to read from , only say 1.txt,2,txt or 1.gz,2.gz
I would have a check for these specified files.If those are true,
for normal files - bufferedReader = new BufferedReader( new FileReader( file ) );
if zip files - fileInputStream = new FileInputStream( file );
gzipInputStream = new GZIPInputStream( fileInputStream );
final byte[] bytes = new byte[ SIZE_OF_ARRAY ];
fileSize = 0;
int byteLength = gzipInputStream.read( bytes );
while ( byteLength > 0 )
{
fileSize += byteLength;
byteLength = gzipInputStream.read( bytes );
}

fileInputStreamForReading = new FileInputStream( file );
gzipInputStreamForReading = new GZIPInputStream( fileInputStreamForReading );
inputStreamReader = new InputStreamReader( gzipInputStreamForReading );

bufferedReader = new BufferedReader( inputStreamReader );

From this buffered reader,I would read all the lines,if its more than to be 5MB,I would write it to CSV,
otherwise,add to the arraylist which would get displayed on UI.
Joe Ess
Bartender

Joined: Oct 29, 2001
Posts: 8924
    
    9

Do you ever get rid of the information you accumulate in the array list or in the GUI?
If not, you will eventually run out of memory.
Anurag Blore
Ranch Hand

Joined: Jan 15, 2003
Posts: 74
Two questions..
you have mentioned you display the file details on screen if the file content is less than 5MB and provide option of
saving the file in .csv format if the file is more than 5 MB.

Are you clearing the bufferes after reading/writing the contents of these files?
Are you really creating a new file everytime or using same file name for each call or is there somekind of synchronization you are using?

In case you are not sure which Object refercne is causing the problem eithe you can refer the link provided by Joe.

In case you have jdk 1.6_07 or above you can run your application on this jdk or higher then you can use Java Visual VM
VISUAL VM to find which object references are cause of this problem.
monica singh
Greenhorn

Joined: Jan 04, 2009
Posts: 26
Hi,

How do I clear my buffers .Are you saying that I have to do a reader.close().
I could not find any method to flush or clear.

I am creating a new file every time since I would not be sure of what file user selects each time.
I am not doing any clearing of arraylist too..
I am using a class variable for that.For each operation,why should not it get cleared if it is an instance variable.

Thanks.
Joe Ess
Bartender

Joined: Oct 29, 2001
Posts: 8924
    
    9

Memory problems are tricky to diagnose. They are impossible to diagnose if we don't have the source to your application and can observe it working. Your best bet, as Anurag posted, is to use Visual VM to try to track what is happening to object instances in your program.
You should invoke close() on everything you open, though that is not likely your problem.
Are you appending the data on the GUI or are you refreshing the GUI with each load?
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12803
    
    5
Does your GUI involve any event listeners?

Improperly disposed of listeners are a prime cause of memory leaks with GUI apps.

Bill
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Observing heap space error when viewing 90MB file from UI after 4 times.