wood burning stoves 2.0*
The moose likes Java in General and the fly likes FileInputStream large files Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Java in General
Bookmark "FileInputStream large files" Watch "FileInputStream large files" New topic
Author

FileInputStream large files

Tom Hill
Ranch Hand

Joined: Aug 24, 2003
Posts: 115
Im developing in java 1.4.2. Im using a FileInputStream to read in (1MB) chunks of a large file and transfer them to another machine which is read to write the chunk to a file. Im getting an out of memory error reading in the file (after about 60mb) any ideas? Have a feeling that there is some zombie references sitting about inside of my reading loop.
Also are there any good mechanisms for checking memory usage?

Cheers

Tom
Ernest Friedman-Hill
author and iconoclast
Marshal

Joined: Jul 08, 2003
Posts: 24183
    
  34

A proper read loop should have constant memory use, so yes, there's probably a leak.

Traditionally, commercial tools like JProbe or OptimizeIt have been the best way to monitor and examine memory use; I understand there are some useful monitoring tools that come with Java 5, however.


[Jess in Action][AskingGoodQuestions]
Tom Hill
Ranch Hand

Joined: Aug 24, 2003
Posts: 115
The out of memory error was being caused by an ObjectInputStream.
The chunks Im reading are being put into a Transfer object that contains some other useful information. I was using an ObjectIn/OutputStream to write the transfer objects between sever and client. I have discovered that references are not being updated. Calling reset causes various Exceptions. Is there an easier mechanism for transfering objects that may change during the reading file loop?
Tom Hill
Ranch Hand

Joined: Aug 24, 2003
Posts: 115
...because Im calling reset on the wrong stream at the wrong time!

Cheers guys, once again youve given me lots to think about!



Tom
Layne Lund
Ranch Hand

Joined: Dec 06, 2001
Posts: 3061
If you still need help, please post some code to show us what you are doing. Paraphrasing often loses important details that we need to help you.

Layne


Java API Documentation
The Java Tutorial
 
 
subject: FileInputStream large files
 
Similar Threads
close and reopen the bufferedreader
Marcus Q# 53 I/O
Not Reading Entire File
nio package and memory mapped interface
strreams 4 massive file?