File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes I/O and Streams and the fly likes File is open for long Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » I/O and Streams
Bookmark "File is open for long" Watch "File is open for long" New topic
Author

File is open for long

Matron disney
Greenhorn

Joined: Apr 24, 2005
Posts: 7
I have an application which opens a file does processing.The file is a big one so we do this in a loop.We read every 1000 lines once ina loop.the whole reading takes approx 20 sec but processing takes 2-3 hours as there are hughe processing logic for that stream.after the (reading+processing) we close the stream.Is this a problem.Will this be consuming more resources as compared to closing the stream in every loop.
Has anybody got any problems not closing the stream for so long.Mind you this is an input stream and not an output stream.
David Harkness
Ranch Hand

Joined: Aug 07, 2003
Posts: 1646
I see no problem with keeping an InputStream open for any length of time. System.in (the keyboard or redirected input, normally) is open for the entire run of your program.
Matron disney
Greenhorn

Joined: Apr 24, 2005
Posts: 7
Thanks for the reply.Just one correction.We do have many files let us say 100 in no.so as the processing for all thesfiles takes couple of hours and reading just a minute or two don't you think it is better to close the file after every chunk of reading.The performance may hit but at least we will not 50 readers open at the same time.My question is;
1)Is reader is associated with system resource which by closing will release it.
2)Is there any limit of opening the readers at one time.
Joe Ess
Bartender

Joined: Oct 29, 2001
Posts: 8997
    
    9

Originally posted by Matron disney:

1)Is reader is associated with system resource which by closing will release it.

Yes.

Originally posted by Matron disney:

2)Is there any limit of opening the readers at one time.

Yes. It is not uncommon to get an error which indicates that there are too many files open with poorly tuned database applications.


[How To Ask Questions On JavaRanch]
Stan James
(instanceof Sidekick)
Ranch Hand

Joined: Jan 29, 2003
Posts: 8791
This is taking me back to mainframe days. Do you have to worry about checkpoints and restart? If the thing crashes halfway through would it be good to know the first n records or files processed fine and you can restart at n+1 next time? That might influence how much you read at a time and how you treat the file between reads.


A good question is never answered. It is not a bolt to be tightened into place but a seed to be planted and to bear more seed toward the hope of greening the landscape of the idea. John Ciardi
Matron disney
Greenhorn

Joined: Apr 24, 2005
Posts: 7
No that is not a big concern.The concern which we have now is should we open the file read the file in chunks,do the processing ,again read in chunks do the processing till athe whole file is processed and then close the reader/file or
we read the file in chunks ,do the processing close the file again open the file and so on.
The processing time in most of these is 1-2 hours,reading is 3-4 min.
Henry Wong
author
Sheriff

Joined: Sep 28, 2004
Posts: 19064
    
  40

Originally posted by Matron disney:
No that is not a big concern.The concern which we have now is should we open the file read the file in chunks,do the processing ,again read in chunks do the processing till athe whole file is processed and then close the reader/file or
we read the file in chunks ,do the processing close the file again open the file and so on.
The processing time in most of these is 1-2 hours,reading is 3-4 min.


Given modern day JVMs and operating systems, I don't think is a problem keep the files open... However, having a device driver development background, I cringe everytime someone abuses resources, when they don't have to. I say if it not a concern, then close those files.

Henry
[ April 28, 2005: Message edited by: Henry Wong ]

Books: Java Threads, 3rd Edition, Jini in a Nutshell, and Java Gems (contributor)
Matron disney
Greenhorn

Joined: Apr 24, 2005
Posts: 7
why type of resources are consumed as I didn't see any increase in memory consumption when more readers are involved.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: File is open for long