I have a text file called 'test'. There is a perl script which randomly populates the text file with some records. A java program is scheduled to read the new data from the text file.
There are cases when there is a burst of records in the textfile. If the text file has more than 10k records it needs to be renamed and kept separately and a new file with the same name should be made in place of it.
The problem here is that my java program should have logic to read only the data which arrives newly in the file. So what i do is...i keep a check on the number of bytes that i read..when new data comes, i open the file again and move the pointer ahead by however many bytes i've already read.
The problem comes when a the text file gets full and a new file comes in it place. In that case I will have to start reading from the first byte. How can i identify if test.txt has only been updated or replaced altogether.
I wanted to try this with the creation date. Storing the creation dates and to check if the current creation date is greater than the old one...but in Linux the creation date is not stored.
The second thing i tried was 'ls -i' seeing the inode information. I thought that the inode would change only when the file is replaced but the inode number seems to change even when the file is updated.
Can you suggest how exactly can i make out if a file in linux has just been updated or has been replaced.
Joined: Dec 21, 2008
This is the snippet of code if it can be of any help:
In this what I need to do is: When the text file is new, dont skip any bytes but start reading from 0. How should I do that I have no clue.
How can i check if the text file is created or has just been accessed using the size. Even at the time of creation and before the java program acceses the text file its possible that 100's of records get populated into it.
Hmm, good point, lets go on a different tack. Is there any reason why you reuse the same file name?
Joined: Dec 21, 2008
I want them to be of the same name because my java program loads a test.txt and keeps reading records from it.
The fact that the file is replaced at the other end now needs to be intimated to java.
Joined: Dec 21, 2008
The exact problem is as follows:
A test is being simulated where the telephone call records, called as the CDR (call detail records) are being produced at random, in bursts(or continuously) and put into a flat file called as test.txt. The text file is read by the java program, each record is picked at a time and processed. Now when the cdr file, the test.txt gets full it needs to be replaced. The java program should now not read from a particular (appended) location but from the start of the file.
That is why i need to see when the file is replaced and when it is modified only.
Since Linux (and therefore Java) doesn't support the created date time option a way around it might be to create a new temporary file, when you detect this temporary file you know you are reading from the new one. Then you can rename the temporary file to test.txt (overwriting the old file) and continue reading from there.
If you are keeping the number of bytes you have read, if a new file is created, the size of this file will be smaller than the number of bytes you have already read (correct?) so you will know you are reading a new file?
The other way you could go about it is at the time your program is creates the new file, then you know the next read you do from the file should start from 0 so use a flag to switch from reading from already saved location or from start of file.
You might want to check the mtime (modification time) of the directory in which your files are located. It will reflect the last time a file was created or deleted in the directory.
Charles Lyons (SCJP 1.4, April 2003; SCJP 5, Dec 2006; SCWCD 1.4b, April 2004)
Author of OCEJWCD Study Companion for Oracle Exam 1Z0-899 (ISBN 0955160340 / AmazonAmazon UK )
Joined: Mar 27, 2003
Charles Lyons wrote:You might want to check the mtime (modification time) of the directory in which your files are located. It will reflect the last time a file was created or deleted in the directory.
Ignore me---I tested it and it updates the mtime for the directory on write operations to contained files. An alternative solution would be to keep a hidden file (e.g. ".newfile") which is empty, and have Perl just "touch" it each time a new file is created. That'll update its mtime only when your Perl script writes a new file, so you can effectively use .newfile to determine the creation time of the latest new file. Though then what do you do if you have two new files between invocations of your Java program??