aspose file tools*
The moose likes Java in General and the fly likes Capturing the changes made to file(s) in a directory Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Java in General
Bookmark "Capturing the changes made to file(s) in a directory " Watch "Capturing the changes made to file(s) in a directory " New topic
Author

Capturing the changes made to file(s) in a directory

Mohammad Akon
Greenhorn

Joined: Jul 17, 2007
Posts: 19
Hello all,

I am developing an application that is monitoring the files (text files only) within a directory. The monitoring part is fine (thanks to threads on file watch). But is there an "efficient" way to capture the modifications made to those files (I need those changes to for reporting)? By efficiency I tried to mean....not going through the whole file again an again and looking for changes....may be using some sort of time-stamps/tag to identify last read line from the files and then read to the EOF?


May be there are other posts on the same topic but unfortunately I could not find them. Would appreciate if anyone can shed some light.

Thanks in advance.

--mmAkon
fred rosenberger
lowercase baba
Bartender

Joined: Oct 02, 2003
Posts: 11475
    
  16

would that work if someone edits the middle of the file? or is that something you don't need to worry about?


There are only two hard things in computer science: cache invalidation, naming things, and off-by-one errors
Mohammad Akon
Greenhorn

Joined: Jul 17, 2007
Posts: 19
fred rosenberger wrote:would that work if someone edits the middle of the file? or is that something you don't need to worry about?


The updates are always appended to the end of file (they are log files to be precise).
Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18886
    
    8

If you are only looking for data appended to the file, then you could keep track of the previous length of the file and, after data is appended, start reading the file after the previous length. (Hint: RandomAccessFile)

However if you're trying to extract data from a log file on a timely basis, it would be more practical to modify the logging so that you didn't have to hack the log files. For example if you were using log4j you could use its appender which sends data to a socket, and write a socket listener which would then receive the logs directly.
James Sabre
Ranch Hand

Joined: Sep 07, 2004
Posts: 781

Paul Clapham wrote:

However if you're trying to extract data from a log file on a timely basis, it would be more practical to modify the logging so that you didn't have to hack the log files. For example if you were using log4j you could use its appender which sends data to a socket, and write a socket listener which would then receive the logs directly.


The Log4j site has appenders for publishing to Multicast (unreliable but pretty fast) and for JMS (reliable). Of course one would need a JMS server to use JMS but there are several free ones out there.


Retired horse trader.
 Note: double-underline links may be advertisements automatically added by this site and are probably not endorsed by me.
Mohammad Akon
Greenhorn

Joined: Jul 17, 2007
Posts: 19
James Sabre wrote:
Paul Clapham wrote:

However if you're trying to extract data from a log file on a timely basis, it would be more practical to modify the logging so that you didn't have to hack the log files. For example if you were using log4j you could use its appender which sends data to a socket, and write a socket listener which would then receive the logs directly.


The Log4j site has appenders for publishing to Multicast (unreliable but pretty fast) and for JMS (reliable). Of course one would need a JMS server to use JMS but there are several free ones out there.


But the problem is I do not have access to the available logger(s). Some the third party applications are generating the log files and I do not know what kind of logger they are using and cannot do any computation with them. All I have are the few log files. And yes, I'll be populating msgs to a remote JMS Queue for further msg consumption.
Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18886
    
    8

Mohammad Akon wrote:But the problem is I do not have access to the available logger(s). Some the third party applications are generating the log files and I do not know what kind of logger they are using and cannot do any computation with them. All I have are the few log files. And yes, I'll be populating msgs to a remote JMS Queue for further msg consumption.


Yeah, I kind of expected this sort of answer. However you might consider inquiring a little more closely; if they are using log4j then you might be able to find the log4j configuration file and add in your own appenders. Provided of course there weren't management or legal issues involved in doing that.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Capturing the changes made to file(s) in a directory