Capturing the changes made to file(s) in a directory

Hello all,

I am developing an application that is monitoring the files (text files only) within a directory. The monitoring part is fine (thanks to threads on file watch). But is there an "efficient" way to capture the modifications made to those files (I need those changes to for reporting)? By efficiency I tried to mean....not going through the whole file again an again and looking for changes....may be using some sort of time-stamps/tag to identify last read line from the files and then read to the EOF?

May be there are other posts on the same topic but unfortunately I could not find them. Would appreciate if anyone can shed some light.

Thanks in advance.

--mmAkon

would that work if someone edits the middle of the file? or is that something you don't need to worry about?

fred rosenberger wrote:would that work if someone edits the middle of the file? or is that something you don't need to worry about?

The updates are always appended to the end of file (they are log files to be precise).

If you are only looking for data appended to the file, then you could keep track of the previous length of the file and, after data is appended, start reading the file after the previous length. (Hint: RandomAccessFile)

However if you're trying to extract data from a log file on a timely basis, it would be more practical to modify the logging so that you didn't have to hack the log files. For example if you were using log4j you could use its appender which sends data to a socket, and write a socket listener which would then receive the logs directly.

Paul Clapham wrote:

However if you're trying to extract data from a log file on a timely basis, it would be more practical to modify the logging so that you didn't have to hack the log files. For example if you were using log4j you could use its appender which sends data to a socket, and write a socket listener which would then receive the logs directly.

The Log4j site has appenders for publishing to Multicast (unreliable but pretty fast) and for JMS (reliable). Of course one would need a JMS server to use JMS but there are several free ones out there.

James Sabre wrote:

Paul Clapham wrote:

However if you're trying to extract data from a log file on a timely basis, it would be more practical to modify the logging so that you didn't have to hack the log files. For example if you were using log4j you could use its appender which sends data to a socket, and write a socket listener which would then receive the logs directly.

The Log4j site has appenders for publishing to Multicast (unreliable but pretty fast) and for JMS (reliable). Of course one would need a JMS server to use JMS but there are several free ones out there.

But the problem is I do not have access to the available logger(s). Some the third party applications are generating the log files and I do not know what kind of logger they are using and cannot do any computation with them. All I have are the few log files. And yes, I'll be populating msgs to a remote JMS Queue for further msg consumption.

Mohammad Akon wrote:But the problem is I do not have access to the available logger(s). Some the third party applications are generating the log files and I do not know what kind of logger they are using and cannot do any computation with them. All I have are the few log files. And yes, I'll be populating msgs to a remote JMS Queue for further msg consumption.

Yeah, I kind of expected this sort of answer. However you might consider inquiring a little more closely; if they are using log4j then you might be able to find the log4j configuration file and add in your own appenders. Provided of course there weren't management or legal issues involved in doing that.