I am doing a project for monitoring the log files. When application(tomcat) start, it start one thread for one monitored file. The thread will run once very few seconds(pause by sleeping). Each time it will open and read the log file(bufferedReader) from the end line of previous reading. if any pre-defined keyword found in the log file, the matched line will be stored to stringbuilder, once it finish reading or the stringbuilder is bigger than a size, it will open particular file and write mathced content to the file.
The issue is that the (daily) log file will grow to be big (hundred Megebyte) from morning to the evening, and I noticed the cpu is 100% running when the file is big. I am wondering if there is good way to do this task like Bare Tail does which do not have much affect on memory and cpu?
david arnold wrote:Each time it will open and read the log file(bufferedReader) from the end line of previous reading.
Instead of just opening a file after a time interval, I would suggest something like checking last modified time. If file is not modified at all, there's no point in opening the file in first place.
david arnold wrote:if any pre-defined keyword found in the log file, the matched line will be stored to stringbuilder
I'm not sure which OS you are working with, but in Linux, this can simply be done by grep command. No need to even open a file using Java classes. On Windows, there might be something similar to it.
I hope this helps.
Regards,
Anayonkar Shivalkar (SCJP, SCWCD, OCMJD)
david arnold
Ranch Hand
Joined: Apr 10, 2009
Posts: 129
posted
0
Thank you William and Anayonkar.
Exactly how are you positioning at the end line of previous reading?
That is not something possible with BufferedReader which would have to read the whole file, resulting in what you observe.
Since I used bufferedReader to read line by line, I have counter to remember the line number.
Also note that a Reader is going to be doing character by character conversion to UNICODE, an expensive operation.
Thank you. I did not know it, but I was wondering maybe it will be more efficient if I use stream. The reason i use bufferedReader is that it help to maitain the line by line format, when a line is read and it has keyword inside, it will be recorded with "\n" to the file, later it will be displayed on a browser and may need to send to email as well. so i do not know if i use stream, it will still easy to have this line format maintained.
Instead of just opening a file after a time interval, I would suggest something like checking last modified time. If file is not modified at all, there's no point in opening the file in first place.
Good point, i will add that.
I'm not sure which OS you are working with, but in Linux, this can simply be done by grep command. No need to even open a file using Java classes. On Windows, there might be something similar to it.
It is windows, it will be wonderful if it has such feature, but I am wondering if there is any.
Exactly how are you positioning at the end line of previous reading?
That is not something possible with BufferedReader which would have to read the whole file, resulting in what you observe.
Since I used bufferedReader to read line by line, I have counter to remember the line number.
You can try the java.io.RandomAccessFile class, which can be used to seek to the point that you want, without having to read everything all over again.
You can try the java.io.RandomAccessFile class, which can be used to seek to the point that you want, without having to read everything all over again.
Thank you Henry. I did think about using the RandomAccessFile before, i will try that.
Currently, I open the file and read and close it every few seconds, I am wondering if I should open the file, then have a loop checking for new data coming in without close it. I am not sure which way is better?
0
