File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Java in General and the fly likes Help: Parse a log file and split every log entry. Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Java in General
Bookmark "Help: Parse a log file and split every log entry." Watch "Help: Parse a log file and split every log entry." New topic
Author

Help: Parse a log file and split every log entry.

java em
Greenhorn

Joined: Feb 17, 2013
Posts: 3
parse a log file and split it by individual log entry. The log entry can be a single line or multiple lines. In addition separate out the timestamp from the rest of the message.

- Must not load entire file in memory (stream)
- Must capture the entire entry (single and multiple lines)
- Must split the entry also in two parts, timestamp and the rest of the entry
- Must use a regular expression to parse
- Must be configurable to parse log files in slightly different formats

example input(part of the log file):


[18/09/2011 06:53:21:021 PM] ERROR Main 70: Running logger, watch out!
java.lang.IndexOutOfBoundsException: Index: 99, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
[18/09/2011 06:53:21:021 PM] INFO Main 73: Running logger, watch out!
[18/09/2011 06:53:21:021 PM] INFO Main 73: Identity manager starting.

and output 3 log entries:
1 -
[18/09/2011 06:53:21:021 PM] ERROR Main 70: Running logger, watch out!
java.lang.IndexOutOfBoundsException: Index: 99, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
2 -
[18/09/2011 06:53:21:021 PM] INFO Main 73: Running logger, watch out!
3 -
[18/09/2011 06:53:21:021 PM] INFO Main 73: Identity manager starting.


My questions:

1. if I use below to read log file, is it correct to meet the requirement (- Must not load entire file in memory (stream))?
BufferedReader br = new BufferedReader(new FileReader(file));
String line;
while ((line = br.readLine()) != null) {
// process the line.
}
br.close();

2. can I save timestamp and the rest of the entry to a HashMap<String, ArrayList<String>>? and key is the timestamp, the rest of entry is saved to ArrayList?
but if the log file is big, the HashMap should be very big, maybe it is not a good way.

3. how can I save the result? Can I save the result to a new log file (every entry has a number)?

Thanks

Carey Brown
Ranch Hand

Joined: Nov 19, 2001
Posts: 174

java em wrote:
- Must be configurable to parse log files in slightly different formats

Configurable might be the stickiest part of the requirements.

java em wrote:
and output 3 log entries

Seems like writing to STDOUT would satisfy the requirements.

java em wrote:
My questions:

1. if I use below to read log file, is it correct to meet the requirement (- Must not load entire file in memory (stream))?
BufferedReader br = new BufferedReader(new FileReader(file));
String line;
while ((line = br.readLine()) != null) {
// process the line.
}
br.close();

Yes.
java em wrote:

2. can I save timestamp and the rest of the entry to a HashMap<String, ArrayList<String>>? and key is the timestamp, the rest of entry is saved to ArrayList?
but if the log file is big, the HashMap should be very big, maybe it is not a good way.

Don't see anything in the requirements that would call for a map or list.

java em wrote:
3. how can I save the result? Can I save the result to a new log file (every entry has a number)?


Redirect STDOUT to a file if desired.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Help: Parse a log file and split every log entry.