How we can implement logger for multithreaded application? The requirement is all the logger messages of each thread should be loaded into separate logger file. i tried with apache log4j logging ?.Please help with your suggetions.Thanks in advance...
One possibility is to continue logging everything to a single file, but make sure each log entry includes the thread id - this can be specified in the log4j configuration file. Then make a separate process to read the main file and write each entry to a separate file, based on the thread ID.
Another possibility is to create a custom Appender subclass that keeps a set of different FileAppenders, one for each thread. A ThreadLocal would be a good way to keep track of these. Then when doAppend() is called to create a log entry, use the ThreadLocal to get the correct FileAppender for the current thread, and call its doAppend(). So your custom Appender will route each call to the appropriate FileAppender. You can associate this custom Appender with the Logger by calling logger.addAppender(). You can either use a single Logger for this, or you can make a custom LoggerFactory to ensure this association is done for every Logger you may create.
There may be additional complications here; I'm not really that familiar with the internals of log4J, so I'm guessing. It may take some work to get this working correctly. Then again, it's also possible that there's some existing tool that will do this for you quite easily, and I just don't know about it. If so, perhaps some other kind soul will tell us about it in a subsequent reply. Good luck...
"I'm not back." - Bill Harding, Twister
Joined: Nov 27, 2007
First option, we tried with FileAppender . But we are not able get the logger messages to respective logger files. The logger messages are logging in jubmpling way to different loggers. Also we tried by giving different log4j properties for each thread. But we are getting the same result.
I have not done anything on it but log4j provides a NDC i.e. nested diagnostic context that can store thread specific informations and this context is sent to the logger. They claim that you can route log messages using filters and appenders based on this information. This and this articles have some info. May be worth a try.
With normal process, there will not be any problem. But if an exception occurs in the thread. It will exit the program in the middle. We cant apply the NDC stratagy in this situation.
Joined: Jan 30, 2000
[Patri]: First option, we tried with FileAppender . But we are not able get the logger messages to respective logger files. The logger messages are logging in jubmpling way to different loggers. Also we tried by giving different log4j properties for each thread. But we are getting the same result.
Well, any suggestion we make can be met with "it didn't work" - it may take effort to debug. It sounds like maybe the appenders are being run from different threads than the originating logging events. Perhaps you can include some calls to print the current value of Thread.currentThread().getName() to see if the threads used by the Appenders match the threads used in the code that makes the logging calls. If they don't match, then you can probably just apply the strategy I suggested at a different level. E.g. you can create a custom Logger rather than a custom Appender, and let the Logger keep a set of other Loggers in a ThreadLocal to look up the appropriate Logger for that thread.
You don't say anything about the first option I suggested. Well, you do begin your reply with "first option", but you appear to be talking about the second thing I suggested. It sounds like my first suggestion may well be simpler. Note that I'm not suggesting different log4j properties for each thread - I'm suggesting using a log4j format that will include info on the current thread. That should be pretty simple. Then write a program that reads the log file and splits it into separate files based on the value of the thread identifier.
Or, regarding the NDC suggestion:
[Patri]: With normal process, there will not be any problem. But if an exception occurs in the thread. It will exit the program in the middle. We cant apply the NDC stratagy in this situation.
Why not? You need to put calls to NDC.remove() and pop() into finally blocks to ensure they're called at appropriate times. That can be tedious and may be error prone, but it's not impossible. If you miss a call to pop(), that just means you'll have more info than you want; it's not the end of the world. And it will be info about what was going on when the error occurred, which quite possibly you want in the log anyway. Just make sure you call NDC.remove() when a thread (or Runnable) ends, so you don't get a memory leak. I haven't really used NDCs myself, so I'm not sure, but I don't see what your objection is here.
Joined: Nov 27, 2007
My intension is not to say anything not working. The 2 options, one deviding the log thread wise at the end process causing performance problems. It was taking long time. the second option also tried. i used ThreadLocal with separated the FileAppenders for each thread. But no proper logging is possible. With NDC also same performance problem came when testing.Thanks for your support....