Win a copy of Mesos in Action this week in the Cloud/Virtualizaton forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Persistence and retrieval of logging

 
Peter Chase
Ranch Hand
Posts: 1970
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
My application has its own logging mechanism. It is similar to Java 1.4's own logging or Log4J. I guess that I could switch to one of these, if necessary. However, neither my logging, nor these others, has any advanced back-end for persistence or retrieval.
The application requires that logs are persistent across runs of the server and that information from the logs can be retrieved, with various types of sorting and filtering. There can be a lot (a few thousand) records, some of the records contain long (up to 1kb) messages, and retrieval speed is important.
One way to achieve these aims would be to back-end the logs with a full-blown SQL database, accessed via JDBC. However, this would be a very heavyweight solution, since such a database is not needed for any other aspect of the application.
Can anyone suggest another way to do it? Are there perhaps third-party open-source projects in this area?
[ November 25, 2003: Message edited by: Peter Chase ]
 
Stan James
(instanceof Sidekick)
Ranch Hand
Posts: 8791
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You might do simple log files AND a reportable database. Get the database off to another machine, use asynchronous messaging to write so it doesn't slow down your main app. Sounds pretty elaborate but your requirements are pretty tough.
Oh, do your reporting requirements have to be up-to-the-minute? Can you import a day's worth of logs into the reporting db every night? Every few hours? Something other than real-time?
My own logger has pub-sub so it supports new listeners to log messages. I just made one that keeps a configurable number of messages in a circular buffer in memory for easy & fast review. If you're writing thousands per second, that would fall apart very quickly, tho.
[ November 26, 2003: Message edited by: Stan James ]
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic