I'm working on a project in which we need to log transactions as to whether or not they have completed, and notify the team when they have not. Please see sample of what we would like to do.
1. write job to a Activity log. 2. track each step by writting them in the Activity log. 3. after final step, write job to a Completed log. 4. REMOVE the entire job entry from the Activity log. 5. if a job's final step, has NOT been logged by a certain time, write job to an Unfinished log. 6. REMOVE the entire job entry from the Activity log.
Steps 3 - 6 is to prevent the Activity log from getting too large with completed and interrupted jobs. Originally we were using a relation database to track activity. However, we thought it was overkill and wanted to try using flat files. We are trying the flat files, but find that it is quite cumbersome, and are now considering using XML documents.
My question is...
Would anyone recommend that we use XML documents for this type of log monitoring, since we need to manipulate the files so heavily?
And, if so, could anyone advise as to how to go about doing this?
A relational database sounds fine, but if you want to use the file system, I don't think that XML is the solution. I think you could change the design a little and achieve the same result a bit more easily.
How about, instead of writing to an activity log, and then trying to remove entries from the log after they've been added, you try this:
write the activity log items to a separate log file per transaction, identified by some sort of transaction id (12345678.log)
when the activity is finished, log to the completed activity log/db table as per normal, and just delete the whole activity log file for that transaction
That way should a transaction not complete, you can easily find all the related activity log entries for that transaction because they all reside in a single file, rather than having to filter some sort of flat file or XML structure.
Also, you then wouldn't have to try and REMOVE log entries from a central activity log file, which is certainly possible using scripting against a flat file or by manipulating an XML file, but is most certainly awkward, and a potential resource strain if working with large XML files. [ August 10, 2005: Message edited by: DW Bolton ]
Joined: Jun 29, 2005
Further to this, obviously a transaction would be incomplete if its activity file remained after the predetermined amount of time. You could just monitor the file system and write out to an unfinished log if that was the case.
Really, a couple of tables in a database would be a great way of dealing with this if you've already got a persistence framework in place, but if you're heart is set on the file system, then I don't think using XML will make your job any easier. XML is a fantastic technology that just happens to be not suited to what you want to achieve.
The only place I can see using XML would be in some final reporting format for completed jobs. If you needed to create printable reports in various formats, XML storage would let you use XSL and related tools to churn out reports. Bill
Joined: Jan 23, 2004
Thank you for your suggestions everyone. I will relay these messages back to my team.