Ideas for logging :: Operation Logs and Recovery

Overview

Most developers have faced the logging dilemma atleast once in their career. It only becomes relevant when developers realize that they have been working on a project for months and there is no safety backup system in place. They then clutter production code with log snippets that write to a file in absolutely no format. Last time I checked the point of logging was to have a paper trail for any transactions and to be able to recover from errors. Without any proper formatting, the logs are not very helpful because it becomes nearly impossible to read them with ease. Software can generate very large logs especially if it is used frequently. Large, disorganized logs are not analysis friendly and can even end up wasting time instead of doing the exact opposite.

Approach

The slide that give me the idea

I have faced this dilemma also and have been trying to come up with a solution that works on the application level. The idea I came up with is basically stolen from database systems and their logging approach. Basically, every transactions / action that needs to be logged should be logged in a standard way. Lets assume we use some sort of a tuple format:

<what, was, is, when, why>

'what' is the data that was changed. 'was' is what the data's previous value was. 'is' is what the data's current value is. 'when' is when the data was modified. 'why' is any notes relevant to the change. Using this tuple, we can log nearly any type of change and keep it in a standardized format instead of having each logging be of a different message.

Recovery

To reiterate, one goal of logging changes is to be able to recover from errors. With disorganized, unformatted logs this becomes an extremely touch manual process. To automate recovery, use the tuple system to find all the pieces of data that where modified in the required time frame and roll back each change one by one. Once each change is rolled back the state of the application should be the same as it was at the beginning of the chosen time frame. For example:

< post.date, nil, '2011-01-01', 'new' >

< user.age, 19, 20, 'birthday' >

< user.age, 20, 22, 'mistake' >

< post.date, '2011-01-01', '2011-01-02', 'update' >

Rolling back the modifications will reset the user's age to 19 and the post date to '2011-01-01'. Of course, the exact format of the tuple and the data stored in it should be changed to fit the application requirements, but the concept remains the same.

Now go ahead and organize that cluttered code from your app!