ABS Framework - Logging & Tracing Module

Log files are the bread and butter of every developer, so every once in a while it is interesting to meet a developer who has never used Trace() before. So how do these people troubleshoot Siebel? Using debugger, amazing!

Although debugging is useful, it’s not always reliable. In complex scenarios, when you have logic across different objects, the debugger just dies, or jumps around erratically. In these cases it’s just more efficient to dump out all the information to a file and analyse what went wrong.

From the most mundane, to the most critical problem, having informative logs is crucial to solving problems quickly.

For this purpose Siebel provides us with a basic tool.

    TheApplication().Trace()

Good old Trace, what this does is allow us to route key information to a log file during script execution. This same log file is used by the Siebel application for its application logging. So the first frustration developers get is all the information they are trying to dump out, is being buried by all the Siebel internal object allocation and destruction information.

Trace() is used by developers during the build phase and is inserted when a problem is found or in key areas for unit testing, and it is removed before the code is migrated into the test environments and ultimately in production.

Trace()
does the job, and gives developers the basic means to troubleshoot problems. Its function is not flashy, and it’s a little better than popping up an alert box, but we all learn to live with it pretty quickly.

[Paradigm Shift]


With that perspective, anyone that looks at ABS business code, will notice one major contrast to the above logging paradigm.

The ABS tracing module is used as part of the program design, rather than to troubleshoot problems as they arise.

In standard code, we normally trace the start, end and catch of each function, and put lots of comments to explain the code.

In ABS code, tracing is done at the start, end, catch, and is put at key decision points in the program, and this is left in the code, even when it is migrated into production.

The advantage of doing this is two fold.

1. The code is self documenting
    By including descriptions of the program logic in business speak, anyone can turn on tracing, perform the function, and the system will spool the program logic to a log for analysis.

2. On demand function execution information
    Since the tracing code is part of the program design, a developer can go into any environment, look at the ABS logs, or re-run the function, and have access to the details of the function execution details.

Similar to the way Workflow provides a debug mode for developers to see step and variable information of the runtime action. This ABS design feature closes the gap on the advantage that declarative configurations once had.

This is quite powerful, because it allows the developer to debug the problem live in an environment with real data, which can’t always be created on a local environment.

[Logging Levels]

Conforming to the Siebel logging standard, the ABS logging module has 5 levels of logging detail. Setting this value controls the amount of detail that is spooled to the log file. By default it is set to 3, but this can be overridden by the developer in the system preference or in the code.



[ABS Logging & Tracing components ]

    1. MSG object
    2. LOG object
    3. EXCEP object


[MSG Object]

One of the key design principles around the ABS System is re-use. Messages can be categorised and parameterised and re-used across the application.

To display errors or log error messages, we need to create our own message category in the Application, ABS encourages this over hard coding custom errors on the fly.

The ABS System creates a category called ABS Defined Messages.

Instead of using numbers and defined ranges for their use, the ABS Logging module uses identifiers with the following naming standard.



The ABS Defined Messages is only used for the ABS framework for standard error messages, to display more specific error messages, we need to create our own custom message category, and follow the above standard to define our messages.

Although it is not enforced, it is encouraged to follow this standard, because it makes managing error messages, and looking up their relationship to the functional areas easier.

[LOG Object]

This object defines how a function is logged, and does the actual job of tracing the messages to a file.

ABS Log files are generated in the following format, but can be changed according to individual user preferences.































 LOG object Methods
 Begin  Tell the logging module that a new function has begun
 Step  Used to record key milestones in the script and document the process
 StepResult  Used to document the result of a milestone
 End  Tell the the logging module that the function ended without any errors
 Error  Logs the error object



The basic construct must be built like this, for the proper logging and log format indenting to happen.



[EXCEP Object]


In a standard application, when an error is raised, the developer can choose to override this error, to provide a different message that is more suited to the context, or bubble it up the the parent function, but local variables are lost when the function loses scope.

In ABS, when an error occurs, the EXCEP object is thrown, the relevant parameters are captured and stored in the EXCEP object, and the original message is discarded after it is logged, the EXCEP object is then bubbled up in the function chain.

When the error bubbles back to the base function, the parameters are written out along with the error message.

This is very interesting, because it allows us to display a meaningful error message with full context of the parameters at the point when the error occurred.

[Example Error Log]



This is the function calling path from the above example.
    BO_Service -> LOV_Service -> LOV_GetValue -> LOV_GetAnyField -> BO_Query -> SYS_Assert

The error is raised at SYS_Assert, but it has no useful information at this point, so the error is thrown up the chain to BO_Query.

BO_Query is the real reason why the exception was raised, the arguments captured by this function are important, and will not be available once the function finishes, so it is bubbled up further along the function chain, with arguments captured from previous functions, until it reaches BO_Service

BO_Service is where the message is finally displayed. The cause of the error is due to a BC query that returned no results, but you want to override this, and display something more meaningful to the end user, but still log the actual technical reason for the developer.

The result is an informative message that contains details gathered from various parts of the function calling chain.

[Consolidated Server Logs]

The ABS Logging module solves a very fundamental problem with Siebel server logs. If you are in a multi-server environment and have 10 Siebel application servers, each will have its own set of application logs. So what happens when there is an error? We need to find out which server we were logged onto!

Why do we have to hunt for our logs?

ABS allows us to save logs from all our application servers in one central location.


Developers and environment guys a like, will really appreciate this feature. The design of ABS is an evolution of the way Siebel manages its logs.

[Self Documentation]

Traditional documentation and diagrams can be quickly outdated, especially if the rules are complex, and change every release. Script should be easy to read and understandable, the ABS system takes this idea further, and uses the logging objects to document the system.

With the correct usage of these functions, the ABS logging module will allow anyone to go into the system, and generate 100% accurate documentation of the system in business speak.

[Pro-active Error handling]

During the build and test phase, a developer builds a piece of functionality, does the unit testing, and migrates the code for testing. The tester will look at the build and raise defects that will event get re-assigned back to the developer. This process can take days.

The ABS system takes a more proactive approach, when ever an error is raised, it uses a SR style ticketing system, and assigns the error, based on its category to the correct position, in real time. This way a developer can get all the details of an exception, even before the tester has even realised there was an error.

[Conclusion]

The ABS logging & Tracing module provides the following features

    1. Detailed contextual error messages
    2. Step execution and function parameter information
    3. Self documentation mechanism
    4. Problem ticket assignment
    5. Consolidated server logs
    6. Works in WF and Script


Developers will appreciate the level of detail provided by the ABS system.

    1. Point of error
    2. Function calling path
    3. Function inputs and outputs
    4. Name, Value, and Type of arguments used in function

ABS can generate a lot of logs, so its important to keep your eye on disk space. The good news is the design of ABS allows all logs to be saved to one central place, so you only have to monitor one location, unfortunately, we cannot do the same for Siebel logs.

The developer shouldn't have to hunt around the logs for an error, then wonder where it exactly originated, and what were the conditions that caused it to error. All this information should be captured at the point of error.

If we reflect for a moment and compare the ABS logging module to the standard Siebel solution, the TheApplication().Trace() feels somehow inadequate, its like trying to fight fire with a water pistol.

The ABS logging and Tracing modules provides advanced tools for projects to quickly identify problems, turn around a solution and focus on the delivery.


Reactions:

9 comments:

  1. This is really nice info and introduction to ABS Framework logging module.

    overriding the exception object and throwing a custom object is really nice but what I had observed is that in case I throw a normal exception object then I don't need to do a RaiseErrorText to show the error message on UI but if I throw a custom object then it goes along the chain as expected but never reaches UI and error message is suppressed.

    Any thoughts on this behavior???

    ReplyDelete
  2. Thanks Neel,

    I tried to give it a broad appeal, because not everyone has ABS code for reference. I'm glad you got some value out of it.

    Are you using ST or non ST engine, can you provide some examples. Send to my email, if you like.

    ReplyDelete
  3. Hello, very good article. Obviously you have lots of experience from real-life projects and a good architectural approach.

    I would like to propose some other areas to be implemented as a some sort of a script library (and native code too, CLib.invoke is a good thing after all if used with care).

    Well, it looks like quite a lot of things which have very wide and common application areas are completely missing from Siebel, which is sad (in spite of the fact that a lot of things Siebel supports out of the box).

    I would thing of supporing the following pieces of functionality as they are reappearing over and over again from project to project:
    1) Queues of entities. I mean queue of service requests, activities etc. which are handled by a pool of users. Well, some might say about Task UI and Universal Inbox, but those of you who had a pleasure to work with above-mentioned functionality might notice lack of architectural idea behind all this. Normally you have a pool of tasks to be perfomed by a pool of users with a complex logic of approval, parallel execution etc. All of these execution blocks should be accompanied by internal queues of tasks which accumulate incoming events based on FIFO or any other principle or sort order.
    2) Exclusive access to some data (analog of semaphores or critical blocks in Windows). In Siebel there is no standard notion of shared global between all the sessions other then database record (S_SSA_ID would be a good example if this approach used within Siebel itself though with some pecularities) and gateway data (which has no public API). In a lot of cases we need to obtain some sort of a global counter or some other shared global data to be accessed by more then one session. Moreover we need to introduce some kind of access policy to avoid concurrent read or write access depending on the task. Well, there are workarounds of course, starting from database's sequence fields and triggers (which in turn employ SELECT FOR UPDATE for example). But this is quite complex for such a simple task and may be greatly improved.
    3) Backgound UI processing. Well, imagine you have external billing or scoring system, accepting Web-Services as a means of communication, or you have your own Business Service that performs complicated computations to be presented to a user. You need to pass the parameters and show the result in an applet. The problem is that you have SLA to show the screen to a user within at most 3 secs, but billing can only provide response within 10 secs. There should be a mechanism to allow lazy population of data based on the availability of those data. In other words, there should be a mechanism of background calculation of data and a mechanism to push the data to the client.
    4) Server-to-client event propagation. This is some generalization of a second part from my previous point. Now there is no theoretical possibility to push event from server to client until client asks server for something (refresh broadcast, leave a record etc.) The only way to push some event to a client is to use non-documented features of Communication Toolbar, but not all projects use it. The simpler analog can and should be introduced (and also without java).
    5) Ability to invoke arbitrary SQL(Visual Basic and CreateObject/ADO don't work on UNIX). Well, I believe this need no further comments. This is not a good practice, this is potentially dangerous, but sometimes you just need to invoke very specific SELECT statement. On Windows this is normally done through VB/CreateObject/ADO and VBC on top, but on UNIX these is no such a possibility. The main problem is not invoking statement itself, but in returning results back to script/VBC/UI.

    Do you have formal scope or at least a list of ideas to be implemented in ABS?
    What's the commercial model you plan to use for ABS?

    ReplyDelete
  4. Hi fkirill

    You have very good questions.

    For question #1, there is an extra in ABS framework called "ABS Business Previledge Service" which should meet your requirements.

    Functionality in points 2,3,4,5 are not implemented in ABS.

    ABS provides other value-added features to EIM and EAI, but i dont have information on the comming features, and there isn't any available commercial model at the moment (that i know of).

    If you are really interested, I can direct you to a proper contact.

    You can reach me at
    jason@impossiblesiebel.com

    ReplyDelete
  5. Hi Jason,
    Very Nice and Interesting post. But where do I find this ABS logging module?

    Thanks,
    Biku

    ReplyDelete
  6. Hi Biku, You need to be on a project that already has ABS =), it currently cannot be openly sourced.

    ReplyDelete
  7. Jason - We've similar requirements as "fkirill" pointed out in this first comment
    1) Queues of entities.....

    and you said "For question #1, there is an extra in ABS framework called "ABS Business Previledge Service" which should meet your requirements."

    can you elaborate on "ABS Business Previledge Service"? point me to the link where I can find documents

    Thanks

    ReplyDelete
  8. Hey this is is a nice poet..I feel very happy to find such kind of informative posts

    ReplyDelete
  9. Hi Anon, you'll need to get in Contact with your ABS vendor for this functionality. This feature is not available in all versions of ABS - Jason

    ReplyDelete

Comments are open to all, please make it constructive.