An XML Logger

This article is Part 2 of Siebel Error Handling Approach

Introduction

When we think of the top Integration tools used in a Siebel project, typical answers such as SOAP UI, Oxygen, XML Spy, Wireshark, Fiddler, MQ Harness, and even Hex Editors may come to mind.

But there is an even more important tool which is missing. You've probably guessed it, what list would be complete without an XML logger?.

Not to be confused with an eScript logger that prints a property set or a WF step that simply writes the XML to a text file. An XML logger is a component, that is part of a more larger Error Handling Framework, which should provide the capability to associate the XML message with the related error information.

Importance of an XML Logger

Once upon a time, I took over the reigns of an Error Handling framework that had an Achilles heel: it chopped off every single EAI message at the 2000 character mark, and all that was captured was a partial message header full of name spaces. Not only was the payload discarded, only part of the message wrapper was captured.

For the benefit of non EAI professionals reading this article, imagine that as a kid, someone gave you a lollipop, except that there was no candy, not only that, half the plastic wrapper was torn off, so you don't know what flavour it was, or even the name of the bastard who gave it to you. This situation could bring any kid or EAI professional to tears.

For an EAI professional, living without XMLs is like working blindfolded, it's almost impossible to pinpoint where failures are occurring.

If the system didnt log the outbound message, the developer would have to spend time recreating the test case, and simulate what the message "could" have looked like at a point in time, assuming the problem is reproducible. Without access to the full inbound message, your hope would rest on the external system providing the message, that your system should have rightfully logged.

Having to live with poor error handling, is a hard lesson to learn, but it does illustrate a common need for projects to have access to all Inbound and Outbound messages across all Interfaces.

EAI XML Queuing Service

Siebel provides a solution in the form of the "EAI XML Queuing Service"

http://docs.oracle.com/cd/B31104_02/books/ConnSAP/ConnSAPeAIQueue12.html

Professionals in the Siebel Community have already discovered this business service has the capacity to act as an XML logging tool.

http://siebelexplored.blogspot.com.au/2011/01/exception-handling-using-eai-queue.html

http://siebeltips.wordpress.com/2009/02/19/eai-queue-for-error-handling/

http://geeksbloggingat.com/2009/05/03/siebel-eai-queue-usage/

To summarise, this out of the box business service can be used to log a SiebelMessage to the Siebel file system for storage and retrieval.

The EAI Queuing service has the following benefits

★ Can store more than 2000 characters! Hooray
★ Message can be logged with Error Information
★ Can be used to replay a transaction
★ OOTB

However, if you have a diverse portfolio of services to interact with, where the SiebelMessages have to conform to Enterprise schemas, or even third party proprietary formats, then you will need the capability to log the raw interface message. This could be in the form of XML, CSV, flat file, or it could be SiebelMessage with a different wrapper.

The problem with the "EAI XML Queuing Service" is that it only supports logging of a valid SiebelMessage, which might seem to dismiss this business service as a credible solution, but there is an easy work around. A simple XML based IO can be created, with one IC, and one Field called "Payload". The XML can simply be injected into this Payload property to store your XML. This XML based IO simply acts as a wrapper to allow the XML to be accepted by the EAI XML Queuing Service.

This works, but it does have a few major drawbacks.

★ The wrapper has to be stripped to reveal the Raw Message
★ The raw message, wont actually be raw anymore. XML cant be nested in other XML without being interpreted as part of the schema, so Siebel will automatically HTML encode your raw XML, so it can be accepted as text.
★ The file type is TXT, and the file name cannot be controlled
★ There is a 1-1 relationship between the error and the message
★ The available fields to store error information is limited

There are major limitations with using this OOTB business service as a standalone solution, because in order to exchange messages with external teams, the EAI Queued message has to be stripped of the wrapper, and decoded before it can be used. EAI Developers can perform this task without complaint, but it is hardly accessible to other teams.

Eg. Testers who have access to the raw messages, can raise defects, and exchange the raw XMLs with external teams directly, but wont necessarily have the tools or knowledge to decode XML, so providing access to the raw XML provides convenience and saves time.

Using the EAI Queue Item as a means to store error details, also means you are limited to storing one message per error record, unless a method is devised to flatten the SiebelMessages, and store them as properties in a wrapper IO, which comes back to the decoding problem described above.

The other issues described in the points above are fairly obvious, and don't need any further narrative, so lets move on, and look at the ideal solution.

XML Log Details

In the last article, I provided a list of error details that should be captured. Here is a revised list of the error details that should be captured when an Integration error occurs. I've expanded on the original list to reveal the specific details an Error Framework and XML logger should handle for Integration (highlighted in blue).


★ User Name
★ Error Time
★ Host Name
★ Primary Id
★ Internal Error Code
★ Error Message
★ Error Category
★ Transaction Reference
★ Transaction Time
★ Calling Function/WF
★ Stack Trace
★ External Error Type
★ External Error Code
★ Transport Time
★ Transaction Id
★ Transport and Transport Parameters
★ XML Logs
   Outbound XML
   Outbound SiebelMessage
   Inbound XML
   Inbound SiebelMessage


It should be clear at this stage, that we'll need a custom table, and a parent child relation between the error message, and the interface messages. It should also be clear that the EAI Queuing Service works OOTB, but dosnt provide the flexibility for complex EAI error handling requirements.

There is a lot to take in, so in the next article, we go through the solution to overcome these limitations, and introduce a new capability, that will allow you to have full visibility of the interfaces firing around you.

Siebel Error Handling Approach

Introduction

Error Handling or Exception Handling, is an essential part of any Siebel project, but where does it come from?. Unlike other commercial development environments, Siebel dosnt provide a standard error handling framework out of the box, or even as a third party add on. Surprising as it sounds, It is left up to each project to implement their own specific error handling capability.

Importance of Error Handling

"Jason, can you help me, my Tools is corrupt, the WF designer discards every one of the error branches that I try to create!"

"I see the problem, you are not providing the Error branch with a name"


The above is a real life scenario of a senior consultant not knowing how to handle errors in Workflow. While Error Handling is important, it seems not everyone knows how to implement it. Its an issue that plagues many developers, because the first step in development is to ensure our build is working first, before we can really understand the circumstances around how it could fail. It requires experience and forethought, to build to components with robust guards in place.

Not everyone gets it right the first time, but that's okay, because robustness is improved over time, but more alarmingly, not every developer designs with Error Handling in mind, nor goes back and puts error handling in place after the build is unit tested.

Error Handling Standards and Features

Error Handling is more than just using TheApplication().Trace at key points in your program, and it is not limited to scripting. Error Handling frameworks vary from project to project, but the fundamentals should remain the same.

Here is a list of ideas, which I've separated into 5 categories, that you could incorporate into your Error Handling Approach.

Scripting
★ Ensure call functions are wrapped in try/catch/finally
★ Put return statement after the finally block
★ Errors should be bubbled up, and handled at the place of invocation
★ Capture stack trace
★ Capture state of variables

WF
★ All major steps should have unhandled exception branches and error checks
★ Unhandled Exceptions should capture as much context around the error as possible
★ Sub processes should return an error property to the parent WF where the error is handled
★ Parent WFs should utilise the Error Process Property or route Errors to a standard project Error Handling Sub Process

Integration
★ All Outbound/Inbound messages should be logged
★ Capability to recognise and handle SOAP Fault and SOAP Warnings accordingly
★ Capability to handle non committed Enterprise transactions
★ Retry capability

BRP
★ Validation errors shouldnt be put into the system error property
★ Errors should be bubbled up, and handled at the place of invocation

General
★ Logging framework to allow debugging
★ Message Lookups
★ Settle on a standard Error Property
★ Provide generic friendly message

Some of the above ideas are standards that need to be enforced through training and technical reviews, while others are capability and tools that needs to be built to support the error handling needs of the project. Some projects take the idea a little further, and provide help desk features such as error assignment, capturing resolution and KPIs.

Show me the details

When an error occurs we want to log as much information as possible, at the point where the error occurred, and depending on the severity, we can make an informed decision on whether to fix the issue straight away or defer it to another release.

Some of the most important error details to log are


★ User Name
★ Error Time
★ Host Name
★ Primary Id
★ Internal Error Code
★ External Error Type
★ External Error Code
★ Error Message
★ Error Category
★ Transaction Time
★ Calling Function/WF
★ Stack Trace
★ XML Logs

Until next time

Imagine that you have an application in production, your users are getting generic error messages that is of no absolute value, and you have to spend hours trying to recreate the issue to get to the bottom of the problem. At this point, you are probably wishing that you had better Error Handling capability, but it is too late, because you know you would have to live with this until the next release, so don't let poor error handling slip by unnoticed.

Having full details of the error, provides you with the confidence to resolve issues, or ignore problems and move on, and defer fixing it in the next iteration of your development. Any reputable System Integrator would have an Error Handling Framework ready, that they would have carried over from past projects, or even if you've got an in-house project, there's nothing stopping you from drawing on the experience of your consultants, or use the above list as fuel, to build your own Error Framework.

Handling errors when working with EAI, requires more specialised capability, and in the follow up to this article, we go through the design process of how to implement the XML logging feature listed above, and in the tradition of Impossible Siebel, bypass a Siebel limitation a long the way.

Part 2 continues here