SiebUnit - an xUnit implementation

Introduction

Over a year ago, I presented a concept of xUnits in Siebel and how they could be used. Because it was only a concept, it was missing implementation details, so there was some doubts to whether Test driven development would work in Siebel, where code is meant to be restricted to fringe cases.

At that time, I ported some code from a back end Java project that I was working on, to automate Siebel interface verification. The idea was to pickup XML templates from a file system, insert randomize data, send it to the ESB, and validate the results.

It was used for several releases, reducing a days worth of effort to verify 50 interface interactions, to about 5 minutes. With the JUnits in place, they could be used to verify an integrated environment instantly on demand, instead of employing highly paid professionals to perform repetitive regression testing.

The tool was written in Java, didnt have a UI, it had access to the eApps SSO hashes, which meant that an evil developer could use it to log onto a server environment, and impersonate anyone they wanted. It was more of a regression testing tool than a unit test tool, and being written in Java, it was inaccessible to most of the Siebel Team.

In the spirit of a true Siebel developer, I thought a real implementation of xUnits in Siebel had to be scriptless.

So "SiebUnit" was designed to be just that, a fully declarative unit testing framework for Siebel developers.

This framework also happens to plug into a rules engine like BRP, OPA, or it could be run against a collection of business services/workflows, as long as you can pass in an Input and get an Output property set, the "SiebUnit" framework handles the rest.

This idea was sold to a Client, who immediately picked it up as a way to get developers to unit test, improve build quality, and reduce dependence on testers, and provide a structured development process.

This article and its followup, takes a step further, providing architectural and implementation concepts. It is aimed at the Technical Architect, who can grasp high level concepts, fill in the low level implementation, possibly innovate on this idea, and introduce SiebUnits into your Siebel project.


What is SiebUnit

If you had to sell the concept of xUnits in Siebel, this slide should sum it up.



This screenshot shows a SiebUnit implementation that was built for a Client.



Features

Notification System for test results

Uses a custom notification system, written in a client side library, to add persistent messages to the current view. The messages can be dismissed, by clicking the X button.


Design and Run Test Cases from the UI

Test Cases are configured, executed and debugged all within a standard Siebel UI. Debugging the rules itself, depend on the capability of the rules engine. Eg. BRP can only be debugged via log files, WFs can be debugged using the instance monitor.


Export/Import test Cases

Allow easy export and import of hierarchical test configuration data between different environments.


Trace Toggling

Toggles developer tracing, for performance testing of rules


Dynamic test data

Uses XML templates, and proxy functions to the BO Library to generate unique test data. Random data can be generated and provided on inputs, or overridden in the case of XML templates.


Expression Builder

A helper utility designed for the developer to construct the test cases declaratively. This is built using HTML, which integrates with the Siebel session.


Other

SiebUnits comes with pre built internal test cases to validate your eScript engine, and profiles ST objects vs non ST objects.

SiebUnits can be configured to run recursively to load test an environment


Case Study - Custom Rules Engine

SiebUnits was used on a project to ensure the integrity of a Rules Engine.

The design of the custom rules engine started off in small components, followed by unit testing, further build and some more unit testing. The components were then integrated, followed by another round of unit testing.

As the design got exponentially more complex, the integration of all these components, made it impractical to regression test all the exception paths, after every change.

SiebUnits was originally built with integration to BRP. So for this project, it was modified to work with this new engine.

A collection of SiebUnits was built to test the following aspects.

Declarative IF conditions
Declarative loops
Declarative procedures
Expression Translation
Expression Results
Business Service Invocations
Property Assignment
Cancel Propagation
Error Messages
Math Operations
Rule inheritance
PropertySet Interactions


Some further internal test cases were also designed to validate the behaviour of the eScript engine. This ensures that if Siebel changes the eScript engine, these internal SiebUnits will pick up on the expected outputs of dependent core functions, and allow the root cause to be addressed.

Once the SiebUnits were designed, and configured, they were used to repeatedly validate the Rules Engine after every change, to ensure previously built functionality still functioned. Changes often broke other unexpected parts of the engine, due to the tight integration and dependency between components, but with the SiebUnits in place, the report of failed test cases helped to quickly pinpoint the cause.

Towards the end of the build, a critical defect was discovered with the eScript engine, that corrupts a core object of the rules engine. This happens during random runs of the test suite, but it was easily reproducible using the load runs.

The problem required a re-design of the core object, which unfortunately touches every function in the engine. Since the new object had a different interface to the old object, it required a re-design of most of the interactions around this object as well.

The affected parts of the program was identified, and a fix was implemented in two hours. The SiebUnits test suite was run afterwards to ensure that the rules engine passed all the test cases. This provided a QA stamp on the product, and was similiar to having a team of regression testers on hand to validate your work at will.

The collection of SiebUnits also allowed load testing of the engine in a real environment. During these load runs, server resources can be monitored to ensure that they run within acceptable standards and allow capacity planning based on expected usage.

It would have been much more difficult if not impossible, to complete such a complex build without SiebUnits in place, and the regression testing and maintenance effort would have been an ongoing burden.


Intermission

Do you have BRP, OPA or a collection of Business Services that you would like to keep in check? The next article will look at the architecture of SiebUnits, and all the components that need to come together to form the framework of SiebUnit.

Note: The term Rules Engine is used broadly in this article. Even though BRP stands for Business Rule Processor, it is actually a declarative eScript replacement engine. OPA is more suited to the term for a Rules Processor, as it takes a bunch of inputs, makes a determination and returns the result. However the Rules Engine interface of Inputs/Outputs is what makes it suited to SiebUnits.

View the next article here


4 comments:

  1. Would love to use this framework. What is the best way to engage with the team?

    ReplyDelete
  2. Hi Thomas,

    These series of articles on SiebUnits, provide insight into the design of a testing framework, which was based off an actual SiebUnit implementation that was built privately for a customer.

    If you would like more information, please contact me directly on linkedin.

    Thanks

    ReplyDelete
  3. Hi there, we would like to understand more about Siebel unit testing using xunit. let us know the possible way we can engage you.

    - sathish Pr

    ReplyDelete
  4. Hi Sathish

    Thanks for your interest. Please contact me on LinkedIn, and I can provide you with more information.

    Thanks
    Jason

    ReplyDelete

Comments are open to all, please make it constructive.