Siebel MTOM Attachments

Audience

This article highlights the importance of MTOM in the enterprise, and provides a high level overview of the challenges that I faced bringing the MTOM standard into Siebel. Due to the proprietary nature of the environment, the source code for the implementation cannot be provided, but it is hoped that this article will provide a strategy for system designers/architects out there wishing to achieve the same goal and bring their project forward into the 21st century.

Background

As we slowly move towards the paperless society, handling attachments becomes a crucial part of the Enterprise Messaging landscape. With the popularity of Web service SOA architectures, SOA professionals found themselves with a tricky challenge. How to send binary data in XML?

XML is a textual data format, it has to be well formed and contain legal characters. If you take a binary string and put it between a set of tags, certain characters of the binary string may prematurely escape the tag, or make the XML invalid. In the early days SOA professionals got around this problem by encoding the binary string into Base64 format, which allows binary data to be represented in ACSII format. Although this does solve the problem of transmitting binary using XML, it is not very efficient. A Base64 string is 33% larger than the raw binary, which means more resources are need to handle the larger message, as well as taking longer to transmit.

The next evolution came when SOA professionals realized they can use MIME to send attachments, and thus SOAP with Attachments (SwA) was invented. In short its just an XML message with the binary data tacked at the end of the payload. The attachment is distinctly separate from the payload, which means the message processor has to implement logic to get the URI and retrieve the attachment, another downside is that SwA does not support Web Service Security (WS-S).

MTOM is the next advancement in attachment messaging, which combines the best of SwA and XML with inline attachments. MTOM is based on a slightly modified version SwA, which utilizes an XML Optimized Packaging (XOP) command to logically include the attachment as part of the message, instead of just referencing to the location of the attachment, it also supports WS-S.

That was 2004, today is 2011.

MTOM and Siebel

Siebel does not support MTOM, or even MIME through WS, the only supported method of sending attachments is through Base64 encoding. Base64 works well for small attachments, and is universally accepted, but what happens when Siebel is required to handle large attachments? There is little choice, it is Base64, MIME (non WS), or hire a technical architect that will design a custom MTOM framework for you.

Creating MTOM Messages

There are several approaches to creating MTOM messages in Siebel, here is a high level run down of the options.

Option A

1. Query IO and get a SiebelMessage with Base64 encoded attachments
2. Pass SiebelMessage to JBS, to construct a MTOM message and create the XML payload
3. JBS converts Base64 to Binary stream and appends to the above MTOM message

Adv: Easy to implement, No file management
Con: The Attachment will be converted from Binary to Base64, and back to Binary each way


Option B

1. Query IO and Get a SiebelMessage with Base64 encoded attachments
2. Convert SiebelMessage to MIME Hierarchy in Siebel
3. Parse MIME binary chunk to JBS
4. JBS reconstructs the MIME to MTOM structure, and inserts the proper XML payload

Adv: No file management, good fun for Java programmer
Con: Difficult to implement for a Siebel developer,
The Attachment will be converted from Binary to Base64, and back to Binary each way


Option C

1. Extract file attachments to filesystem
2. Pass file locations to the JBS
3. JBS reads file into memory, creates MTOM message and XML Payload
4. Delete extracted files

Adv: Easy to implement, Attachment is never converted to Base64.
Con: File management. Increased disk IO.


MTOM messages can be created using standard MIME classes, since MTOM is packaged as a multi part MIME message. After the MTOM message has been created, it can be passed back to Siebel or sent to the transport directly from the JBS.

The main theme in these options, is the usage of Java Business Services (JBS). eScript does not have any methods to retrieve binary from a PropertySet, hence it needs to be passed down to a JBS where it can be retrieved, and manipulated.


Importance of MTOM

Dealing with attachments is Business As Usual for many industries. For example, record keeping is a core part of every Government departments DNA, it serves as a means for accountability, and preserves memory for historical purposes. With the proliferation of broadband, hospitals can send X-rays and other medical documents to roaming doctors. Insurance companies can process claims through PDF and automatically store them in the document management system. Integrating with document management systems becomes one of the key programmes of work in the Enterprise, and handling attachments efficiently becomes a key non functional requirement.

The world has moved to MTOM, and for Siebel to remain relevant it must keep apace with new technologies, otherwise it risks becoming a legacy system. There is no immediate risk for Enterprises that deal with small file sizes, as Base64 is still an acceptable technology, as long as your ESB still supports it, but it will not scale well into the future, especially with High Res/HD content becoming mainstream on the horizon.

The problem arises when your Enterprise makes the decision to embrace MTOM, and when the Enterprise architects are deciding which projects will provide the file attachment functionality.

The question they will be asking is your Siebel project MTOM capable and ready?