SOAP Services With MTOM

As you probably know, SOAP is an XML based protocol which means that all data inside a SOAP message must be text based. If you want to include binary data in a SOAP message, it too must be text based. To achieve this you can convert binary data to a base64 encoded string and simply embed that string inside the SOAP message. The diagram below shows a sample SOAP message with binary data embedded as a base64 string.

Sending Binary Data Over SOAP Without MTOM

While this is the simplest approach to dealing with binary data over SOAP, there are a few things to consider. When binary data is base64 encoded it increases in size by approximately 30%. For small amounts of binary data this probably won't be an issue, but for larger volumes of data the increased message size can significantly impact performance.
Another point to consider is the processing overhead for the XML parsers that process the SOAP messages. A large amount of binary data will result in a huge base64 encoded string, and more CPU intensive parsing for the consumer.

Introducing MTOM

Message Transmission Optimisation Protocol or MTOM for short, can be used efficiently handle binary data transmission via SOAP.  Rather than base64 encoding binary data and embedding it in the SOAP body, the binary data is sent as a MIME attachment. As shown in the diagram below, the binary data (a PDF in this case) is sent in the HTTP request/response as a MIME attachment. The SOAP message contains a unique key that is used to reference the MIME attachment

Sending Binary Data Over SOAP With MTOM

The SOAP message remains lean, as we avoid the base64 encoded bloat that we saw with the first approach. A smaller XML payload also means less resource intensive XML parsing is required.

Sample Code

In the next few sections I'll use code snippets to show you how to use MTOM with a CXF service. The full source code for this post includes a fully working MTOM enabled CXF service and integration test. Feel free to pull it from github before reading on.

Schema Definition For Binary Elements

An account SOAP service returns simple bank account data, represented by the Account XSD type defined below.  On  line 7 a new base64Binary typed Statement element has been added to represent a PDF document.

1
2
3
4
5
6
7
8
9
<xsd:complexType name="Account">
    <xsd:sequence>
       <xsd:element name="AccountNumber" type="xsd:string"/>
       <xsd:element name="AccountName" type="xsd:string"/>
       <xsd:element name="AccountBalance" type="xsd:double"/>
       <xsd:element name="AccountStatus" type="EnumAccountStatus"/>
       <xsd:element name="Statement" type="xsd:base64Binary"/>
    </xsd:sequence>
</xsd:complexType>

When the WSDL2Java process is run, JAXB generates a POJO with a Statement instance variable of type byte array as shown below.

JAXB Generated Model For Base64 Binary

Using a byte array for the Statement element means that service consumers will have to read the entire binary statement into memory in one go. This can be improved by telling JAXB to use a DataHandler instead of byte array. DataHandler returns an InputStream which allows the client application to stream the binary data if needs be. This is particularly useful when dealing with large amounts of binary data.

To switch from from byte array to DataHandler you need to update the XSD with the expected MIME content types. On line 7 below xmime:expectedContentTypes="application/pdf"  indicates that we are expecting the binary data to be of MIME type application/pdf. The xmime:expectedContentTypes attribute can be set to any valid MIME type or a comma separated list of MIME types.

1
2
3
4
5
6
7
8
9
<xsd:complexType name="Account">
    <xsd:sequence>
       <xsd:element name="AccountNumber" type="xsd:string"/>
       <xsd:element name="AccountName" type="xsd:string"/>
       <xsd:element name="AccountBalance" type="xsd:double"/>
       <xsd:element name="AccountStatus" type="EnumAccountStatus"/>
       <xsd:element name="Statement" type="xsd:base64Binary" xmime:expectedContentTypes="application/pdf"/>
    </xsd:sequence>
</xsd:complexType>

When I run the WSDL2Java process JAXB regenerates the domain model and the Statement element is now typed as a DataHandler and annotated with @XmlMimeType(application/pdf).

Generated JAXB Model When Expected MIME Type Specified

Configuring MTOM with CXF

Once the XSD has been correctly defined, enabling MTOM is very simple. The Spring configuration below defines an EndpointImpl class using an injected CXF Bus. The endpoint is configured to use 2 interceptors for logging and MTOM is enabled on line 21 by simply setting the mtom-enabled key to true.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
@Configuration
@ImportResource({ "classpath:META-INF/cxf/cxf.xml" })
@PropertySource("classpath:application.properties")
public class Config {

    @Bean
    public ServletRegistrationBean servletRegistrationBean(ApplicationContext context) {
        return new ServletRegistrationBean(new CXFServlet(), "/*");
    }

    @Bean
    public EndpointImpl serviceEndpoint(Bus cxfBus, 
                                        AccountServiceEndpoint accountServiceEndpoint,
                                        @Value("${mtom-enabled}") Boolean mtomEnabled,
                                        LoggingInInterceptor inInterceptor, 
                                        LoggingOutInterceptor outInterceptor) {
  
       EndpointImpl endpoint = new EndpointImpl(cxfBus, accountServiceEndpoint);
       endpoint.getInInterceptors().add(inInterceptor);
       endpoint.getOutInterceptors().add(outInterceptor);
       endpoint.getProperties().put("mtom-enabled", mtomEnabled);
       endpoint.publish("http://localhost:8080/mtom-demo/service");
       return endpoint;
}

The sample code includes an integration test that calls the service and logs the SOAP response. Below is an extract from the logged response showing the SOAP body and MIME attachment. Note that there is no binary data in the Statement element, but an xop:Include element instead. The href value on line 20 references the Content-ID value of the MIME attachment on line 31.
The MIME attachment content type on line 28 is application/pdf.  This is consistent with the MIME content type we set in the XSD.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
ID: 1
Response-Code: 200
Encoding: UTF-8
Content-Type: multipart/related; type="application/xop+xml"; boundary="uuid:74cee6e6-296e-44f8-9de6-a2d5da7b4f2b"; start="<root.message@cxf.apache.org>"; start-info="text/xml"
Headers: {}
Payload: --uuid:74cee6e6-296e-44f8-9de6-a2d5da7b4f2b
Content-Type: application/xop+xml; charset=UTF-8; type="text/xml"
Content-Transfer-Encoding: binary
Content-ID: <root.message@cxf.apache.org>

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
   <soap:Body>
       <AccountDetailsResponse xmlns="http://com/blog/samples/webservices/accountservice">
          <AccountDetails>
             <AccountNumber>12345</AccountNumber>
             <AccountName>Joe Bloggs</AccountName>
             <AccountBalance>3400.0</AccountBalance>
             <AccountStatus>Active</AccountStatus>
             <Statement>
                <xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:eec6e626-5536-4778-a821-37de8e1f018b-1@com"/>
             </Statement>
          </AccountDetails>
      </AccountDetailsResponse>
   </soap:Body>
</soap:Envelope>

--uuid:74cee6e6-296e-44f8-9de6-a2d5da7b4f2b
Content-Type: application/pdf
Content-Transfer-Encoding: binary
Content-ID: <eec6e626-5536-4778-a821-37de8e1f018b-1@com>

%PDF-1.4
%????
2 0 obj
<</Filter/FlateDecode/Length 565>>stream
x????j?0 E?z
-?E]???-??B? ?L ?,? ????_?H??2 ? ]?|t?????T?< ?=??K? B??  7?????????a??:?9?~)C?????f$??w

Sample Code

As mentioned earlier, feel free to grab the source from github and have a play around. If you have any comments or question feel free to leave a comment below.

Comments

Post a Comment

Popular posts from this blog

Spring Web Services Tutorial

Image
Update
I've recently written a more up to date version of this post which describes how to build a contract first web service using Apache CXF. Check it out here.

Introduction
Modern enterprise applications are rarely stand alone and often rely on data and services provided by external systems. In order for different types of systems to communicate there must be a  communication protocol of some sort, a standard way of sending and receiving messages in a format that is recognised and supported by all major platforms. SOAP (Simple Object Application Protocol) is such a protocol, and allows applications to communicate by exchanging messages in a standard XML format.
SOAP Web Services provide a platform agnostic integration mechanism that allows disparate systems to exchange data regardless of the platform they are running on. For example, SOAP web services are commonly used to integrate .NET applications with applications running on the Java platform. Almost all modern platforms and …
Read more

REST Endpoint Testing With MockMvc

In this post I'm going to show you how to test a Spring MVC Rest endpoint without deploying your application to a server. In the past, full integration tests were the only meaningful way to test a Spring REST endpoint. This involved spinning up a test server like Tomcat or Jetty, deploying the application, calling the test endpoint, running some assertions and then terminating the server. While this is an effective way to test an endpoint, it isn't particularly fast. We're forced to wait while the entire application is stood up, just to test a single endpoint.
An alternative approach is to unit test by manually instantiating the Controller and mocking out any required dependencies.  While such a test will run much faster than a full integration test, its of limited value. The problem is that by manually instantiating the Controller we're bypassing Springs Dispatcher Servlet, and as a result missing out on core features like
request URL mapping request deserialization re…
Read more

An Introduction to Wiremock

This post provides a brief introduction to Wiremock, showing how it can be used to to quickly and easily mock remote API calls. We'll use Wiremock to write some integration tests for a simple Dropwizard app and show you how it can be put to use in a real world scenario.

Why would I need to mock external API calls? There are a number of scenarios where it makes sense to mock an external API rather than call a live service.
The external API may still be in development and not yet available for integration. In this instance as long as a data contract has been defined (e.g. Swagger spec, WSDL), the remote API can be stubbed based on the data contract. Stubbed endpoints allows a team to continue development even when external APIs isn't fully implemented.You may have little or no control over the external API uptime in development or test.  As a result you cannot guarantee it will be available to call when running integration tests. In this instance it makes sense to use mocked respo…
Read more

Spring Batch Tutorial

Image
Introduction Spring batch is a lightweight framework that provides a solid foundation on which to build robust and scalable batch applications. It provides developers with a set of tried and tested patterns that solve common batch problems and allows developers to focus more on the business requirement and less on complex batch infrastructure. Spring batch contains a variety of out of the box configurable components that can be used to satisfy many of the most common batch use cases. Extensive XML configuration and an extensible programming model mean that these components can be customised and used as building blocks to quickly deliver common batch functionality.
This tutorial will show you how to build a very simple batch application to read fixed length data from a flat file and write it to a database table. This is a common batch use case and should be sufficient to demonstrate some of the fundamental concepts of Spring batch and provide you with a foundation on which to build mor…
Read more

Externalising Spring Configuration

Enterprise applications typically contain a number of  properties and  runtime resources that must be configured before the application is deployed. Property and resource values can either be baked into the artefact at build time using something like Maven Profiles or derived at runtime using Spring Profiles. My preferred approach is to externalise all configuration so that the artefact taken from the build server is environment agnostic and can be deployed onto any environment . For example, we could take a WAR file from our build server and  deploy it to our development, UAT or production environments without changing any configuration in the WAR file itself.

Spring Property Configuration Spring applications use the PropertyPlaceholderConfigurer to load application properties. Property files are typically loaded from the class path but the example below uses a slightly different approach, loading the properties file from a directory outside of the application itself. This allows the…
Read more

Docker & Spring Boot

Docker allows you to package an application with its dependencies, into a light weight, portable container that can run on almost any environment.  You can think of a Docker container as a run time, a mini virtual machine that encapsulates your application and its dependencies.
In order to run a container you need a Docker image. An image is like a template that defines everything that will exist within the container. You can almost think of an container as a run time instance of the image it was created from. In this post we'll define and build 3 slightly different Docker images that run a simple Java app.

Installing Docker If you haven't already done so you'll need to install Docker. The official documentation is pretty good so following it step by step should see you up and running in about 15 minutes.
I've installed Docker on Windows and Ubuntu but to be honest I prefer running it on Ubuntu and have found it a bit more reliable than with Docker Toolbox on Windows.��…
Read more

Axis2 Web Service Client Tutorial

Image
Introduction
Axis2 is a Java framework that provides comprehensive support for exposing and consuming web services. This short post will look at its SOAP client support and how it can be used to get a simple web service client up and running.
For convenience I'm going to be calling a web service that I recently built as part of another blog post.  If you don't already have a web service to call you can grab the full source code for my sample service from github. Simply run a Maven build and deploy the WAR to your Servlet container.

What is a Web Service Client
This post doesn't attempt to explain the detailed inner workings of a web service client, but its still pretty useful to have an idea of what's going on under the hood. Most web service clients provide the following
A client side proxy for the remote service we want to call, that allows our application to invoke a SOAP service using a simple method call. The proxy insulates our application from the intricacies of s…
Read more

jQuery Tables

Introduction
This post will show you how to create rich data grids using the DataTables plugin for jQuery. DataTables is a flexible framework that allows you to create grids using a variety of data sources. This tutorial will show you how DataTables can be used with DOM and JavaScript data sources.

DOM Data Source
The simplest way to use DataTables is with a DOM data source. This approach simply 'decorates' an existing HTML table and is ideal for enhancing plain HTML tables created using template languages like JSP or Velocity.

MakeModelBody ShapeEngine Size (Cubic Capacity)Horse PowerBMW320dSaloon2000181BMW330dCoupe3000231BMW335dEstate3000286BMW335iSaloon3000306BMWM3Coupe4000415BMW520dSaloon2000181BMW530dSaloon3000231BMW535dEstate3000286BMW535iSaloon4000306BMWM5Saloon5000508
To turn this rather dull looking HTML table into a rich data grid we simply decorate the table on document load.
1: <script type="text/javascript"> 2: $(document).ready(func…
Read more

Health Checks, Metrics & More with Spring Boot Actuator

Image
I've been working with Spring Boot for a few years now and I'm a big fan. There are lots of things to like about Spring Boot, but one thing that really stands out for me is the emphasis on production readiness.
Production Readiness Production readiness is about looking beyond functional requirements and ensuring your application can be properly managed and monitored in production. Some key things to consider when thinking about production readiness are...
Health checksViewing application configuration - application properties, Environment variables etc. Viewing and altering log configurationViewing application metrics - JVM, classloader, threading and garbage collection. Audibility of key application events Spring provides all of this functionality out of the box via Spring Boot Actuator, a sub project of Spring Boot. A range of RESTful management and monitoring endpoints are provided so you don't have to implement these features each time you build an application. To enable…
Read more