Minimal IT - Minimal integration 8: do you need tools?

22 November 2005

Minimal integration 8: do you need tools?

Before buying an integration tool, do not just ask "Is this the best?", but "Do I really need it?"

There are three types of integration tools that work on data as it is sent from system to system.

Problem solver tools take data that is not really suitable for integration, and turn it into a well defined interface. This includes tools that map databases or objects to messages, screen scrapers, and connections to legacy systems.
Brokers take one well-defined interface and map it to another well-defined interface. These are often used with message-based middleware, but there are also brokers for file-based transfers.
Process management tools control a series of related interfaces that between them form an identifiable business process.

Be very critical of the value that these tools claim to bring.

Problem solver tools can be useful for solving point problems, but always consider them tactical. I have had this type of tool presented to me as a strategic toolset - as if, strategically, I wanted to have the problems. It is better to put your money into making each system produce and consume well-defined XML-based messages.

Brokers can be useful within a standards-based integration architecture. However, if the broker applies significant business logic, such as the calculation of a discount on an order, the owners of the connected systems can lose sight of this logic. In my experience, brokers are suitable for simple reformatting, but processing which changes the business meaning of the data is best left within the systems themselves.

Process management tools are sold on the promise of business flexibility. Sales demonstrations show graphical process design tools, and suggest that entire business processes can be changed by dragging and dropping a few icons.

I am sceptical of these claims. I have seen process management tools used successfully in three ways:

To control the steps in a human workflow. They calculate what needs to be done next, and push the work to an appropriate person.
To manage the technical complexities of integration. For example, they automatically produce a confirmation message back to a system to indicate that a message has been received.
To manage the processes within a single system. The tool may control interfaces to and from other systems, but it is basically just the mechanism for implementing the logic of the system of which it is a part.

I have never seen a process management tool used to control a real end-to-end business process across different systems. I am sure some people have done this, and I would be interested to know the details. But in my experience, these tools do not live up to the promise of a significant across-the-board improvement in business flexibility.

Do not think of any of these tools as a "must have". I recommend:

Concentrate on establishing a standards-based integration architecture based on XML. Base your strategy on standards, not tools.
Evaluate the case for tools critically. The hand-written alternative, such as using XML stylesheets (XSLT) for translation, is often as simple.
Beware that tools can confuse ownership of business rules. Sometimes it is better to leave the work in individual systems where it can be properly understood and owned.