6 September 2005

Shared database? I wouldn't dream of it

By Andrew Clifford

A lot of what we think of as good practice in IT has no benefit, but just adds cost. For example, the dream of an enterprise-wide shared database has caused massive cost and waste over the years.

There are a lot of good ideas in data design. It makes sense to standardise on terminology and define data items carefully. It makes sense to have one corporate view of the way to represent the relationships between our main data such as customers, products, and orders.

This makes sense as a consistent logical view which we use as a basis for designing our systems, or understanding the impact of packaged systems.

The problem comes when we dream of implementing our standard model as a single database to which all systems attach.

Surely a single enterprise-wide shared database is a good idea? It would enforce all those good standards we have for data design. It would get rid of the need for all that difficult integration. Even if we can't completely build it, isn't it a good ideal to aim for?

I don't think so.

I doubt that any large organisation has managed to build a single shared database (though businesses run on a single large package like SAP might be getting close). But I think that pursuing the dream of single shared database has led us to a whole host of ineffective solutions and extra costs:

Despite this, we still cling to the dream of a single shared database. Some are now considering federated databases (technologies to make multiple databases look like a single database) as a way of building a single view. But databases are inconsistent and continue to diverge for all sorts of commercial, technical and political reasons. Putting on a consistent facade will not fix the underlying inconsistencies, and will hit problems as the databases diverge further.

I believe we need to accept the inconsistencies of different business areas, systems and databases. At a technical level, we need to keep databases separate, and use explicit integration to reflect flows of responsibility, meaning and control. This will allow us to build an architecture which is realistic and flexible, and systems that can be meaningfully owned by business. We must wake up from our dream of the single shared database.