The root of all evil

By Andrew Clifford

The major problems in IT all have roots in the technical structures upon which we base IT solutions.

The three big problems in IT are:

• The high failure rate of large, expensive projects.
• The growth in unmanageable "legacy" systems.
• The shortage of skilled staff.

There are many causes to these problems, but a large part of each problem can be traced back to the way that we structure technology, and particularly to the designs that we have historically had to use to provide workable and cost effective solutions.

To illustrate this, I have drawn a diagram of IT problem cause and effect, which is accompanied by full notes. It is not possible to fully explain the diagram in this newsletter, but I will summarise some of the main features.

The problems with major projects are largely caused by unrealistic expectations and a lack of control. These are related: if the drivers for the project are unrealistic there is nothing firm to control the project against. There are a number of factors that lead to unrealistic expectations: politics, a lack of clarity of how IT will deliver value, and an overselling of IT projects.

These factors are caused by the difficulties of establishing business value and focus across the complicated and shared structures of IT. They are compounded because the IT organisation is in the difficult position of being the only group that understand IT but is not well placed to lead business change.

These factors in turn have their root in the complexity of the IT structures and their misalignment to business structures, both of which are caused by the underlying technical design.

The growth of unmanageable legacy is largely a management problem, rather than a direct technical problem. The main management problems is the lack of a consistent handle to manage the legacy by - something to show what to change, what modifications to make, and how to justify it.

This is caused by the difficulties of understanding and communicating complicated technical structures. We can handle upgrading hardware and renewing software licenses. But we can not grasp and can not make a case for the really important stuff - proactively maintaining the underlying complex of functional capability that supports business activity. We limp on until it breaks, because it is just too hard to disentangle.

Shortage of skills is partly rooted in the other problems. Huge projects and a growing burden of legacy soak up good people. The way that we structure IT exposes a lot of the underlying technology, and encourages ambitious technical solutions, which increases our need for skills.

This is only the briefest of summaries. Look at the diagram for a more complete write-up. I am not suggesting that everything on the diagram is necessarily correct, but to me the weight of argument shows that there is a case to be answered.

On our heretical journey, we have seen that structures in IT are based on historical engineering necessity, and that this contributes significantly to the major problems of IT. Our current management approaches, effective though they are, do not fix the underlying problem and can entrench it by increasing our tolerance.

Next week I will suggest what we need to do to break out of these problems.

Subscription