29 April 2008

The next big thing has been cancelled

By Andrew Clifford

In the past, major shifts in IT architecture have hidden weak long-term management. With no prospect of a similar shift in the near future, it's time to tighten up our act.

When I started in IT, the hot technology was online transaction processing (OLTP). Businesses were moving from keyed data entry and batch processing to an exciting new online world. Batch systems were being rewritten to take advantage of new technologies like IBM's CICS.

OLTP went through a few generations. I spent a lot of time on fourth generation languages (4GLs). 4GLs were much more productive than COBOL, and we could build much richer, more usable systems. Many COBOL OLTP systems were rewritten in 4GLs.

Then along came PCs, and client/server. We used Visual Basic 3 and database gateways to replace those clunky character-based 4GL systems with new client/server systems.

PCs developed rapidly. 32-bit computing would get rid of memory addressing problems for ever. Application servers helped us replace the inefficient two-tier client/server architectures with 3-tier architectures. Early client/server systems were rewritten because of the benefits of the new architectures.

Then we discovered the Internet. We could reach customers and partners all over the world. We could centralise our systems again, back to our controlled data centres. We redeveloped our systems using exciting new technologies like CGI and PERL.

Web technologies matured, with a new generation of web application servers, new standards and new tools. We redeveloped in Java, and more recently, with frameworks like Struts. We used open source technologies like Linux. Those first generation web applications were so resource-intensive, so hard to manage, we just had to redevelop them.

Rapid technology change has forced us to redevelop every few years. It has both forced us and allowed us to take a short-term view of systems. Our main objective has been to deliver projects quickly and keep up with technology. Managing for the long term has been meaningless.

Will this continue?

The changes I have seen are the result of three big changes: online systems, PCs, and the Internet. Each of these has spawned a few generations of implementation technology.

In the future, there will continue to be significant technology improvements: processing, storage and networks will improve in performance and price, and technologies like virtualisation will increase flexibility.

For applications, however, I can not see a change anything as fundamental as online processing, PCs or the Internet. I can not see a "next big thing" that will force wholesale replacement.

This means we have to adjust our management. We can not assume there will be pressing technology drivers to replace systems. Management that focuses on big replacement projects, and does not consider the long-term management of systems, will seem as old-fashioned as the OLTP systems I worked on when I started.

We need to change our management. We need to excel at the swift execution of small incremental projects. We need to design our technology solutions to allow for piecemeal change, not wholesale change. We need to look after what we have better, with methods like system quality management.

We can't rely on the next big thing to hide weak management. The next big thing has been cancelled.