System governance: defining criteria in detail

By Andrew Clifford

Criteria used for system governance must be defined in detail. This includes their business significance, what questions you would ask, and model answers.

In IT, we often use lists of criteria: in requirements documents for new systems, as evaluation criteria in a system selection, or even just the headings for system documentation. System governance is based on criteria which reflect what is important to your organisation about IT systems

We often have problems with criteria. Different people interpret them in different ways. We give too much emphasis to technical areas we understand, rather than business areas that seem a bit nebulous. When we carry out systems selections, we get one line answers where we wanted an explanation, or pages of explanation where we only wanted one line.

To avoid these problems, define each criterion with the following:

• Name. Give every criterion a unique name.
• Description. Provide a short paragraph which explains what the criterion is about. Do not assume that anyone else looking at your criterion will understand what it means.
• Significance. Explain the significance of the criterion to the broader business.
• Question. Consider what questions you would ask to find out about this criterion.
• Answer criteria. Describe what you would look for in an answer to the question to make sure that the question has been answered.
• Weighting. Provide each criterion with a weighting which represents its relative importance to the organisation.

Business significance is really important to get support for managing to this criterion. Most criteria can be translated fairly directly into business value and risk. Some criteria just reduce technical risk or lower long-term costs, but the connection with business risk and costs needs to be carefully explained. Meaningless phrases like "IT best practice" often cover up unjustifiable pet subjects.

It is also useful to define grades which characterise different answers. There is a likely to be a grade for an ideal answer; an answer that is acceptable, but only just; and an unacceptable answer. There may be grades in between. Each grade needs:

• Name. A name or short sentence which sums up the grade.
• Description. A fuller description of when the grade should be awarded.
• Score. How "good" this grade is.

For some criteria, there is no right and wrong answer (all scores are 100%). For example, a criterion about data confidentiality is important because it alters management decisions (systems with confidential data need stringent security). However, of itself, there is nothing good or bad about data being confidential or not confidential.

Cross check the definition. Does the question really find out what you have described? Do the answer criteria tie back to the questions and the grades? Are the grades comprehensive?

Defining criteria fully is always worthwhile. When selecting a system, it focusses your mind on precisely what you are looking for, helps you ask the best questions, and gives you a defendable process for comparing competing options. When documenting systems, it gives you consistency.

System governance is a way of formalising this use of criteria, using them to define what is important about our IT, measure how well we do, and define the actions we need to improve. System governance demands well-defined criteria.

Subscription