Effective selection: beyond the questionnaire

By Andrew Clifford

A good questionnaire is a really useful tool for technology selection. It frees the decision makers to focus on the most important parts of the decision.

I have often advised management on technology selection. But recently, instead of being an advisor, I have needed to make the final decisions on technology purchases for my own company. This has given me new insights into the evaluation and decision-making process.

We provide our main product, Metrici Advisor, as a service over the internet. We rent the servers on which it runs. We felt that our existing service provider would not be able to support our future plans, and decided that we needed to move to a new provider.

Perhaps ironically, Metrici Advisor is itself a tool for automating the advice process, so we used Metrici Advisor to structure the evaluation. We developed an evaluation questionnaire, with a weighting and scoring scheme. We created rules for Metrici Advisor's expert system to identify issues with potential providers.

It took us a couple of days to develop a questionnaire that covered all our requirements: connectivity, capacity, scalability, resilience, support arrangements, data centre protection, commercial viability and costs.

We interviewed potential providers by phone. Having a clear set of questions cut through the marketing fog, and we managed to get the answers we needed quickly. Even though the interviews took only 20 minutes, one provider commented that it was the most thorough set of questions they had ever been asked. Metrici Advisor quickly identified which providers could meet our needs and which could not. We ended up with a short list of three providers, all of whom had good scores and no issues.

As we went through the process, our ideas on what we were looking for developed. For example, a couple of the vendors suggested that our needs could best be met by using Xen-based virtualization.

As well as refining our solution, going through the process gave us a much better feel for the providers' technical competence, whether they were the sort of people that we could work with, and how much they wanted our business. These are all vitally important for making the decision, but hard to capture on a questionnaire.

In the end, we selected Goscomb Technologies. They were not the cheapest or the biggest, but we were really impressed by them, and felt that they were the sort of business we could work with very effectively. We have now started to move over to our new servers, and are very happy with the new arrangements.

Using a questionnaire helped a lot. It was a good basis for meaningful discussions with providers, helped us gather information quickly, uncover issues, compare options, and come up with a short list. (And using Metrici Advisor made the process even quicker and more effective, of course.)

But just as importantly, having a good questionnaire freed up our time and attention to focus on the most important parts of the decision. What is the best solution? Which businesses fit with ours? Who do we feel most comfortable with? By combining a formal evaluation process with a less formal exploration of the options, we have made what we believe to be a very good selection decision.

Subscription