XML-based programming 4: testing

By Andrew Clifford

XML provides excellent capabilities for building test harnesses and test data, and helps structure systems to make them easier to test.

XML brings a lot of flexibility and consistency to systems design and programming. However, the greatest advantage I have found to XML-based programming is that it makes it much easier to develop really thorough tests.

In using XML for testing, I was keen not to reinvent the wheel. Ant is a superb tool for running tests, and JUnit is excellent for low-level unit tests.

I started using XML for tests that could not readily use JUnit. Our XML framework uses services that take request parameters and return results as XML. To test these, we developed a simple test harness that calls the services using XML, and compares the responses with expected results. This approach allows all the business logic to be developed and tested independently of the web application that ultimately uses it.

Since all the data, both in files and on the database, can be represented as XML, all required test data can be held using the same formats as the test scripts. This allows us to create complete XML-based test packs, without using other technologies such as SQL or Java programming.

Some testing is more complicated. For these, we use XML-based macros developed using XSLT.

For example, we use the macros to test the online web application. We use a low-level Shell service to execute external programs, and we use macros to encapsulate utilities that get and process web pages (wget, curl, tidy), for extracting data from the results (XSLT) and for compares (diff).

On top of these low-level macros, we use additional macros to get web pages and capture results, or compare all or part of the page with expected results. And on top of these macros, we use even higher level macros such as "Follow Link" or "Submit Form" that mimic user interaction.

This allows us to build complete test packs for web applications, using the same tools and technologies that we use for testing individual components. We now have a fully automated UI test which runs through 500 page retrieval and compare steps.

As well as running tests and creating test data, XML helps to make systems more testable.

XML can represent all the data at any point in the processing. This allows complicated processes, such as database extraction, data analysis and reporting, to be broken down into separate steps, connected with XML. This makes it easier to test each step in isolation (and also makes the steps reusable).

Wherever testing is really hard, it is possible to break it down further by using an intermediate XML data structure part way through processing one step, allowing different parts of the step to be tested independently. This makes it feasible to test very deeply without the explosion of combinations that you would get if, say, you tested from data in the database all the way to finished reports in one test.

Good testing is never easy, and XML does not make testing magically easier. However, it does make it possible to build thorough test packs for all parts of the system, using a consistent set of tools and technologies. I could not now imagine testing a complicated system without XML.

Subscription