Advanced and specialised development tools may be unsuitable for complicated functionality because of difficulties with testing.
I like testing. I try to write tests before I write code, and even when I do not, I make sure I have automated tests for everything before it goes live. Development sometimes take a little longer, but I feel this emphasis on testing is the only thing that guarantees product quality. We have invested enough in our testing to be able to completely test a new version of the system in about one hour. This helps us respond quickly to new requirements, and lets us compete with businesses many times our size.
Last week, our development came to a complete halt because we had absolutely no idea how to test what we were writing. It took us a couple of days to work around the problems, and it made me realise that there could be a similar problem with many other development tools.
We were developing rather complicated analysis routines for our Metrici Advisor service. We write this type of analysis using the XML processing language XSLT. Mostly, this works very well. It handles complicated data sets easily, and allows us to develop with relatively little code.
It is usually easy to test XSLT by running files of XML through a command-line processor. But there can be problems. Although you can modularise the code, you can not run these modules independently. XSLT is entirely driven from its input, so to test each condition you have to craft complete input files.
This is not a problem with simple code and data. But as code and data get more complicated, you need to break the tests down into smaller unit tests. But this is impossible in XSLT which expects complete input data files.
After a couple of days scratching our heads, we solved the problem by using some add-ons to XSLT which allow us to pass test data directly into the modules, without creating large input files for every test condition. This halved the amount of code we needed, and reduced the amount of test data and scripts significantly. Without the add-ons, it would have been virtually impossible to test the code properly.
This problem is not limited to XSLT. It is a more general problem with advanced and specialised development tools. I can think of two requirements for any tool for developing complicated functionality, which these tools sometimes omit.
First, they must allow code to be modularised. A drag and drop development environment may look good, but if you can not create your own modules or objects then they are unsuitable for complicated functionality.
Second, it must be possible to test the modules or objects in isolation, without running a full instance of the system. You need to be able to run individual tests for individual pieces of code with little bits of test data, and build these up to a higher level.
Many modern tools fall short of these requirements. They can still be valuable, particularly for building rich interactive systems quickly. But they may be the wrong tool for building complicated functionality, because, as we found, it can eventually become impossible to test the system properly.