14 February 2012

A change of heart and a bright idea

By Andrew Clifford

Is there demand for a standard library for server-side JavaScript?

Many applications embed programming languages to let "power users" add extra functionality.

In version 1 of Metrici Advisor, we embedded the XML processing language XSLT for user-defined analysis and reporting. This made a lot of sense. XSLT works well in our XML-based architecture. XSLT is very coherent, simply transforming an XML input to an XML output. It is non-procedural, which means you can do a lot with a small amount of code.

XSLT does have drawbacks. Its main problem is its weirdness. It is very different from other languages, and relatively hard to learn. We therefore had a change of heart, and for our current version of Metrici Advisor we decided to go for simplicity and familiarity rather than architectural fit and purity. Instead of using XSLT, we embedded JavaScript, using the Rhino JavaScript engine. (This is JavaScript running on the server, not the more familiar use of JavaScript running on a browser.)

JavaScript works well as an embedded language. It is easy to write simple scripts, but also supports more advanced programming. We use it for general scripting, validation, calculating derived data, creating new UI features, analysis, reporting, data triggers, and creating reusable services.

JavaScript does have drawbacks. Because it is a very permissive language it is easy to write unstructured code. There is a simple API into Metrici Advisor's data structures, but navigating the data requires a good knowledge of the system's internals. We have created analysis and reporting libraries using JavaScript, but it is hard to keep use of these completely consistent. JavaScript can easily get a bit messy.

This had been in the back of my mind for some months, then last week I had one of those lightbulb-over-the-head moments.

For browser-based scripting, which is more complicated, various JavaScript libraries have been created to simplify programming, of which jQuery is the best known. In jQuery, the difficult parts of programming have been simplified around a small number of concepts. The core concept is selecting a set of elements from the HTML document, which can then be used as the basis for further processing. jQuery has a well-defined plug-in architecture, which allows it to be extended to cover all manner of high-level features.

The bright idea, obvious in hindsight, is to create the equivalent for our server-side use of JavaScript. We might call this "nQuery", where the "n" stands for "node", which is our basic data unit. Using similar semantics to jQuery, nQuery could be used to select and process server-side data. We could deliver analysis and reporting components as nQuery plug-ins. nQuery would make the environment more familiar for programmers experienced in browser-based JavaScript. nQuery would bring coherence and power to our JavaScript, without the weirdness of XSLT.

Creating "nQuery" also opens up an intriguing possibility.

jQuery has become a de facto standard for browser-based programming. Something like nQuery, mapped to a generic data API, could be used much more broadly than as a library for embedded JavaScript in Metrici Advisor. Is there demand for a standard library for server-side JavaScript?