or, how I learned to stop worrying and love the XMLHTTPRequest…

This is the first part of what will become an ongoing series.

If you’ve built a website in the last few years, most likely you’ve adopted an architecture similar to Model-View-Controller, or MVC. If not, well, either your website is terribly simple, you haven’t had to modify it yet, or your code is spaghetti and you should be fired. Just kidding. (Or maybe you’ve come up with an even better architecture, in which case you should share your insights with us mere mortals.)

In MVC architecture, the model reads and writes data to and from a back-end data-store, and organizes the relational data in a nice, hierarchical fashion to be used by the controller. The view accepts input from the controller and generates output HTML, XML, RSS, JavaScript, SVG, PDF, or whatever you want to send to the user’s browser. And the controller accepts browser input, figures out what to query the model for, and picks which view to use and what data to send it.

This separation is nice because changes to the database engine or relational structure are isolated from the output generation, and adding new output formats for the same data or radically modifying existing output generation is similarly isolated from the database queries. Although controllers sometimes simply pass database results through to a view, the controller stage is important because there is very often idiosyncratic processing that has to happen between user input and database access, and between results from the database and output generation.

There you have the state-of-the art in website architecture, circa 2002. But what happens when you take an existing MVC website and turn it into a responsive AJAX site by adding a bunch of JavaScript to update the page, via XMLHTTPRequest¹, without refreshing the page? You have to add views that generate XML as well as HTML, often for the same queries. You have to generate or modify HTML browser-side by accessing the DOM with JavaScript, and often this HTML must exactly match the HTML generated for similar data by your server-side view.

Suddenly half of the benefit of using MVC is gone; your output HTML generation is no longer nicely encapsulated in a single set of views on the server side; it’s spread out across two separate parts of the code, which use two different languages (JavaScript and whatever you wrote your web app in), two different kinds of input (XML from the XML HTTP response or native language objects from your controller), and two different ways of generating HTML (DOM or whatever templating language you use), and you have more JavaScript to test on different browsers. Yuck.

With an architecture like this, you might as well go back to the good old days when web apps mixed SQL generation, SQL queries, and HTML fragment generation in a single function. Yee-haw!

How can you clean this architecture up? First, focus on what you can’t avoid. If you’re building a responsive AJAX web app that uses XMLHTTPRequest, you cannot avoid generating HTML with JavaScript. What would happen if you moved all of your HTML generation to JavaScript? The views on the server-side would generate only XML responses for XML HTTP requests from JavaScript. In the browser, your app would consist of static HTML and static JavaScript that queries the server for data when the page loads, and whenever user action requires it.

Suddenly your HTML generation is nicely encapsulated in a single place again. And the “view” in your MVC web app generates only XML. From the browser’s point of view, the JavaScript is a kind of model, querying a back-end engine (the web server) for data and massaging it into a form that the controller (the HTML) can display.

Now, this architecture is a bit idealistic. You’re not going to want to give up your powerful server-side templating language and do all your HTML generation in JavaScript with the woefully inadequate DOM interface (or worse, with document.write and .innerHTML). You’re still going to have to generate formats other than HTML server side, like PDFs or RSS feeds. You might want to use the JavaScript HTML generation pattern for some of the more dynamic features of your site, and server-side HTML generation for other, more static, features. But MVC is an idealistic goal too — even the best-compartmentalized MVC apps probably have bits of controller code sprinkled in views and models. Once you start down this road, you’ll find it harder and harder to justify server-side HTML generation at all.

When I stumbled across Google’s AJAXSLT a few years ago, I wondered why on earth anyone would take the time to implement the (bizarre) XSL templating language in JavaScript (also not my favorite language). Now I know. It lets you generate HTML in JavaScript based on XML responses from a web server. You could even use AJAXSLT to fetch XSL templates from a server, and if your server-side views were in XSLT too, you’d be able to use the same XSLT to generate HTML on the browser and server sides.

Another solution would be to keep generating xHTML server side, and write JavaScript to copy that xHTML verbatim, from the XML HTTP response, into the DOM. Either way, you still have JavaScript updating static HTML.

If you’re still doubting that this is the way to go, go to Gmail and view the source. It’s almost all JavaScript. None of the buttons or links that you see in the page are actually in the source HTML — they are all generated on the fly by JavaScript. This is not to say that Google’s always right, or that they are doing exactly this; it just shows that it is possible to write a complex web app where the HTML is generated by JavaScript.

There’s one final missing part to this parallel between MVC on the server side and JavaScript model and HTML controller on the browser side. What’s the analogue of view on the browser? It’s supposed to be CSS. The HTML is supposed to represent the abstract, hierarchical nature of the information being displayed. And just as the controller feeds hierarchical objects to the view, the HTML provides hierarchical data for the CSS to style.

Separating CSS into layout/spacing CSS, and color/background CSS makes it easy to allow users to choose, or make their own color themes without royally screwing up layout.

Unfortunately, CSS is still woefully inadequate for this job, but it’s nice to dream, isn’t it?

In the end, this isn’t so much of an AJAX vs. MVC deathmatch as it is a JavaScript-CSS–HTML meets Model-View-Controller hippie love-fest. It is possible to retain the modularization of the Model-View-Controller pattern and still write a modern, responsive web app, by extending the MVC pattern into the code running on the browser.

1. For simplicity, I’m going to ignore JSON, <iframe>, and other ways of communicating with the server without refreshing the page. However, the same problem exists: generating some HTML in JavaScript, and some on the server, is messy. Also, JSON is ugly and insecure. [↩]