Sunday, January 20, 2008

Why is Software Development on the Web so messed up?

It's been said that those who do not learn history are doomed to repeat it. In the spirit of absolutely not wanting to go again through the excruciating pain of software development that was the norm in the '90s and the first 5 - 6 years of the 2000s, I am going to dissect the following historical document: XML, Java, and the Future of the Web, published on October 2, 1997, by Jon Bosak.

As I walk through some of the salient points exposed in that portentous documents, I will try to bring to your attention certain glaring errors in understanding and judgment regarding what is the goal of computing and of the information processing on the web, in particular.

Let us then begin with the opening sentence:

"The extraordinary growth of the World Wide Web has been fueled by the ability it gives authors to easily and cheaply distribute electronic documents to an international audience."

Right out of the gate, Mr. Bosak takes the left turn and commits himself to the implementational aspect of the technology he is discussing. The key concept, of course, is 'electronic documents'. In his view, the web is useful in its ability to offer an affordable distribution channels for prefabricated electronic documents.

The problem with the Ivory Tower views of the web is that, being high-tech bent, the high priesthood inhabiting the Ivory Tower tends to view the smallest unit of processable information as an object, such as electronic document.

Following at the heels of the opening salvo, the second sentence in this paper claims:

"As Web documents have become larger and more complex, however, Web content providers have begun to experience the limitations of a medium that does not provide the extensibility, structure, and data checking needed for large-scale commercial publishing."

This is now quite confusing. If the smallest unit of information processing on the web is an electronic document, what need for extensibility, structure and data checking is he talking about?

Let's look at the third sentence; maybe we'll find some clarification in there?

"The ability of Java applets to embed powerful data manipulation capabilities in Web clients makes even clearer the limitations of current methods for the transmittal of document data."

Huh? Java applets? OK, this is completely obsolete, but still belies the unexplained bias toward powerful data manipulation capabilities in Web clients.

In the fourth sentence (and opening a new paragraph), the author introduces the concept of Extensible Markup Language, and explains:

"... an Extensible Markup Language (XML) for applications that require functionality beyond the current Hypertext Markup Language (HTML)."

What is not clear yet is what kind of "functionality beyond the current Hypertext Markup Language (HTML)" is he referring to?

HTML -- imaginary limitations

After delivering the opening salvo, as exposed above, the author goes on to elaborate on the severe constraints that HTML introduces and how debilitating these constraints appear to him. Here is the author's list:
  • Lack of Extensibility
  • Absence of Structure
  • Inability to perform Validations

Extensibility

This feature is the holly grail to developers who are keen on creating their own, proprietary protocols. The much celebrated power that Extensible Markup Language (XML) brings to the table -- namely extensibility, is at the same time the most troublesome aspect of the failed attempts to engage in collaborative computing. If anyone and everyone is free to use the technology that enables them to daydream any protocol they wish (in effect creating their own unique language), the Babilonian confusion created that way must quickly reach cosmic proportions. This is exactly what's happening right now on the web -- everyone feels entitled to dream up their own bloody protocol, and the potential online conversation tends to fall mostly on deaf ears.

The author states this clearly: "HTML does not allow users to specify their own tags or attributes in order to parameterize or otherwise semantically qualify their data." The key phrase is "their own" -- which amounts to "invent your own language".

Structure

Similar to the old adage 'form follows function', structure typically follows the meaning of the information. Mr. Bosak's lament in this respect is largely unfounded, since HTML does offer fairly comprehensive structure for expressing the semantics of the represented information.

Whether the author was aware of that ability to express semantics using HTML, remains a bit of a mystery. In any event, what the author seems to be complaining about with regards to the lack of structure in HTML is the erroneously perceived need for representing complex hierarchical dependencies.

There is really no evidence that an average consumer perusing the web resources has such a pressing need for consuming large and increasingly complex hierarchical constructs. In author's own words: "HTML does not support the specification of deep structures needed to represent database schemas or object-oriented hierarchies."

Yes, the above is true, but the question is: why would anyone have the need to consume 'deep structures, object-oriented hierarchies, or database schemas'? The question, of course, remains unanswered.

Validations

Let's examine Mr. Bosak's objection regarding HTML's inability to perform validations: "HTML does not support the kind of language specification that allows consuming applications to check data for structural validity on importation."

I must admit that I'm a bit confused -- why would anyone use a semantic markup language, such as HTML, to perform validations on 'importation'? Importation to where?

Responsibilities of the Web Client

The author then enlists some imaginary responsibilities that are, in his view, mandatory for the web client:
  1. Web client may have to mediate between two or more heterogeneous databases
  2. There is a need to distribute a significant proportion of the processing load from the Web server to the Web client
  3. Web client often has to present different views of the same data to different users
  4. Intelligent Web agents will attempt to tailor information discovery to the needs of individual users
The above list belies a strong desktop-centric approach to software development. All of the 'needs' listed in the four items above fall naturally into the server-side jurisdiction. The client is only responsible for rendering the representation that the server side business logic prepares.

Linking

Toward the end of his paper, Mr. Bosak addresses some perceived inefficiencies that he sees in the hypertext system as implemented using HTML. His list is, again, very revealing. Here is his list of perceived inefficiencies when implementing linking via HTML:
  • HTML lacks location-independent naming
  • It lacks bidirectional links
  • It lacks the ability to define links that can be specified and managed outside of documents to which they apply
  • No N-ary hyperlinks (e.g., rings, multiple windows)
  • No aggregate links (multiple sources)
  • No transclusion (the link target document appears to be part of the link source document)
  • No attributes on links (link types)
Below is my response:

Location-independent naming: Huh? If I link to a resource representation, such as google.com, how's that location dependent? Where in that hypertext link do we see any information that would reveal location of the resource representation named google.com?

Bidirectional links: This is the engineer's wet dream; having links that will never get broken. Not gonna happen, time to grow up and deal with reality.

Links that can be specified and managed outside of documents to which they apply: It's the damned documents again! Snap out of the document dogma and start living in the world of resources and their representations. In other words -- content!

N-ary hyperlinks (e.g., rings, multiple windows): Not sure what this means, but I really cannot see any real life use for it.

Aggregate links (multiple sources): OK, 10 years ago no one apparently knew about real simple syndication and such, plus the aggregators etc.

Transclusion (the link target document appears to be part of the link source document): Also, no clue about the mashups that will become all the rage 10 years later...

Attributes on links (link types): What on earth does that mean? Link types? Like what?

Giving Java something to do!?!

Finally, the author cites the slogan "XML gives Java something to do" (no sources quoted in the original paper).

What does he mean by this battle cry? What would he envision Java will do with XML?

It's hard to say after more than 10 years had passed since that battle cry was uttered, but my hunch would be that he was referring to the 'nomadic' nature of Java. In the olden days, Sun Microsystems was selling the "network is the computer" slogan, and so at that time Sun's trailblazing language was Java. According to Sun, Java lives on the network, and is nomadic, courtesy of RMI (Remote Method Invocation). Snippets of Java code can travel the network, on demand, in order to supply just-in-time computations on the requesting client.

Supposing that the client had just received an XML document but doesn't know how to present it to the user, a snippet of Java code could be invoked and downloaded on a whim. Once installed on the client, Java routine will massage the XML etc.

This dated model of 'distributed computing' has long been pronounced dead and buried. The problem is, we still have armies of software developers who tend to think in a similar vein. Because of that, software development on the web has turned into a terrible chore, and the resulting web sites tend to be nightmarish.

The moral of this story is to learn the lesson from the history, become aware of the blind alleys that have been paved by the developers a decade ago, and learn how to avoid the minefields, pitfalls and traps of the obsolete software development practices.

You're welcome!