Resource Oriented Architecture

The Age of Epic Battles Raging Over Location

2010-02-02T15:42:00.000-08:00

We are now beginning to see the signs of the raging battles over location picking up all over the place. By location, we don't mean physical land, water, or any other physical resource. Instead, we're talking about the physical location of an individual who is a potential consumer of resources.

Up until very recently (and I mean that literally, in the sense of saying 'up until a few years ago'), it was fairly easy to control the physical location where many of the coveted resources (i.e. products and services of the current economy, or cultural artifacts of the current civilization) were to be consumed. For example, just as devout religious practitioners are expected to congregate in a physical church or other places of worship, so were consumers of cultural goods, who expect to enjoy works of art or plays or concerts, expected to congregate in a physical location (such as a concert hall, or a theater, a museum or an art gallery, etc.)

Similarly, movie goers are expected to consume the motion pictures at a very specific physical location, such as a movie theater. Same goes for people who enjoy going to the circus, or to the country fair, etc.

The ability of the businesses (who produce and publish products to be consumed by the market) to fully control the exact physical location where the act of consuming is to take place, is the vital part of the profit making model that such businesses pursue. Thanks to the virtue of the fact that such physical locations are enclosed in wired fences and walls, with only a single point of entry where the business can charge the admission fee, the businesses can ensure that only consumers who pay the fair price get the privilege to enjoy the offered products and services.

What we're talking about here is the good old world of 'haves' and 'have nots', which is the time honored way that human civilization works. The elites (the 'haves') find themselves in a very fortunate situation where they are in the possession of an upper hand, meaning they have sufficient means at their disposal to go ahead and produce things and services that the underprivileged, the 'have nots', cannot produce. This arrangement then creates sharp and extremely lopsided inequality, where the underprivileged (who by far outnumber the privileged elite) are starving and craving for the products and services that the elite can offer. And the elite is indeed offering these thing for a fee, often by charging what we call the 'admission fee'.

For example, if I wish to enjoy the much ballyhooed movie "Avatar", I have to make a trip to the local movie theater, where I'll be expected to pay the admission fee before I'd be allowed to get in and watch the spectacle.

If I then, upon finishing watching that movie, leave very impressed and the next day decide I'd like to see that movie again, I must do the same thing all over again: make a trip to the movie theater (a physical location where that movie is playing), and pay the same admission fee yet again. The business that's running this operation (i.e. the movie theater) doesn't care whether I'm their regular patron or that's my first visit ever -- the admission fee is the same for everyone. No discrimination there.

That same model applies to many other businesses, such as museums, art galleries, circus tents, Disney Lands and Disney Worlds, theaters, concert halls, stadium, and so on. All those business ventures operate on the 'admission fee' model, which, of course, implies the necessity to make a trip to the actual physical location where the event is taking place. And once there, it would be extremely difficult to cheat and beat the system by gaining the entry without paying the admission fee. There are physical barriers that are legitimately erected for the exact purpose of preventing anyone from cheating and getting in for free.

If you were to pay closer attention to how does the above system actually work, you will no doubt notice that it is all based on the assumption that end user, the actual consumer, has no other ways or channels whereby he or she could consume the content. The consumer is forced to make that trip, to visit the designated physical location, otherwise he or she won't be able to experience the event.

Where this model breaks down is in the cases where the consumer gets to be sufficiently educated and self-reliant. For example, let's talk about a consumer who is literate. Armed with solid education and a healthy does of self-esteem, such consumer now gains extra freedom and can get a book and can then consume that book in ANY location. All of a sudden, we see that certain resources, such as books, magazines, newspapers etc. do not require people to visit an exact physical location before they can consume these products.

As the technology keeps maturing, we see more and more of such liberating instances being offered to the general public (to the typical 'have nots', the unwashed masses). The invention of radio, for example, with the supporting wireless technology, was one of the mold-breaking advancements that allowed the unprivileged masses to consume a play or a concert or a sporting event without having to make an actual trip to the physical location. Of course, the invention of television brought that liberation to an even higher level, where it was now possible to enjoy movies and cartoons and all other kinds of motion picture products, without being expected to make a trip to the exact physical location.

The trend was slowly becoming apparent: the more an individual is educated and the more the technology had matured, the less was that individual forced to make a trip to the specific location in order to consume an event. Conversely, those who lack education and access to the mature technology, had no choice but to make the trip and pay the admission fee, if they were to participate in the scheduled event.

For example, with the advent of the recording technology, people who have access to the playback equipment could obtain the desired recording and enjoy the performance in the comfort of their own homes. Or, in the comfort of their own vehicle.

The ones who were left behind were basically uneducated on how to set up and use the playback equipment. But as soon as they've upgraded their education, they too could then afford the luxury of enjoying the performance in their own homes.

The music recording industry didn't follow the model of the movie making industry. Instead of setting up music listening halls, where consumers could gain admission by paying the admission fee and then listening to the recorded albums, the music industry opted for manufacturing a product (a vinyl record) which is then being sold to the general public. This was a significant shift in the battle for location, as the control of the location has now shifted from a particular physical building or stadium, controlled by the business, to the individual homes. Just as long as an individual had legitimately purchased the product (the vinyl record), that individual was free to listen to it in his or her home. But the significant game changer here also was the fact that the consumer could consume the product more than once.

This game changer had, of course, opened a can of worms. Not surprisingly, the movie industry followed suit by issuing products that contained recordings of the movies and TV shows, suitable for playback in the comfort of the consumers' homes. No longer were the consumers expected to make a trip to the movie theater in order to watch the movie. And even more significantly, once they obtained the recording of the movie, they were free to watch it over and over, and thus enjoy limitless playbacks at no extra charge.

Such is the liberating power of the constantly maturing technology. What once was scarce is now abundant. However, not everything in this arrangement is positive and optimistic. There are flies in the ointment, and we'll examine them next.

The first proverbial fly in the ointment stems from the newly won privilege to enjoy the performance unlimited number of times, all for the same price. Under the old regime (the one that were forcing people to pay the admission fee each time they wanted to enjoy the performance), businesses were guaranteed repeat income from their consumers. Under the new regime (the one where people can enjoy the performance unlimited number of times for a single admission fee), this repeat income has been lost. Furthermore, thanks to the technological advancement, the consumers can now not only enjoy the performance unlimited number of times, they can also lend the product to others, who then get to enjoy it for free.

To curb that undesirable consequence of the maturing technology, the businesses came up with the hair-brained concept of residual income. What that concept states is that, even though someone can obtain the rights to enjoy the performance by purchasing the container (a record or a VHS tape or a disc) containing the performance, that arrangement still does not entitle the purchaser to further freely distribute the container in question. In case the original purchaser wishes to further distribute the artifact, that purchaser is obliged to pay the royalty fee to the maker of the product.

This royalty fee is a very problematic concept, as it is incapable of enduring any protracted logical scrutiny. In a nutshell, royalty fee is based on the concept of intellectual property. Unlike objective, tangible property, such as real estate, or vehicles, furniture, clothing etc., intellectual property is a very nebulous concept that gets applied in a wishy-washy fashion, and is thus entirely indefensible.

For example, if I have an idea for a song, and I then work on writing that song and recording it, once I publish that song, it is protected under the auspices of the laws governing the intellectual property. Anyone who wishes to use my song, either in its entirety or a portion of it, is requested by the law to first contact me and obtain the permission to do so, after which each subsequent use of my intellectual property is followed by the user paying the royalty fee to the owner.

In a way, the above arrangement is all fine and dandy, but the problem lies in the fact that it is governed by the double standards. And any time we have a situation where the law does not apply equally across the board, we know we have a serious problem on our hands.

To illustrate the inequality of the laws pertaining to the concept of intellectual property, suppose I'm an artisan cabinet maker. I get an idea on how to make a completely original chest of drawers, for example. I do it, and then I sell that chest of drawers. Where's the problem, you ask? Well, in case of a chest of drawers, my original idea, my intellectual property, is not protected by the law. In other words, the law claims that an artisan cabinet maker who expects to receive any residual income after the initial sale of his product is simply dreaming.

But why? Why am I, as a creative individual who happens to make armoires and shelves and chairs and tables and chests etc. not entitled to the legal protection of my intellectual property, while a book writer or a composer etc. are entitled to it?

The answer to the above question is very simple: the technology hasn't been invented yet that would enable people to create an exact replica of my chest of drawers with a simple push of a button. Hence, I have no right to ask for the legal protection, because I am already protected simply thanks to the immature state of technology. Once we reach the stage where it will be cheap and easy to create an exact replica of the cabinet maker's products, the law of intellectual property will surely come into full effect.

So it's easy now to see that the law of intellectual property is a very arbitrary concept, as it hinges entirely on the ability of people to make cheap and easy copies of the products that embody the intellectual property. Those products that cannot be copied with a minimal cost are exempt from the legal protection.

This inconsistency is the Achilles' heel of the law of the intellectual property. Because of that, the entire setup is invalid and illegal.

On to the second proverbial fly in the ointment. All the existing legal constraints that have been formulated around the concept of intellectual property are built on the presumption that the product in question is going to be consumed at the predetermined location. In other words, either the consumer will be expected to make a trip to the theater, a concert hall etc. before she can enjoy the performance, or she will be privileged to enjoy the performance in her own home. This arrangement is bound by the sheer physicality of the location delivering the actual performance. Either the location is a physical public building (such as theater, or a stadium, etc.), or it is a physical residential building. The only third category of physically defined location where the performance could be enjoyed is a private vehicle, a car.

Furthermore, the carrier of the performance is assumed to be a physical object, such as a book, a magazine, newspaper, vinyl record, magnetic tape, or a digital disc.

But what if the content can be delivered without requiring a container? For example, I can today stream digital music wirelessly to any location of my choice, and furthermore, I can stream several identical recorded performances to several location of my choice at the same time. Suddenly, the concept of the location starts to lose it's meaning.

The battles over the location of the performances to be enjoyed by the paying customers have traditionally been fought by erecting barb wires, walls, and placing the bouncers around the physical locations. With the maturation of the technology, stringent laws have been put in place to protect the unauthorized playback of the copy-protected recorded performance. These copy-protection laws all hinge upon two things:

There is a physical object, a container that carries the recorded performance
There is a pre-set physical location where this recorded performance will be enjoyed

The above presumptions have now been invalidated thanks to the unstoppable march of technology. The first presumption (that there always must be a physical container carrying the recorded performance) has been rendered meaningless by the wireless streaming of the content to any chosen location. I can store my content on the Amazon S3 service, that exists somewhere in the cloud, and I can then easily stream that content wirelessly to any location on the planet. That content is not bound by any physical container, as its actual location is indeterminate, and its delivery is wireless, intangible.

The second assumption (that there is invariably a pre-set physical location where the recorded performance will be enjoyed) has also been invalidated by the maturation of the technology. I can travel to Sao Paulo, Brazil, visit any Internet cafe in the city, and right there stream my content and play it back while I sip my latte. No constraints of the physical container carrying the recorded content, no constraints on where can that content be consumed.

In conclusion, the epic battles over the location have now been rendered meaningless. It remains to be seen how will the intellectual property bullies attempt to deal with this new reality of containerless content and locationless location.

No One Wants To Visit Your Web Site

2009-11-13T15:39:00.000-08:00

As we've seen in my previous post (Content and Containers), people at large have lost interest in containers. As soon as anyone notices how easy it is nowadays to get any desired content delivered right into their browser, they drop any interest in any other containers.

Why is that? Two things: affordability and convenience. It tends to be much cheaper (read: almost always free) to get your content via the publicly vetted general purpose network (i.e. the world wide web), then it is via some proprietary delivery channels (such as your local newsstand, your local cable, etc.) On top of that, it is incomparably more convenient to enjoy the desired content when it suits the consumer, and so this 'on demand' delivery is like a dream come true. Once people experience this kind of freedom, no one in their right mind would ever want to go back to the old, proprietary and costly way of consuming content.

People are therefore abandoning proprietary containers in troves, and are embracing the 'free range' content as it lives on the public web. There is one peculiar detail that seem to have fallen through the cracks, though: no one seems to be aware of the fact that, talking about proprietary containers, a web site is also a proprietary container. Just as a TV cable provider wants to lock customers in and turn them into hostages by disabling them to even begin looking for help, web sites want to lock their visitors/members into their proprietary dungeon, looking for tricks that would prevent them from ever leaving.

However, one thing that the purveyors of proprietary web sites seem to gloss over is that their consumers visit their proprietary sites only because they tend to find the content they're interested in hosted within the bowels of that site. The consumers don't really care about the container that delivers that content.

That sentiment explains why have RSS feeds become so popular. It's the content, stupid! No one cares about your site, no matter how much effort you've invested in prettying it up, choosing the right fonts and font sizes, the right layout, the right colors etc. Who cares? The only thing that matters is the content found on your site!

But guess what: that content is in no way locked in on your site. Other sites can graze the content of your site, slurping it up, repurposing it, making it even more attractive for the consumers than you've ever dreamed possible.

So you, as the web site creator and host, are now faced with two choices: resist and die a horrible death of attrition, as your consumers abandon you, or open up, and die a nice, pleasant death of being swept away by the tsunami.

The choice is yours. But remember -- no one in their right mind wants to visit your site!

Content and Containers

2009-11-13T15:22:00.000-08:00

It wasn't that long ago that content was, in general, scarce. I still remember those long gone summers at my grandma's farm house, and how eagerly I used to await Fridays when I'd get to read my latest comic strip periodical. Those were the days when content was scarce, the medium was expensive, and the attention of the targeted audience was guaranteed.

Today, we're all of a sudden faced with the exact opposite situation: content, all kinds of content, is copiously abundant, the media is super cheap (actually, it's practically free), but the attention of the targeted audience is virtually non-existent.

Our ages old fascination with containers (i.e. the carriers of content, such as newspaper print, radios, TVs, VCRs, record players, CD players, DVD players etc.) has now taken a serious nose dive. No one cares anymore how the content gets delivered. We have reached the point where we simply want to consume the content, on demand, at the very moment when we feel like it.

And modern technology is delivering precisely that -- content on demand. If I wish to watch an episode of my favorite TV sitcom, I don't have to check the TV Guide anymore, and wait for the episode to air. I simply watch it on the web, on demand, online.

This is a very essential and a very significant change in the way we traditionally used to consume content. While up until very recently it went without saying that a desired content can be delivered for our consumption only via expensive containers, today we've grown accustomed to the fact that the container doesn't matter anymore. Whether we wish to read the latest news, or a timeless book, or listen to the radio program, or watch TV, or listen to music, watch a movie and so on, we simply get that content delivered directly to us via the web. It's all digital now, so the container doesn't matter.

The moral of the story is that anyone who is planning to make money by charging people for using their proprietary containers is on a fool's errand. These days are gone, and today no one is fascinated with the containers.

The Terror of Cascading Style Sheets

2009-10-20T11:53:00.000-07:00

I've been involved in designing and building hundreds of web sites throughout my career. There is one common red thread that always seems to be present on any project that involves building and launching a web site: that thread could be summed up as 'layout and styling management'. The reason I'm mentioning that thread is to illustrate how, of all the other activities related to building/launching a web site, that one is the least predictable. However, at the same time, the layout and styling always gets buried in the mix, so that no one seems to be aware of the issues surrounding that activity.

Now, if we were to break down the effort needed to build and launch a typical web site, I'd say that, in my experience, the layout/styling efforts can unapologetically eat up anywhere between 15% all the way up to 60% of the overall time and effort. Now, that's a significant chunk of resources that somehow never gets properly acknowledged. But in reality, and in hindsight, we all know that there always seems to be a lot of fiddling with the cascading stylesheets (CSS).

For some reason, no one seems to mind this wastefulness, but guess what -- I do! Rather than burning untold hours on making sure that certain CSS trick will work in certain versions of Microsoft Internet Explorer, I'd like to simply launch the site and move on to doing more lucrative things.

But the issue is with the CSS police (or, as I like to call them, CSS fascists). They insist that CSS is god, and it must be served and worshipped. So my question here is: why? What is it about CSS that should command a lot of our time, effort, and money? Why have we enslaved ourselves and found ourselves serving the gods of CSS? What could be realistically gained from this ridiculous system of slavery?

My answer is: nothing could be gained. We have merely fallen victims of some witless, clueless hackers who are terrorizing us with their oh-so impressive CSS skills. Who cares? And, oh, by the way, those hackers should get a life! Or at least get a girlfriend.

"But what's the alternative?" those who are more observant among you might ask. The answer is dead simple -- the alternative to CSS is NO CSS! Just abandon the crappy thing altogether.

"But, but, but, then our web sites will end up looking ugly!" I can hear you retort. Well, here's where the interesting bits come -- I've been lately launching a number of web sites that contain zero CSS code, either as separate CSS files or inline. The site's presentation is rendered by the browsers that strictly read the semantic markup and render the content in a knee-jerk fashion.

Guess what -- the customers love it! No one ever called to complain how the site is missing that inexpressibly lovely pale coral red background color, or what have you. Time to wake up, people: no one cares about that crap! No one is going to stop and marvel at the sublime colors, fonts, and layout decisions you and your team of designers have poured over for months (minor correction: no one except those same CSS fascists).

This is the ugly truth in today's world of web sites. Web designers of all colors will vehemently deny this, of course, but don't let them bully you. My advice is: be brave, be bold, be radical, lead the pack and release that site with no styling whatsoever. See what happens. And see how much money and time you save by firing those bloated designers and, by proxy, reducing the bloat of your site.

And in case you get cold feet, just placate yourself with these thoughts: "Hell, we can always add the facelift once the site is up and running; that's always a no-brainer. Throw in some colors, some layouts, fancy fonts, whathaveyou. But first, let's see if there are any detectable issues with the bare bones site."

Look at it this way: most people go on the web with a specific question in mind. All they are looking for is an answer to the specific question. And they really don't care about the appearance of the answer to that question, so long as the answer was easy to obtain.

For example, if I have a flight tonight, all I'd like to know right now is whether my flight is on time, or delayed, or cancelled. This information would be super important to me right now, wouldn't you agree? I'd like to be able to simply plug in my flight number, and get back one of the three possible answers:

The flight is on time
The flight has been delayed for x hours
The flight's been canceled

Would I really care if the answer comes back to me in a certain font? No, no way! Would I get upset if the answer is not styled at all? Again, no way!

Time to wake up and smell the roses, folks. This may hurt for you to hear, but people in general don't care about visiting some dinky web site. All people want is the ability to ask a question and get back the correct and legible answer in less that 2 seconds. The appearances don't matter one bit.

Now that we know how things really work out there in the big bad world of world wide web, time to get back to the office and fire those pesky CSS gurus.

Can We Afford to Lose the Web?

2009-08-14T10:50:00.000-07:00

Things are funny the way they tend to evolve. For example, common wisdom holds that hardware is the underpinning of software. However, in the recent years, we've seen software being used as an underpinning of hardware (in the form of virtual machines being hosted inside a software program).

Or, in the early days, one could have argued that the economy is the underpinning of literacy. In other words, it was thanks to the economy that some people were in the position to learn how to read and write.

Today, the situation is exactly the reverse: literacy is the underpinning of the economy. Being illiterate ensures that you cannot participate in any economic activities.

To go back to the question asked in the title of this post: "can we afford to lose the web?", the answer is "yes, provided that we can afford to lose our civilization."

In a manner similar to the above examples, civilization used to be the underpinning of the web. Meaning, without the civilization, the web couldn't have emerged.

But today, the evidence seems to be accumulating that the opposite is true, namely, without the web there couldn't be a civilization.

Very strange, but that's how things are. Of course, there is, as always, a whole whack of naysayers, people who are denying any significance to the web. Some of them are simply knee-jerk, some come with certain hilarious arguments. I will present here one interesting counter-argument that I've found while perusing some mindless discussion among some techno-geeks who seem to revel in having no life whatsoever (in case you're wondering why am I lurking around these terrible sites, I am merely doing a little bit of social study, observing bizarre behavioral patterns in the wild):

Any, ANY, website could go down - hell the entire internet world wide web could blow up - and I'd be fine. Sure I'd have to drive to the bank (where... omigod... they use desktop software). But I'd survive, just as well. In fact, I think that my boss and my wife would both be happier.

Bottom line: No website matters to my well being (or anyone I know of for that matter). I'm happier to have them, but could go on just as well without them. The only people who care, are the people who make the sites, and the people who have nothing better to do with their time.

I've withheld the name of the clueless person above, because it may not be his real name. Other than that minor edit, the entire quote is taken verbatim off the discussion thread.

Let's now examine the interesting points he brings up. The first point is the self-centered survivalist argument ("Any, ANY, website could go down - hell the entire internet world wide web could blow up - and I'd be fine.") This is reminiscent of the claims made by some survivalists who organize their lives around the assumption that the end of civilization is near, and so they are fully prepared for the power grid to go down, they can still be OK without electricity. Thus, we can afford to lose the web providing that we're fine with losing civilization.

He then goes on to say that his boss and his wife would be happier if indeed we were to lose our civilization. Of course, if we were to lose many of the trappings of the civilization, such as TVs and so on, that distract us daily, we'd be left with more free time on our hands, free time which could be more readily exploited by our business and personal partners. Does it necessarily follow then that all these trappings are unnecessary?

He concludes with "no web site matters to my well being." With such blatantly self-centered conclusion, he then triumphantly dismisses any need for having the world wide web. How brilliant! It's like an extremely selfish person, who has no children, wanting to abolish all medical professionals who specialize in healing sick children because, hey, he doesn't need them, since he doesn't have any children.

The Web Continues to be Disruptive

2009-03-31T13:39:00.000-07:00

When the web hit the mainstream some 15 years ago, it was largely regarded as being a massive electronic library. These were the days of the 'read web', and this read web was considered as an extension of the then leading edge technology: CD ROM.

The web quickly evolved into the read-write web, whereby it opened up to accept user-contributed content. At that point, content was king, and many online business models have been centered around that notion.

Fast forward to today, when the so-called social, or participatory web had clearly demonstrated that conversation, not content, is king.

All along the way, the web has proven itself as being the most disruptive and the most subversive technology ever. And it continues to roll into the future as the most surprisingly disruptive platform there is. There is hardly a way to envision how could this supremacy of disruptiveness ever be interrupted, or supplanted by any other platform.

A case in hand to illustrate the unstoppable subversiveness of the web: a well known online vendor, 37 signals had made their success riding the subversive nature of the web. Back in the early 2000s, the company had embraced the social aspect of the then budding participatory web in order to harness the ground swelling support of individuals/businesses in the light of the fact that the existing software productivity applications suck big time. They have boldly utilized the small footprint, lighthearted nature of the web by offering small, nimble web applications powerful enough to replace the bloatware that is Microsoft/IBM/Oracle etc. productivity suites.

The risk paid off, and 37 signals made their name thanks to their unreserved embracing of the subversive and disruptive nature of the web.

But the web didn't reach its pinnacle of subversiveness with 37 signals. The web continued to roll with its unstoppable disruption. Today, we have a comical, nay hilarious situation where the same business that once praised web's openness and subversiveness is crying foul for the same reasons! Today, 37 signals complain how it is unfair that other budding businesses and communities, who are embracing the conversation-is-king mantra, are disrupting their precious business (Get Satisfaction or Else...)

The thing that allegedly hurts 37 signals is the fact that they cannot seem to be able to lock their customers down into their proprietary web site. They'd like their customers to recognize that they don't have to go anywhere else on the web in order to get the service they're hoping to get. However, communities such as People-Powered Customer Service have further embraced the disruptive nature of the web and have shown the audacity to elaborate on the existing model (i.e. they've embraced the remix culture of the web). They've accomplished that by extending 37 signals' resources and allowing further conversations about pertinent topics. All this is perfectly fitting for a technological platform such as the world wide web, which is a fully democratic end-to-end communication platform where no one has to ask for permission to communicate with anyone else.

Of course, a proprietary vendor such as 37 signals would have none of that, hence their crying wolf in their today's blog post. Their wet dream is to create a locked-in hostage customers who will have no other choice but to only go to their web site for all their productivity tools needs. This is the exact same model that Microsoft and countless others old-school software vendors have been providing for decades. It is very sad that we're seeing the same greedy bullying on the web. Because of that, I say let's boycott the bullies, and let's turn instead toward the free software community offerings. Enough of supporting these fear mongering vendors!

User-contributed Resources

2009-02-20T15:34:00.000-08:00

As we have already discussed on more than one occasion, the world wide web is best described as an ever growing collection of resources. What usually doesn't get discussed is how many of these resources are offered by the original authors, vs. how many resources are user-contributed.

By 'original authors' I mean people who have initially built the web site(s) hosting the resources. For example, if I build a web site dedicated to voting on the best sites available on the web, I may initially 'seed' that site by offering some of my best picks. These best picks would be offered as URLs to the original sites that I am offering as candidates to being voted on.

After the initial launch of my 'best of the best' web site, others may join and may start contributing by adding their picks. All these additional URLs will then be user-contributed resources.

In general, as the so-called participatory web keeps gaining momentum, we will start seeing more and more resources being user-contributed. In the early days of the web, most, if not all published resources, have been contributed by the original authors, or creators of the web sites. Today, we see increasing quantity of resources published on the web that are user-contributed (or, sometimes called member-contributed).

Let's examine now how do users, or members, contribute resources. Typically, there are two ways that resources get published on the web:

1. By filling out a form and successfully submitting it

2. By uploading some digitized information

Whichever the case may be, it is interesting to examine the nature of the content that is being published. For the most part, one would expect that the content is pure text. Sometimes, the content is binary, meaning a sound clip or a jpeg file (representing a photo) or a video clip.

Because the web is basically text-driven, we will now turn our attention to the text-based content. At this point, the question becomes: what is this text-based content supposed to represent?

By and large I'm sure we'll all agree that the text-based content is related to the ongoing conversation on the web. But that's not the whole story. The web as the medium isn't suitable only for enticing and recording various conversations. The web can also enable acting upon the ongoing conversations.

In order to understand in what ways can the web enable this acting upon, let's first refresh our memory on what the resources on the web are:

Resources are abstractions, or entities, published on the web
Resources know how to represent themselves upon request
Resources know how to make a transition from one state to another, upon request

When a request reaches a resource on the web (routed via that resource's Uniform Resource Identifier, or URI), that event triggers a response. To provide a response is the responsibility of the resource. And, from the perspective of the consumer who had initially requested the response, it is anyone's guess what form will that response take.

It is customary to expect the resource to respond to the request by sending back a response containing the representation of that resource. Of course, a single resource can represent itself in more than one way. Often times, the mode of representation will depend on the context of the receiving request. If the request is, for example, coming from a human user, the response will most likely be in the form of a hypertext markup, which can be rendered in the user's browser. If, on the other hand, the request is coming from a machine, the response will quite likely be rendered as an XML document (although other machine-readable formats, such as YML or JSON are possible). These are merely the most generic scenarios, and there are of course different circumstances, where the resource may represent itself in a binary format, such as a PDF document, or a JPEG image, or an MP3 audio file, or even as a movie clip and so on.

Another interesting thing that deserves further examination is where is this response going to be coming from? It is almost always blindly assumed that if the request gets sent to the resource that is published in a specific domain (such as bestofthebestsites.com, for example), the response will arrive from that same domain, or URL.

However, there is nothing in the web architecture that may enforce that convention. Let me illustrate this point by providing a specific example: if someone sends a request asking for the number one best web site ever to the bestofthebestsites.com domain, that request may travel to hypothetical URL such as bestofthebestsites.com/top_site. Upon receiving this request, the top_site resource will respond by preparing its representation which will be shipped to the consumer. But in addition to that, another, more elaborate and more involved representation could be readied and sent over to some other resource, which may live in a completely different domain. From there, the original requestor may be notified by receiving yet another representation. There really are no limitations to chaining events in such fashion.

At this point, this discussion deserves a more elaborate illustration. Suppose we launch a web site that allows users to publish their own resources, such as hand-made merchandise that they put up for sale. The resource will typically be published by asking users to log in, fill out the form representing the resource, and then submitting that form. If the submitted form gets properly validated, the resource will be published on our site.

Let us now examine what could one possible representation of the resource 'merchandise' be. This resource may have a name, or a title, a description, a date when it was created, a date when it was offered for sale (i.e. published on our web site). In addition, it must have asking price (if it's for sale). It may also be offered for bidding between potential buyers, meaning that the merchandise will have an expiry date, after which it gets sold to the highest bidder.

And of course, that resource may also have a pictorial representation, such as one or more JPEG images, or video clips etc.

After filling out the form specifying many or all of the above properties of the resource, the form will be validated and, if correct, the item will get published on the site. At that point, all the information (i.e. content) will get stored on the hosting site, and the state of that resource will be persisted.

From that moment on, this resource will live in its own address (e.g. http://merchandisehost.com/items/123546). Any request arriving to this address will elicit a response from that resource. And, as we've already discussed above, the shape and form of the response will depend on the circumstances -- meaning, is the request coming from a human user or from a machine, and furthermore, are there any other constraints that could govern the specifics of that response?

So in this simplified hypothetical example, the entire interaction between the resource and its consumer is based on the assumption that the resource only knows how to represent itself by shipping the response from the http://merchandisehost.com domain.

The above constraint, however, is largely simplistic and overtly naive. It is presupposing that the resource on the web is only defined by its static content. But a web resource is not limited to only having static properties, such as textual or visual representation. In addition to having state, web resources also exhibit behavior.

The above statement may seem superfluous in the light of our previous understanding that web resources already know how to behave, by knowing how to represent themselves upon request and by knowing how to make a transition from one state to another, also upon request. But the kind of behavior specific to the web resources that hasn't been discussed yet is the user-contributed behavior. That aspect of the web resources could be the most interesting, and yet the least explored opportunity that the web has to offer.

Time for another illustration: let's say I create a series of prints and offer them for sale on http://merchandizehost.com. I supply the title of the print series, the number of available prints for sale, the item price and the photo illustrating the print. Nothing unusual so far; but suppose now the merchandize host allows me to specify what I would like to happen each time someone buys one of my prints. Even that is not unusual, as many websites today allow people to click on a checkbox saying "Send me an email or SMS when someone buys this". But that's not what I'm talking about here; I'm asking you to imagine a host which will allow us to specify the URL which will receive HTTP POST request on event, such as sales purchase.

Now what could that URL be? Why, it could be anything. It is user-contributed. And once the HTTP POST arrives at that URL, it is anyone's guess what happens next. The user who had contributed that URL is now in the driver's seat. Upon receiving the HTTP POST request, that user's code could post a Twitter update containing the tiny URL representing the item just sold. Furthermore, that same code could further elaborate by sending a request to the author's supplier requesting replenishment for the sold item. And so on, the sky is the limit.

The event when someone had purchased an item on the http://merchandizehost.com site can thus easily mushroom into all kinds of user-defined events that could continue percolating throughout the web. The user who had created the resource may then get notified about the purchase from his/her Twitter feed, or from any other source of alerts they care to set up. Also, the customer who had purchased that item may get confirmation not only from the original web site, but also from some other sources, driven by the author-contributed behavior enhancements.

We see from the above hypothetical scenario that user-contributed resources enhance and embellish not only the content on the web, but also the behavior on the web. Resources on the web put end-users in the driver's seat. Thanks to the radically distributed, fully democratized end-to-end nature of the world wide web, humans have finally achieved the ultimate freedom to fully control what happens to the resources, as they create them.

The Age of Unlimited Copying

2009-02-02T16:20:00.000-08:00

I have a mix of good news/bad news to deliver today. The realization came about when I was pondering the choice of notions and words when evaluating our strategies geared toward our future survival and sustainability. One of the issues is that we seem to collectively believe that we're at the cusp of entering the Age of Ubiquitous Computing (or Ubiquitous Computation). You know, what with the World Wide Web and all that jazz.

But wait a minute, what if the notion/word 'computing' (or, 'computation') is a misnomer? After all, what is it that we're all so ubiquitously computing?

As the clear answer to the above question is not forthcoming to me, that is always a surefire sign that we're dealing with a misnomer here. While there is definitely no denying that we are pushing vigorously toward the 'ubiquitous' something, what that 'something' is isn't quite clear yet.

Allow me now to examine that elusive activity. And please bear with me, because this examination is going to prove very useful during the forthcoming analysis.

There isn't really that much computation, or math, that is transpiring in a day-to-day usage of the web. On the web (which is this machinery that is seemingly busting at the seams in its attempts to reach ubiquity), most of the activities revolve around searching/browsing/consuming/publishing/participating etc. Very little math overall.

That being the case, isn't it then kind of wrong to call all the above activities 'computations'?

Yes, but if those are not 'computations', what are they? I'd suggest switching to another notion, a different concept: these are largely acts of copying.

So the world-wide machinery that we have collectively built (i.e. the World Wide Web), which we're now nursing and growing in the hopes of seeing the day when, to use Bruce Sterling's precious analogy, it will become a fully qualified varnish on barbarism, is basically best suited for one, and pretty much only one activity -- copying.

As Cory Doctorow had put it so eloquently, there is never going to come a day when copying will get any harder than it actually is today (forgive the absence of the source, I promise that I will work on digging up the link documenting this gem uttered by Doctorow). Starting today and moving forward, copying on the web is only going to get easier and easer. So we see plainly that, as long as we stay on the web, in the cloud, etc., the activity of copying has a real bright future.

So why is copying so important, and why would it be so desirable? Isn't the act of copying in itself kind of worthless? If you have an original, and then you make a copy, isn't that like degrading the original? Since when is being derivative so desirable?

True, in the pre-internet days (that is to say, in the days when digital technology was in its early infancy), the act of copying couldn't amount to much. And the situation in the pre-digital days was even worse, much, much worse. To copy anything worth copying in the olden days would mean wasting a lot of time and energy producing something that was destined to end up being merely a pale copy of the original. Because of that, only few very specialized craftspeople have ever been engaged in the copying activities.

However, the unstoppable march of human inventiveness and ingenuity had shown us that we humans are pretty much hitching our star, our overall destiny, to the technology bandwagon. And that being the case, we have gladly accepted the fact that a digital artifact, a digital replica, is often times just as good as its analog counterpart (and in some cases it is perceived as being even better than the analog original).

So now we already find ourselves in the position where we often times prefer getting our greedy little hands on a digital copy of a book/article/song/photo/movie etc. then perusing the original, analog object. Digital version seems infinitely more pliable for our consumption, because it is demonstrably easer to thumb forward/backward through a digital copy without the fear of wearing it out, search through it at will (and at random), get it to follow us everywhere we go, store it safely and forward it to our friends, and so on. Not only that, but once digitized, the copy of our analog object becomes infinitely more malleable to any ad hoc efforts toward remixing and cutting and pasting through it to our heart's content.

In other words, a digitized copy opens up completely new and unforeseen doors of creativity and imaginative actions. We humans seem to, for some reason, like that a lot.

But one thing that barely crosses our minds as we're perusing and (ab)using these digitized representations of our analog object is that, each time we do something to it, it gets copied. And the cost of copying it is so negligible, that no one ever notices it. Now this fact is very significant, as it points to the very future of human society. Please make a mental note of this fact now, as it will come very handy a bit later in this essay.

Let me now talk for a moment about the maturing of our overall technological strategy. In the olden times, when human technology was in its germinating stages, we lived in the world of customers. This was the industrial society, where customers were expected to buy/consume products (physical objects) that were being produced by the machines. These crude days of the initial introduction of technology into the human society are now largely behind us.

According to Bruce Sterling, we have now made the transition from the customer/consumer-based economy to the end-user-based economy. Bruce claims that we now live in the world flooded with gizmos (one such gizmo being a cell phone, a veritable staple of the early 21st century economy). Just by casually glancing around, it's getting increasingly hard to argue with that claim.

We can actually track all these advancements of technology by matching it up with the breakthrough advancements in the general purpose machines. The first major breakthrough that pretty much announced that a change everything movement was afoot, was Alan Turing's universal ordinating machine (a.k.a. the predecessor of today's ubiquitous computer). What this invention offered was a platform, a ground to stand on. From there, another groundbreaking invention wasn't far off -- the Internet (or, the ubiquitous planet-wide network, built on the end-to-end organizing principle).

The above two groundbreaking inventions have indeed become so ubiquitous that they have penetrated lives of all people who today coexist under the auspices of Western Civilization. It basically took a meagre 50 - 60 years for this portentous event to take hold and penetrate almost every pore of our social lives.

One crucial thing to remember when we're talking about those sorting/ordinating machines (i.e. computers) and their network effects once these machines get plugged into the world-wide cloud (i.e. the Internet, or the World Wide Web), is that both the machines and the resulting network are general purpose by design. This is significant, because the general purpose nature of these technological inventions is what turns them into tools capable of bringing different kinds of freedoms to humans.

For example, the most liberating aspect of this technological platform is that it is a general purpose copying machinery. Being a general purpose copying machinery, it has the capacity to free us from numerous constraints by offering a generous sharing platform for many of the important artifacts that our culture is comprised of. As we've already mentioned, it is never going to be harder to copy something on the web than it is today. This fact gives us a reason to rejoice, as it offers each and every one of us a free chance to participate in sharing many facets of our cultural heritage.

As described thus far, this generous technological platform, this stupendous copying machine, is founded upon two legs: one leg is the general purpose filtering/sorting machine (a.k.a. the computer, the ordinator), the other leg is the general purpose end-to-end communications mechanism (a.k.a. the Internet). However, for this platform to be fully functional, a third leg is required. This third leg is needed in order to give the platform true stability, so that it could reach out into the material, three dimensional world.

We've seen that the giant ubiquitous copying machine can even today afford us the ability to make limitless copies, at practically no cost, of any cultural artifact that is of interest to us. It goes without saying that any residual cost of making a copy, no matter how tiny, will continue to shrink rapidly, until it approaches the ideal no-cost asymptote.

This is significant for various reasons. The most important reason today is that this arrangement is allowing us to rescue, at a marginal cost, copious amounts of otherwise doomed cultural artifacts. By this we mean countless out-of-print books and magazines, as well as countless other artifacts, such as sound and visual recordings. In addition to that, this ability to endlessly copy at little or no cost has opened up the doors for many talented people to freely express themselves without having to wait to be 'discovered' by the officially established publishing mechanisms.

By far, the most interesting and exciting aspect of this newly arrived situation is our ability to produce, at will (or on a whim), something that we call object on demand. If someone discovers in their grandparents' attic an interesting, out-of-print archival photograph, and then they make a copy of it by digitizing it using their digital camera, they can then further copy that cultural artifact by downloading it into their computer or mobile phone, publish it on the web with a simple push of a button, thus allowing anyone in the world to continue copying it till the proverbial cows come home. What's even more exciting than that is how anyone who obtains that copy can treat it as an object on demand, meaning they can, on a whim, demand that this copy be rendered as an object (e.g. a photo) that could be framed and hung up on their kitchen wall.

This production cycle, sometimes described as cradle to cradle, is definitely a novel way to look at how we handle our everyday affairs. The most interesting and revealing change that has made its way into the mainstream society is the turning of the tides in the way we regard the hardware/software equation. In the olden days, hardware was traditionally meant to be used for hosting software. Today, the tables have turned on us, and we now view software as the platform for hosting hardware. Things got turned on their head, all thanks to the simple fact that we have built a stupendous, ubiquitous general purpose copying machinery. What we have at our disposal today is the ability to demand hardware products at will, and on the cheap (this demand comes with minimal, or sometimes no cost at all). Thus I can copy as many Linux servers as I wish and demand that my hosting provider renders them for my perusal. All that power at a ridiculously low price (which is going to quickly erode in no time). Compare that with millions of dollars that businesses of yesteryear were expected to shell out for a single server (and a pretty lame one at that). The million dollars price tag was the norm not that long ago.

Times have changed all right, but that's nothing compared to the change that is afoot. Right now we have the ability to render, on demand, an inexpensive replica of a mind-blowing plethora of very desirable cultural artifacts, such as an out-of-print book, and out-of-circulation vinyl record, a rare photo, a forgotten TV show, a salvaged old newsprint, a powerful web hosting server, and so on. But the interesting thing is that we don't stop there; once we're satisfied with consuming the rendered object, we can safely pass it on, share it with our friends, neighbors, the community, the world. All that with a simple click of a button, at a next-to-nothing cost. Fantastic, right? And quite unthinkable only 15 - 20 years ago.

But what if we wanted to demand an object such as a plain glass of water? Where do we find a copy of that object, so that we could demand it?

In order to answer that question, we need to look a bit more deeply into how is it that we are able to demand, from the comfort of our homes, objects such as books, photos, movies, music, web servers and such. The answer is actually quite simple -- we can afford to demand such objects because we have obtained the blueprints of these objects. If I spend some time building and configuring a web server, such as a Linux box, I have in effect created a blueprint of that object. Once created, that blueprint can be freely copied and distributed at no extra cost. Anyone in the world who has the need to demand a web server can conjure it up by utilizing the blueprint I have provided.

Here's another example: if I record my guitar performance and publish it on the web, others can freely demand that object (i.e. the recording of my performance) due to the fact that I have provided the blueprint for that object. In this case, the blueprint used for rendering my performance comes in the form of a special arrangement of bits that get copied all around the world. This unique arrangement of bits is then used by the machinery to render the signal that will be interpreted by the listeners' brains as a 'guitar performance.'

So we see that once we obtain the blueprint, we can demand that the object that is represented by that blueprint be rendered. Following that logic, before I can demand an object such as a plain glass of water, I need to first obtain the copy of that object's blueprint. However, such a blueprint doesn't seem to be available anywhere. Hence the conundrum.

"But wait", I can already hear your cries, "even if such a blueprint was available, of what use would it be? Surely you cannot hope to be able to produce a glass of water by using an LCD screen or a pair of speakers or even some fancy holographic whachamacallit projector!"

True, my hopes of obtaining a blueprint for the plain glass of water are nothing more than castles in the sky. It's wishful thinking, mere daydreaming.

It is precisely because of this vanity that we don't see any blueprints for things such as plain glass of water being published. The exact same thing held true in the days before the invention of a phonograph -- no one was producing vinyl records (i.e. blueprints for reproducing musical performances) before they knew that there exists a machinery that can read the blueprints and render the object on demand.

So instead of putting the cart before the horse, the community is waiting for the machinery that will be capable of rendering various blueprints of various objects. But unlike machinery such as phonographs and TV screens and headphones etc., which are highly specialized machines, this hypothetical machinery must not be specialized. It must be general purpose.

This necessity simply stems from the already established two legs of the technological platform. Recall that both the ordinating machinery (the computer) and the end-to-end communication network (the internet) are general purpose machines. For the third leg (the one that will give the platform its final stability) to function properly, it must also be a general purpose machine. Once we have three general purpose legs supporting the world wide technological platform, we will get pretty darn close to understanding what does it mean to change absolutely everything.

If the general purpose ordinating machine is called 'the computer', and if the general purpose end-to-end communication machine is called 'the Internet', what is the general purpose object-on-demand machine called? Not surprisingly, we already have a name for that machine -- it is called general purpose nano assembler (also called molecular assembler). A general purpose nano assembler is a machine capable of ordering and ordinating the building bricks of material reality (i.e. molecules and atoms) according to the instructions contained within the supplied blueprint. Allow me to provide a hypothetical example:

Suppose I wish to bake a loaf of bread, but then realize that I am out of wheat flour. Suppose now that I have a general purpose nano assembler mentioned in the previous paragraph, and that furthermore I can ask for a copy of the blueprint containing instructions on how to render wheat flour. I should then be able to feed that blueprint to the nano assembler, which will then, following Michael Braungart's and William McDonough's waste=food model, allow me to exercise my object-on-demand privileges. The outcome of this exercise will be a bag of flour made out of waste. Needless to say, I could then use that waste-turned-food bag of flour to bake a loaf of bread.

In and of itself, the above hypothetical event illustrating the act of turning waste into food on demand is mind blowing. However, much larger mind blowing things lay ahead. Let's examine it this way: in the event of reaching the point of actually producing the general purpose nano assembler (which today still feels like pure speculative science fiction), the question remains -- who are those among us who will be privileged to own these revolutionary machines? But the funny thing is that anyone who asks that question is actually missing the point, because what is being overlooked here is the fact that the first, initial general purpose nano assembler, the one with a bar code that specifies its serial number as reading something like 0000000001, has been assembled from a blueprint. What that means is that a general purpose nano assembler can bootstrap itself and assemble more versions of itself. All it basically takes to produce another general purpose nano assembler is a simple push of a Copy button.

So if I have access to a nano assembler, and my less fortunate friend over there doesn't have access to it, that's a non-issue, because I can exercise my object-on-demand rights and produce a copy of a nano assembler for him.

So we see that a general purpose nano assembler is akin to the genie who resides inside Aladdin's magic lamp. A wish fulfilling genie who obeys his master's orders and can conjure up different kinds of objects -- render them on demand. But the nano assembler we're talking about here is capable of going one up on the genie -- it can replicate itself, thus endlessly spreading the joy and the wealth. (in the original fairy tale, genie's masters were not able to give genie an order to create more genies)

Conceived in this fashion, the giant general purpose copying machine has the capacity to encompass the entire planet, fulfilling many of our wishes by rendering various objects on demand by recycling waste (even in this aspect general purpose copying machines are superior to the genie since they conjure up objects by reusing waste, unlike the genie who conjures objects out of thin air, thus not doing anything useful toward reducing waste; in fact, the traditional genie only litters the place, which means he is downright harmful).

So far, all of this has been nothing but good news, excellent news in fact. So where's the catch?

As I've mentioned at the beginning of this article, I also have some bad news to communicate. Consider the following: in the age of unlimited copying, where pretty much any object can get copied as many times as we wish, while at the same time eliminating waste, money all of a sudden begins to pale and lose its luster. Pretty soon, money becomes meaningless. When that starts happening, we'll all find ourselves staring at a universal crisis of ownership. In an age of unlimited copying, where absolutely everyone is entitled to produce as many copies of any object as they wish (while at the same time reducing and recycling waste), it gets impossible to continue assigning the concept of ownership to any particular person or legal entity. The days of exclusivity-based economy then come to a screeching halt.

Consider now how everything, absolutely bloody everything that we know about human life revolves around this concept of ownership and exclusivity. We have been bred, for countless generations, to deal with scarcity in our lives by employing and applying the concepts of ownership and exclusivity, and, by extension, the concept of control. In order to control something, you need to first own it by excluding others from getting anywhere near it. If we hope to get to the point where the very idea of ownership disappears, where, as John Lennon sang "Imagine no possessions, I wonder if you can?", how are we to deal with it, if we can't even imagine it?

If I go on the web today and I download a wikipedia article that I wish to read, can I honestly feel that I 'own' that article simply by the fact that I've downloaded its copy onto my screen? Knowing that, at the same moment, potentially millions and millions of people worldwide have also downloaded that same copy to their screens, I cannot possibly feel the pride of ownership. This artifact, this wikipedia article is an object rendered on demand, and does not entitle me to assume any exclusive position in society.

The same sentiment will prevail once we get to the point where we're able to demand any object from the giant copying machine. The pride of ownership and exclusivity will simply not be present in that transaction.

This will then pose huge problems for most humans. Human civilization and human culture in its entirety is a veritable treasure throve filled to the brim with nuggets of wisdom on how to deal with scarcity. Scarcity comes natural to us, almost like breathing. We have never, ever in the recorded history been forced to deal with universal abundance. Making this inevitable transition from the economy that's based on scarcity to the post-scarcity and post-exclusivity economy (the one that's based on limitless and harmless universal abundance), will have a completely unpredictable freak-out effect on most of us. Outbursts of mindless violence will be almost unavoidable.

The most interesting thing, however, is that we have already been through this transition from scarcity to all-inclusive abundance once, and we've seen the consequences, and they were mostly ugly. I am talking about the infamous days of Napster, the first massively popular online resource sharing machinery. Back in the prehistoric days of the ancient web (8 - 9 years ago), Napster gave us the first glimpse and the first foretaste of things to come. Many of us, including myself, got our first cognitive dissonance shock upon experiencing Napster's object-on-demand genie in action. One thing that most of us had trouble initially wrapping our heads around was the concept of turning scarcity into abundance; turning exclusivity into all-inclusive abundance. Here is how the cognitive dissonance typically went (at least for me, but I've also checked this with some of my friends, and they all share similar war stories):

We all intuitively know, deep down in our bones, that you can't have your cake and eat it too. If, for example, I obtain a bag of apples, the more I consume them, the less apples remain in the bag. Therefore, apples are a scarce resource, the one that has to be rationed and consumed carefully. That's common sense knowledge, the one every child learns in kindergarten, if not at a much earlier age.

Applying that common wisdom to the machinery such as Napster, I knew that if there is a rare out-of-print song that is much coveted by the community, and was then suddenly made available to the community, that event should create a so-called race condition. In other words, whoever gets lucky enough to be there first and grab the much coveted object, consumes it, squirrels it away, thus winning the race, and as a consequence ends up being the envy of the group. That person can now claim victory and the rest of us can only curse our bad luck.

But shockingly, this 'the more you grab and consume, the less of it remains' mentality got drastically turned on its head when using Napster. On Napster, a strange thing became apparent -- the more we rush to grab the object and consume it, the more that same rare and coveted object becomes readily available. More consumption actually paves the way toward much smoother, easier consumption. Sheep shit grass. Amazing!

But how's that possible? Again, we were mystified because at that time (back in 1999 - 2000) we weren't aware of the fact that we are, for the first time ever, dealing with a massive machinery that offers unlimited, general purpose copying free of charge! By consuming all those super rare and out of print cultural artifacts, we were getting engaged, for the first time in history, in the object-on-demand revolutionary activities. Not only that, but with each individual, and let's face it, selfish action of consuming an artifact, we were making the world for the ones who follow in our footsteps much smoother, much friendlier. Thanks to our efforts, the next wave of object demanders could get those once scarce and coveted objects much faster. Hence the cycles were getting shorter and shorter.

Until, of course, those who fancy themselves as exclusively 'owning' the blueprints that were being shared on Napster, issued a court injunction on March 5, 2001, ordering Napster to cease and desists and to shut down its machinery. Thus, the first innocuous attempt at turning scarcity into all-inclusive abundance and making the transition from scarcity based economy to the post-scarcity economy was a spectacular failure.

There are no indicators that would point to a different scenario once we start entering the next phase ushering into the post-scarcity economy. The crisis of ownership that this movement will entail will definitely stir up many negative, even violent sentiments, and those who view the world as consisting of the 'haves' and the 'have nots' will insist, with all their might, that they remain in the 'haves' camp, and that the 'have nots' remain in the gutter, as they always have been, labeled as the 'unwashed masses'.

How will that conflict play out is anyone's guess, but I am personally not convinced that the resolution could be reached in a peaceful manner.

In conclusion, we have seen that historically, all our efforts invested so far in the technological progress seem to be now converging toward the universal, all-inclusive general purpose technological platform. In other words, the odds seem to be in our favor when it comes to reaching the age of unlimited copying. What that means is that, thanks to the fundamentally generous nature of this general purpose technology, every human being is entitled, by their birthright, to engage in this copying activity to their heart's content. And because the objects rendered on demand will use waste as the source material, no danger of over-consumption is imminent. If anything, the more we consume under this model, the more we get rid of the waste.

This arrangement is unprecedented in human history. It by far surpasses even the most imaginative fairy tales (such as the already discussed Aladdin's magic lamp). Only the Buddhist lore, which has been, from the very outset, portraying the worlds of bewildering abundance, talks about something similar to what we've discussed here (the Buddhist portrayal of the abundance that surpasses even the wildest imagination has its utmost culmination in the world famous Diamond Sutra).

But just as much as this prophecy excites us, the flip side of it should gravely concern us. As we've already mentioned, arriving at a non-rivalrous position, whereby any desirable goods could be easily rendered on demand by each and every human being, means almost certain death of economy as we know it (simply due to the fact that economy is based on the exclusive ownership of the means of production which, once everyone owns all the imaginable means of production, becomes a non-issue). Once that happens, uncontrollable outbursts of violence may be imminent. Managing and channeling this irrational side of our collective minds will be our biggest challenge in the future -- the future that is rapidly and unstoppably rushing toward us.

Software Development Detox Part 5: Session

2008-06-20T10:58:00.000-07:00

In my previous installment (oh-my-god, it's already been a year!), I've discussed the hard-wired need to retain central point of control when building software. Today, I'm going to examine the central software concept that implements this urge for control: session.

Session is a vessel used for retaining the memory of all the events that had occurred during the software interaction. It often gets implemented with the intention of providing continuity during the interaction.

Because of its central position in the overall architectural solution of a networked software, session as a concept is very brittle. It can quickly gain entropy and can end up costing a lot in terms of computing infrastructure.

On the web, however, since the prevailing architecture is stateless, the concept of session is entirely unnecessary. The memory of an interaction that may occur on the web need not be retained by any participant.

Realization of this simple dictum is proving to be extremely difficult for many software developers. They feel that, without having the crutches and training wheels that the concept of session provides, they will have no way of knowing how to make a decision on what to do next. At least when they use the state of the conversation, as recorded in the session, they can write some logic that would help them execute some meaningful actions. But without that knowledge available to them, they feel lost.

This sentiment bellies the lack of understanding of the web architecture. On the web, each request contains all the information necessary for the server code to make a decision on what to do next. There is absolutely no need for the server to keep track of any previous requests. Not only is such a task exorbitantly expensive, it also creates a single point of failure, which is the primary cause of faulty software being deployed live worldwide.

So the advice to all budding software developers is: abandon the idea of a session, and rely on your own wits when building the business logic that drives your site. Going through such software development detox procedure will ensure that you build and deliver robust, lightweight web sites that will be easy to host and maintain.

Why is Software Development on the Web so messed up?

2008-01-20T10:55:00.000-08:00

It's been said that those who do not learn history are doomed to repeat it. In the spirit of absolutely not wanting to go again through the excruciating pain of software development that was the norm in the '90s and the first 5 - 6 years of the 2000s, I am going to dissect the following historical document: XML, Java, and the Future of the Web, published on October 2, 1997, by Jon Bosak.

As I walk through some of the salient points exposed in that portentous documents, I will try to bring to your attention certain glaring errors in understanding and judgment regarding what is the goal of computing and of the information processing on the web, in particular.

Let us then begin with the opening sentence:

"The extraordinary growth of the World Wide Web has been fueled by the ability it gives authors to easily and cheaply distribute electronic documents to an international audience."

Right out of the gate, Mr. Bosak takes the left turn and commits himself to the implementational aspect of the technology he is discussing. The key concept, of course, is 'electronic documents'. In his view, the web is useful in its ability to offer an affordable distribution channels for prefabricated electronic documents.

The problem with the Ivory Tower views of the web is that, being high-tech bent, the high priesthood inhabiting the Ivory Tower tends to view the smallest unit of processable information as an object, such as electronic document.

Following at the heels of the opening salvo, the second sentence in this paper claims:

"As Web documents have become larger and more complex, however, Web content providers have begun to experience the limitations of a medium that does not provide the extensibility, structure, and data checking needed for large-scale commercial publishing."

This is now quite confusing. If the smallest unit of information processing on the web is an electronic document, what need for extensibility, structure and data checking is he talking about?

Let's look at the third sentence; maybe we'll find some clarification in there?

"The ability of Java applets to embed powerful data manipulation capabilities in Web clients makes even clearer the limitations of current methods for the transmittal of document data."

Huh? Java applets? OK, this is completely obsolete, but still belies the unexplained bias toward powerful data manipulation capabilities in Web clients.

In the fourth sentence (and opening a new paragraph), the author introduces the concept of Extensible Markup Language, and explains:

"... an Extensible Markup Language (XML) for applications that require functionality beyond the current Hypertext Markup Language (HTML)."

What is not clear yet is what kind of "functionality beyond the current Hypertext Markup Language (HTML)" is he referring to?

HTML -- imaginary limitations

After delivering the opening salvo, as exposed above, the author goes on to elaborate on the severe constraints that HTML introduces and how debilitating these constraints appear to him. Here is the author's list:

Lack of Extensibility
Absence of Structure
Inability to perform Validations

Extensibility

This feature is the holly grail to developers who are keen on creating their own, proprietary protocols. The much celebrated power that Extensible Markup Language (XML) brings to the table -- namely extensibility, is at the same time the most troublesome aspect of the failed attempts to engage in collaborative computing. If anyone and everyone is free to use the technology that enables them to daydream any protocol they wish (in effect creating their own unique language), the Babilonian confusion created that way must quickly reach cosmic proportions. This is exactly what's happening right now on the web -- everyone feels entitled to dream up their own bloody protocol, and the potential online conversation tends to fall mostly on deaf ears.

The author states this clearly: "HTML does not allow users to specify their own tags or attributes in order to parameterize or otherwise semantically qualify their data." The key phrase is "their own" -- which amounts to "invent your own language".

Structure

Similar to the old adage 'form follows function', structure typically follows the meaning of the information. Mr. Bosak's lament in this respect is largely unfounded, since HTML does offer fairly comprehensive structure for expressing the semantics of the represented information.

Whether the author was aware of that ability to express semantics using HTML, remains a bit of a mystery. In any event, what the author seems to be complaining about with regards to the lack of structure in HTML is the erroneously perceived need for representing complex hierarchical dependencies.

There is really no evidence that an average consumer perusing the web resources has such a pressing need for consuming large and increasingly complex hierarchical constructs. In author's own words: "HTML does not support the specification of deep structures needed to represent database schemas or object-oriented hierarchies."

Yes, the above is true, but the question is: why would anyone have the need to consume 'deep structures, object-oriented hierarchies, or database schemas'? The question, of course, remains unanswered.

Validations

Let's examine Mr. Bosak's objection regarding HTML's inability to perform validations: "HTML does not support the kind of language specification that allows consuming applications to check data for structural validity on importation."

I must admit that I'm a bit confused -- why would anyone use a semantic markup language, such as HTML, to perform validations on 'importation'? Importation to where?

Responsibilities of the Web Client

The author then enlists some imaginary responsibilities that are, in his view, mandatory for the web client:

Web client may have to mediate between two or more heterogeneous databases
There is a need to distribute a significant proportion of the processing load from the Web server to the Web client
Web client often has to present different views of the same data to different users
Intelligent Web agents will attempt to tailor information discovery to the needs of individual users

The above list belies a strong desktop-centric approach to software development. All of the 'needs' listed in the four items above fall naturally into the server-side jurisdiction. The client is only responsible for rendering the representation that the server side business logic prepares.

Linking

Toward the end of his paper, Mr. Bosak addresses some perceived inefficiencies that he sees in the hypertext system as implemented using HTML. His list is, again, very revealing. Here is his list of perceived inefficiencies when implementing linking via HTML:

HTML lacks location-independent naming
It lacks bidirectional links
It lacks the ability to define links that can be specified and managed outside of documents to which they apply
No N-ary hyperlinks (e.g., rings, multiple windows)
No aggregate links (multiple sources)
No transclusion (the link target document appears to be part of the link source document)
No attributes on links (link types)

Below is my response:

Location-independent naming: Huh? If I link to a resource representation, such as google.com, how's that location dependent? Where in that hypertext link do we see any information that would reveal location of the resource representation named google.com?

Bidirectional links: This is the engineer's wet dream; having links that will never get broken. Not gonna happen, time to grow up and deal with reality.

Links that can be specified and managed outside of documents to which they apply: It's the damned documents again! Snap out of the document dogma and start living in the world of resources and their representations. In other words -- content!

N-ary hyperlinks (e.g., rings, multiple windows): Not sure what this means, but I really cannot see any real life use for it.

Aggregate links (multiple sources): OK, 10 years ago no one apparently knew about real simple syndication and such, plus the aggregators etc.

Transclusion (the link target document appears to be part of the link source document): Also, no clue about the mashups that will become all the rage 10 years later...

Attributes on links (link types): What on earth does that mean? Link types? Like what?

Giving Java something to do!?!

Finally, the author cites the slogan "XML gives Java something to do" (no sources quoted in the original paper).

What does he mean by this battle cry? What would he envision Java will do with XML?

It's hard to say after more than 10 years had passed since that battle cry was uttered, but my hunch would be that he was referring to the 'nomadic' nature of Java. In the olden days, Sun Microsystems was selling the "network is the computer" slogan, and so at that time Sun's trailblazing language was Java. According to Sun, Java lives on the network, and is nomadic, courtesy of RMI (Remote Method Invocation). Snippets of Java code can travel the network, on demand, in order to supply just-in-time computations on the requesting client.

Supposing that the client had just received an XML document but doesn't know how to present it to the user, a snippet of Java code could be invoked and downloaded on a whim. Once installed on the client, Java routine will massage the XML etc.

This dated model of 'distributed computing' has long been pronounced dead and buried. The problem is, we still have armies of software developers who tend to think in a similar vein. Because of that, software development on the web has turned into a terrible chore, and the resulting web sites tend to be nightmarish.

The moral of this story is to learn the lesson from the history, become aware of the blind alleys that have been paved by the developers a decade ago, and learn how to avoid the minefields, pitfalls and traps of the obsolete software development practices.

You're welcome!

Resource Representation and Discoverability

2007-12-21T10:53:00.000-08:00

We now seem to be entering the phase where we are in the process of reclaiming the web. Who are we reclaiming it from? Why, from no one else but the hordes of software developers.

The web has been hijacked by software developers. It is only natural that the people who dedicate their lives to becoming as intimate with the machines as is humanly possible get to be the people who take center stage in building the web. But, there is trouble in paradise. The web is, in its essence, the exact opposite of the machines. The web is all about humans.

Now, software developers are humans, however they are humans who value serving machines more than they value serving other human beings. Because of that, the web is a horrific place right now, fully dedicated to serving the machines.

This terrifying state of affairs must change. And the only way it will change is if humans reclaim the web, pry it out of the slaves-to-the-machine greedy hands, and install it at its proper place -- as a platform for serving the needs of the human mind and the needs of the human social dimension.

Resources and representation

As has been argued elsewhere, resources on the web are mere abstractions. They are not the tangibles that one could indulge in. At best, one can hope to consume the representation of the abstract resources scattered around the web.

It is often erroneously assumed that resources are directly exposed on the web, in their raw form. One example would be the unanimous conviction that URLs are web's native resources. Thus, a URL such as craigslist.org is viewed as a web resource.

Nothing could be further from the truth. A URL (such as craigslist.org) is not a web resource, it is merely a representation of a web resource. There seems to be an abstraction labeled as craigslist.org and hosted somewhere on the web. But what that abstraction (i.e. craigslist.org) really is, we have no way of knowing.

What we can learn about is one or more representations of that resource. For example, one representation of the craigslist.org resource is its URL. Another one could be the list of all the sites offered by craigslist.org. And so on.

What's the use of representation?

Some people fail to see the usefulness of the representation. They'd much rather get their hands on the resource itself, instead of beating around the bush of resource representation. The best way to explain this problem is to employ assistance of some heavy duty science:

Map is not the territory

This famous premise was issued by Alfred Korzybski, one of the seminal thinkers who helped shape the communication and information theory.

A map is the representation of the territory. Without the map, we'd be lost when traveling through the territory.

In a similar fashion, resource representation could be viewed as a map that eases our voyage through the territory (i.e. the resource we're exploring).

Very few people tend to question the usefulness of maps. It is our hope that, similarly, people will learn to embrace the usefulness of the resource representation.

Discoverability

The web is intended for human consumption. Humans are notorious for having fairly constrained short-term memory buffer, and are thus forced to consume information in a piecemeal fashion. The architecture of the web is therefore tailor-built to serve exactly that constraint -- consume the resource representation at a fairly leisurely pace. Expecting humans to consume the resource representation in a single giant gulp would be utterly unrealistic, and so the fundamental architecture of the web is based on the principle of discoverability.

What that means is that, on the web, we are serving byte-sized chunks of resource representations. These byte-sized chunks are intended to be consumed by human users pretty much at a glance. Any need for consuming more, for learning more, gets fulfilled by letting the users explore further, allowing them to discover more intricate details of the resource representation.

Away from the sociopath web

Today's web is mostly built by humans who crave serving the machines. As such, the way resource representations are typically architected on the web today belies the optimization to the way machines consume information. There is typically very little discoverability offered on most web sites, and the consumers of the representation are usually expected to digest insanely vast quantities of intricate information in a single gulp.

It is painfully obvious to even a very casual observer that the web today is not tailored for easy consumption by humans. It is as if some sociopaths, who lack any degree of empathy with fellow human beings, have built most of the web sites in operation today. Come to think of it, sociopaths is the fair characteristic of individuals who value interactions with the machines more than they value interacting with other human beings.

It is high time we start working on getting out of this sociopath hell.

Why is ROA Hard to Understand?

2007-08-16T10:52:00.000-07:00

Resource Oriented Architecture is the term I've coined a year ago, only to discover almost immediately afterwards that others have been thinking independently along the same lines. During the past 12 months, I've witnessed numerous discussion threads and debates centered around the importance of ROA. Most of these discussion threads belie complete misunderstanding of the Resource Oriented Architecture, which is very alarming, given that ROA is the native architecture of the web.

And web is the native infrastructure for information dissemination, exchange, and sharing.

Old habits die hard

Most software developers in circulation today have cut their teeth by learning how to implement computing algorithms. Something akin to: "here is a jumbled sequence of numbers, write a computer program that will sort them in ascending order".

The emphasis in such an exercise is always on exhibiting reasonable amount of direct control over the execution logic. The would-be programmers have been groomed for generations to master the skills of gaining full control over the stream of instructions that the CPU executes. In other words, to be a computer programmer/software developer implies to them that a full control over all the processing aspects must be in place.

It's a single point of human control, same as it's a single point of machine control (i.e. the Central Processing Unit, or CPU).

But in the world of the web, such notions of centralized control are not only ridiculous, they are down right harmful. Still, armies of CPU-trained programmers have now been employed and deployed to develop software that will run on the web.

Is it a small wonder then that such 'central point of control' minded individuals are creating fully dysfunctional web sites? Indeed, old habits do die hard.

New habits are slow to emerge

What the above described situation then means is that the existing workforce that is dedicated to developing software programs is pretty much a write-off. Because old habits die hard, it is quite likely that it will be much harder to unlearn the old habits and relearn the new ones, than to start from a clean slate.

However, given the vast demand for web-based software that is out there, we must work on creating some sort of a compromise, in the attempt to bring the crusty old 'central point of control' software development workforce into the web fold. For that reason, we need to continue explaining the difference between the Central Processing Unit software architecture, and the Resource Oriented Architecture.

Transitioning from doing to managing

One of the hardest things to do in life is to make a successful transition from doing something to managing that same thing. The temptation to roll up one's sleeves while managing the process is simply too strong to resist. And that's why not too many people end up being good management material.

In the world of software development, right now there is a huge demand for people who are capable of abstaining from rolling up their sleeves and instead delegating the responsibilities to the 'workforce' out there.

What does that mean? In a more technical parlance, it's been proven that once software developers get trained in making procedure calls when attempting to process some information, they tend to get addicted to that gateway drug. Similar to how to a person with a hammer everything looks like a nail, to a programmer who had learned how to make procedure calls, every information processing challenge looks like a string of procedure calls. This algorithmic frame of mind is very hard to get rid of, once a person gets it contracted. And in order to become an efficient web developer, each and every procedure-besotted person must learn to let go of the procedure call driven world view.

Transitioning from local to distributed procedure calls

Most people learn how to program computers by implementing procedure calls that are local. What that means is that their world view is CPU-centric. In other words, the executable code in its entirety is sitting on a local box, usually right next to the developer's desk. The developer then intuitively grasps the world as consisting of a single Central Processing Unit which acts as a traffic cop, deciding on who gets to do what and when. The 'who' in the previous sentence refers to the procedure. Typically, a programmer will modularize his code into a fairly large number of procedures. These procedures will then be called at opportune moments, will perform the intended processing, and will then recede in the background.

This situation is the prescribed way for teaching newcomers how to develop software. It willfully ignores the real world issues, such as that there is very little use and need in having a 'desert island', 'single sink' scenario, where a computer user will be sitting alone engaged in the 'glass bead game' on the computer. In the world of business at least (and also in the world of entertainment), multiple users are a must. Meaning, CPUs and people will be networked (999 times out of 1,000).

There inevitably comes a time when a software developer wannabe realizes that a transition from a single CPU to multiple CPUs connected via the network, is unavoidable. At that point, the newbie usually gets struck by the immense complexities that such transition entails. And it is at that point that the newbies all over the world decide to drag in the procedure call world view into the networked world.

Now the problem of control in a multi-CPU environment arises. Who's in charge here? It used to be that a single human mind was in full charge and control over the execution of computing instructions, via the single CPU. Now, all of a sudden, multiple CPUs which are distributed across vast networks enter into the equation. The remote nodes start posing the problem when it comes to controlling the proceedings.

The only way for such developers to try and tame this complexity is to wiggle in their chairs and to fabricate some hairy-brained scheme for enabling remote procedure calls (or, RPC). Various mind-boggling protocols get implemented (CORBA, RMI, DCOM, SOAP, SOA, etc.) All of these protocols are incredibly arbitrary, non-compelling, frighteningly complex and brittle.

As we can see, this transition from making local, simple-minded procedure calls to making distributed, remote procedure calls never goes well. In the end, it inevitably results in fragile, brittle software that requires constant duct-taping and chewing-gumming in order to stay afloat.

Who's going down with the ship?

It is obvious to even a casual observer that the majority of software projects, today as in the past, fail miserably. As the complexities of the business scenarios continue to increase at a frightening rate, this rate of failure will only skyrocket.

At this pace, it won't be long before the software development industry turns into another Titanic, if the majority of developers continue to charge down the remote procedure call path. Developers who refuse to go down with the ship have only one option -- abandon the RPC ship and hop into the ROA boat.

But as we've already seen, the transition from navigating a huge, unwieldy ship to rowing a tiny nimble ROA canoe is proving more challenging than most developers are prepared to bear. That's why there must lie a major shakeout ahead of us, a shakeout that will separate the monolithic totalitarian software practices from the nimble, distributed ones. And ROA practices are certainly on the opposite side of the monolithic remote procedure calls practices.

How to make the transition?

It seems to me that, in the process of making the transition from RPC to ROA, the biggest stumbling block for the entrenched software developers lies in the fact that ROA proposes erecting an impenetrable barrier between the client and the resourses on the server. In the bad old RPC world, such barriers are unheard of. In other words, if the client is privy to the protocol that the service-oriented server had arbitrarily and unilaterally made up, the client can intimately manipulate the state of the server.

No such thing is ever possible in the ROA world. And that's the biggest put off for the honest-to-god procedural crowd. They simply don't seem physically capable of conceiving the world where it would not be possible to 'crack the code', learn the protocol, and then rule the server on the other end.

So in order to make a successful transition to ROA, these software dictators must learn how to abdicate from their dictatorial throne. What are the chances of that happening en masse any time soon?

Slim to none? What do you think?

Resource Representation

2007-08-15T10:50:00.000-07:00

Software developers seem largely incapable of understanding the difference between resources and their representation. Understanding that difference is absolutely vital in understanding REST and the Resource Oriented Architecture (ROA).

Is it a small wonder then that we are beginning to see articles like this one ("Why is REST so hard to understand?") pop up in the publications devoted to the web architecture? It is absolutely clear that we won't be in the position to advance ROA practices unless we reach a critical mass of people who do possess a solid grasp of what is the difference between a resource and its representations.

Right now, it seems like only an infinitesimally small percentage of software developers are aware of the significance of that difference. For example, if we say that there are ten million software developers worldwide, 9,999,000 of these developers are blissfully ignorant that such a thing as resources and their representation even exists. Which leaves us with only about 1,000 developers who truly understand what it all means. Not a very encouraging statistics. As a matter of fact, the situation today is right down dismal.

We need to do something quickly in order to remedy this catastrophic situation. My small contribution here is to attempt to clarify the difference between resources and their representation. I will use mostly concepts from everyday life for illustration purposes. I will then attempt to draw the parallel between these concepts and the concepts one encounters in the arena of software development.

What is a Resource?

Let's start from the top: a resource is anything that humans find useful. For example, a house might be a resource. Or, money might be a resource. We find houses very useful, because they offer certain level of comfort in our daily living. We also find money very useful, because it's a resource that enables us to get other resources, such as houses.

Ten years ago, when I was buying my first house, I wanted to see what kind of a house can I afford. To that end I met with a mortgage broker, who interviewed me in the attempt to assess my buying power. Among other things, he needed to know how much money can I allocate towards the down payment on the house.

Once I told him how much money I have for the down payment, he was able to plug that information into his spreadsheet and then tell me what kind of mortgage do I qualify for. So I was then all set to go out and shop for the house, right?

Well, not so fast! Just because I've blurted out a dollar figure that I had in mind for the down payment, didn't automatically mean that I fully qualify for that mortgage. Because, you see, my word in these matters was viewed as being largely subjective, that is, prone to miscalculations and all kinds of other aberrations. What the bank needed is a more objective assessment of my buying power. Talk is cheap, and anyone can claim that they have one hundred thousand dollars tucked away in their bank account, ready to be plopped in toward the down payment, but the reality might be somewhat different.

Because of that, the banks have devised a more impartial practice, whereby the applicant must provide what is deemed to be a more objective proof that the money really is in the applicant's bank account.

In this example, the money that has been allocated for the down payment on the house is regarded as a resource.

What is a Representation?

Now, the down payment money we are talking about may indeed be sitting in my bank account. But how am I to truly convince the mortgage broker that it is really there? Well, I use the means of representation, that is to say, I use something else that acts on behalf of that money. For example, I use my words to convey the fact that I do have $100,000 in my bank account. My words then represent my money.

But, as we've seen, that representation (my words uttered in a casual conversation) is apparently not good enough for the mortgage broker. He needs something more 'solid' before he is fully convinced that the money is actually there. Meaning, he needs a different form of representation.

Typically, what the mortgage broker would deem a fairly impartial and objective representation of my money is a piece of paper issued by my bank that claims that it is true that I do have $100,000 in my account. In other words, once the mortgage broker receives that piece of paper, he can 'take it to the bank', so to speak.

That piece of paper is what we call a representation of a resource. The real money is nowhere to be found in that representation, yet in a somewhat superstitious manner, all the parties involved almost religiously obey and agree that a piece of paper is as good as the real thing.

Back on the Web

The exact same protocol abides on the web. There, as in everyday, non-virtual life, we have resources which are tucked away somewhere, and all we get to see, feel and touch are the representations. Resources on the web are merely concepts that are being represented using other concepts. On the web, as in real life, no one ever gets to see, hear, feel and touch a resource. All we get is mere representations.

We are free to attempt to manipulate those representations. And even in the cases when our manipulation yields perceptible differences, we have no grounds to claim the we have actually manipulated the resource itself. All we can conclude is that the resource representation has changed under the influence of our actions, nothing more, nothing less.

For example, if there is a resource such as a volleyball tournament published on the web, I can get its representation by sending a request for it. I may then decide to destroy that tournament by manipulating the tournament's representation. And my action may result in an affirmative representation (i.e. the resource may respond by sending me its representation which states that it has destroyed itself). I may then conclude that the volleyball tournament has, indeed, been destroyed. But I really have no way of ascertaining that. All I can claim is that I have acted upon the initial representation of that volleyball tournament, I have sent it the request to destroy itself, and I have received the confirmation that it is now destroyed. However, the resource itself may still be alive and well, existing somewhere beyond the reach of the resource representation. The only thing that is manifestly certain is that from that moment on, the resource will refuse to render its representation for the incoming requests.

This is similar to how, in the real world, no one would ever suggest that I take the mortgage broker into my bank's safety vault and point him to the pile of cash that's sitting there, and tell him: "Here, here's my $100,000, please count them and let's get on with it!" No one goes straight to the resource itself in real life, and the same holds true in the digital life as it gets distributed on the web (not to mention that even the physical cash is merely a representation of some other concept, etc.; one can never possibly get to the bottom of it, that is, the real resource is nouminal, unreachable by us mere mortals).

Understanding this distinction is vitally important if we are to move forward successfully in building the functional web of information. It is my experience that none of the developers I've ever worked with understand this situation, and that all of them erroneously believe that they are, at all times, working with the 'real thing'. Nothing could be farther from the truth, so I implore all developers to take a moment and try to digest the difference between resources and their representations.

ROA and Hypertext -- a False Dichotomy

2007-08-14T10:50:00.000-07:00

It was around this time last year that I've published my first reference to Resource Oriented Architecture (August 8, 2006). I've received numerous feedback in the past year or so, most of which revealing the fundamental misunderstanding of what does Resource Oriented Architecture (a.k.a. ROA) really mean. One of the most virulent errors in judgment revolves around the false dichotomy between resources and hypertext.

This misapprehension, that I've been hearing a lot about, typically goes as follows: "yes, there are resources published on the web. But they could hardly be regarded as being pivotal to the architecture of the web. A much more fundamental concept that makes the web tick is hypertext."

This line of reasoning belies a fairly feeble mind. It fails to recognize and acknowledge that hypertext is also a resource. Such reasoning takes one type of a resource (i.e. hypertext) and pits it against all other types of resources.

The false dichotomy described here serves only to confuse the issue. Nothing is clarified if we arbitrarily single out one resource out of many existing ones, unilaterally assign that resource some 'special' status, and then claim that all other resources are of lesser importance. That smacks of sectarianism and therefore lacks the intellectual rigor of even the most basic kind.

It is high time we realize that the web is comprised of resources and resources only. Be it a regular text, a hypertext link, or any other media type. None of these resources warrant some elevated, special status. They are all equal citizens of the web.

Software Development Detox Part 4: Tribal Computing

2007-06-14T10:48:00.000-07:00

One of the absolutely most heinous types of software intoxication is what I like to call Tribal Computing. This is the form of computing that is tied in with the primitive, proprietary setup and is a remnant of the old days of early adoption of computing by the businesses.

Ontogeny recapitulates Phylogeny

Ernst Haeckel (a German natural historian) wrote in 1868: “Ontogeny, or the development of the individual, is a short and quick recapitulation of phylogeny, or the development of the tribe to which it belongs.” (in this context, ontogeny is the development of the fetus, and phylogeny is the evolution of a species). Haeckel was referring to the way the fetal development of mammals seems to parallel the evolution of all life on earth. The fertilized mammalian egg first resembles a single-celled amoeba, then a multi-celled sponge, then a jellyfish, then an amphibian, then a reptile, then finally becomes recognizable as a mammal.

Applied to the field of computing, one could say that the development of computing seems to parallel the development of human society. At first, the society was segmented into small primitive tribes, which slowly evolved into larger units, such as cities, counties, provinces, regions, countries, nations, and so on.

We are today standing on a threshold of global computing (i.e. the web). But, in certain ways, we still seem deeply entrenched in the primitive, stone age world of tribal computing.

Think Locally

Most software in operation today is severely localized. Meaning, it was built to live on a single box, single CPU. Everything about it is very closed, very proprietary, and extremely local.

Some software allows for certain level of connectivity, whereby other software systems are given an opportunity to connect to the localized software and share/exchange some information.

Only the latest, most recent batch of software products (the so-called social software) has left the world of tribal computing and is reaching out to the global computing space.

Control Locally

Most tribal software was/is built with an engineering frame of mind. Whenever we approach building something with an engineering outlook, we are striving to introduce maximum level of control into the system.

One of the most detrimental side effects of building software with an engineering slant is the temptation to retain the state of the conversation. As we've seen in our first installment (State), the best way to create brittle and buggy software is to insist on retaining the state of the conversation that had transpired during the operation of the software product.

In addition to that, insisting on staying local (i.e. tribal, single box, single CPU etc.) means that the point of control also stays tribal. There is a single authoritative instance that claims to know everything and that controls what can and cannot happen on the system. That instance then becomes a single point of catastrophic failure.

Relinquish the Control Globally

In contrast, non-tribal software exhibits stunning capabilities for growth thanks to relinquishing the rigid engineering attitude. One of the fundamental reasons why web is such a spectacularly successive computing platform lies precisely in this abandoning of the tribal past and moving beyond the need to control and retain the state of the conversation.

By deciding to not care about the state of the systems engaged in the conversation, the non-tribal, globally oriented software is free to grow in any direction and to scale to any level of complexity.

We will see in the next installment what are the most optimal ways to achieve that level of robustness. Stay tuned.

Software Development Detox Part 3: Join the Conversation

2007-05-29T10:47:00.000-07:00

So far we've established that the unsophisticated software solutions tend to be brittle and unreliable due to the following erroneous decisions:

Keeping track of the state of the conversation between the client and the server
Choosing to expose services through messaging interfaces (a.k.a. Remote Procedure Calls)

Today we're going to look into the minimal requirements for establishing the platform for enabling successful implementation of the business computing. We will see how important it is to separate the state of the conversation, that may have transpired between two or more business parties, from the transformations of the state of the participating parties. Also, we'll look into the advantages of abandoning the messaging interfaces when conducting business conversations.

Sustainable Business Strategies

Abandoning the world of software for a moment, let us examine the strategies that facilitate success in the world of business. Experience has shown that short term success is certainly achievable by exclusivity (i.e. by locking customers into some sort of a proprietary business model). However, for achieving any level of sustainability, businesses must, willy-nilly, open up and introduce free choices and embrace diversity.

In order to attain happy equilibrium of a sustainable business growth, one component of the business interaction must be optimized -- conversation. Businesses thrive on open-ended conversation. What that means is that while initially only two privileged parties may begin business conversation, other interested parties should be able to join and participate at any point.

For that to happen successfully, the barriers to entering the stream of conversation must be placed extremely low. In practical terms, anyone interested in joining the conversation should not be expected to learn any arbitrary set of rules, any new language, or any new protocol.

In addition to this fundamental requirement, parties joining the conversation midstream should also be able to easily examine the history. Anyone joining the conversation should be able to immediately gain access to the 'minutes' of all the previous conversations. That way, an interested party could, at their own convenience, examine the minutes and learn about some vital stats of the ongoing business transactions.

These two (the ability to join the conversation at any point in time and the ability to examine what had transpired up until that moment) are the most vital and essential requirements for conducting successful, sustainable business transactions.

Sustainable Business Computing Strategies

When it comes to achieving sustainable levels of business computing, the challenges tend to be even greater. Computing infrastructure and practices, that are being placed between various business parties interested in joining the conversation, serve to raise the barriers to entry.

Still, the promise of information technology is to make those barriers even lower than they tend to be for the non-digital business interactions. And in all truthfulness, that really is the true calling of this technology. So, the question then is: what went wrong?

The simplistic answer to the above question is that someone along the lines made a couple of wrong decisions (see the erroneous decisions listed above) and created a complicated situation resulting in businesses being unable to join interesting and potentially profitable conversations that are happening online.

Our job, then, is to remedy this situation, and to bring the technology back to the level where pretty much any interested business party is capable of easily joining the conversation. Not only that, but any party should qualify for easily examining the history of the conversation by replaying the 'minutes' of what had transpired while they were away.

Simplifying the Medium

In the world of information processing, bits and bytes could represent anything. But in the world of business computing, the medium that is of most interest is text. So in most situations, what is of particular business interest is content that is rendered as text.

Of a much lesser significance, but still figuring rather prominently in the world of business computing, are images. Less prominent media types would be audio and video.

So the viable computing platform that would fully facilitate successful business conversations is mostly focused on the content rendered as text. That content should be marked up in order to achieve certain level of semantic order. But the markup shouldn't be blown out of proportions.

Simplifying the Protocol

Once the medium for exchanging business content is sufficiently simplified, we must make sure that the channels for communicating that content are also fully simplified. Instead of expecting any interested party to learn the intricacies of a foreign language that is unilaterally enforced by the vendor, we must offer a severely restricted list of possible actions that are publicly vetted and extremely non-volatile.

For any resource that the businesses may be interested in, we can safely establish that only four actions would suffice when it comes to maintaining that resource:

Add resource
Remove resource
Modify resource
Fetch resource

The above four-pronged protocol is sufficient for conducting any business conversation. No further elaboration on the business computing protocol should ever be required (there is no need for any kind of messaging interfaces).

Joining the Conversation

When an interested party learns about the ongoing business conversation and wants to join in, all it has to do is fetch the state representation of the resource in question. From that point on, the interested party is free to propose any transition of the state of that resource. In addition to that, the interested party is also free to replay the conversation as it had unfolded prior to joining the conversation.

The above scenario is universal, and it applies across the board. Following this model, we can ensure that any business parties will have a successful transition to participating and contributing to the ongoing stream of business conversations.

Software Development Detox Part 2: Remote Procedure Calls

2007-05-29T10:45:00.000-07:00

We've seen in our first detox installment that it is detrimental to maintain the state of the conversation between participating parties. The conversation that may have occurred between the interested parties and a targeted resource may have affected the state of the targeted resource. But that doesn't mean that the various stages of the conversation itself need to be retained.

Today we'll look into another form of software intoxication; this one has to do with how is the conversation implemented.

Typically, when two or more parties get engaged in a conversation, they tend to accomplish successful conversation by sending each other specific messages. For example, if an airplane is approaching the airport, the messaging between the pilot and the control tower gets peppered with code words such as "roger" and "over". That way, the conversation protocol gets established in the attempt to minimize the noise and maximize the signal.

In a loosely coupled world of business computing, free market forces dictate that many solutions providers can compete for the most successful solution. That situation creates a wild diversity of proposed conversation protocols. What's happening is that basically any vendor (i.e. solution provider) is free to come up with a completely arbitrary set of formal rules outlining how is the conversation going to take place. That creates a hell of a noise in the solution space, resulting in brittle and faulty implementations.

The culprit most of the time boils down to the methodology known as Remote Procedure Call (RPC). The remote procedure proposed by the vendor is a completely arbitrary, unilaterally constructed expression that the vendor expects the consumer to learn. In addition, the vendor reserves a unilateral right to change the expression encapsulated at the RPC level, and the consumer has no recourse but to re-learn the intricacies of how to talk to the vendor.

This unfortunate situation creates endless problems and headaches. All the consumers of the proposed vendor services are expected to keep an ever growing inventory of those custom Remote Procedures, and are on top of that left vulnerable to the future changes of that inventory.

Because of that, the effective detox therapy strongly advises against using the RPC model when architecting and designing your software solution. We will see in the next installment what is a much better way to go about accomplishing the robust software architecture.

Software Development Detox Part 1: State

2007-05-26T10:42:00.000-07:00

In the first installment of the software development detox program, I am going to review misconceptions and misunderstandings of one of the most fundamental software concepts -- state.

The problem with understanding state in the context of information processing/software development arises when people fail to recognize and acknowledge that there are two distinct aspects of state in the arena of software development. By bundling up these two aspects, people end up projecting an incorrect picture and consequently paint themselves into a corner by choosing the unsuitable architecture.

I am now going to (temporarily) abandon metaphors (such as 'paint oneself into a corner' etc.) and switch to using simple, albeit somewhat exaggerated examples.

Software Types

In this example, I am going to review a common occurrence of a typical software construct, such as date. Date is an abstraction devised to encapsulate and express human concept of time. In the world of information processing, we use software constructs, such as types, to encapsulate and express abstractions such as calendar date.

Suppose someone offers a software abstraction (i.e. type) called CustomDate. This abstraction is supposedly capable of doing accurate date calculations, and is endowed with certain conveniences. One such convenience being the ability to express, in calendar terms, such human concepts as 'tomorrow', 'yesterday', 'next week', etc.

So we see that this type is capable of certain behavior (such as being able to answer the question 'what date is tomorrow?', etc.) But, in addition to discernible behavior, software types typically also possess state. For example, our CustomDate may possess a state of knowing what date is year-end.

This state may change (different corporations have different year-end dates). And the instance of the type is expected to remember the changed state.

What can you say and how it gets interpreted

Upon acquiring a new software type, such as CustomDate, we will be expected to learn about its capabilities. We are not expected to understand how it is working. We're not even expected to understand all of its capabilities. We are free to pick and choose.

For example, if the CustomDate possesses 50 different capabilities, and all we want from it is the ability to tell us what date is the year-end, we should be able to safely ignore the remaining 49 capabilities.

To violate this basic agreement would result in creating brittle, unreliable software. Here is one fictitious example that illustrates this problem:

If we instantiate CustomDate and assign that instance a handle such as customDate, we should then be able to talk to that instance. If we are only interested in learning about our company's year-end date, we can send a message to our customDate instance, as follows:

customDate.year-end

In response to receiving that message, the customDate instance will return the actual year-end date to us.

The above described scenario should always yield the same behavior. There shouldn't be any surprises in how an instance of customDate behaves upon receiving the year-end message. If there is even a slightest possibility that the established message may render different, unexpected behavior, our software is not only brittle, but extremely buggy.

By now you may be wondering how could there be a possibility that the above scenario ever yields any different behavior than expected? Let me explain with another example:

We've learned so far that, when dealing with an instance of customDate, we can say year-end and it will be interpreted as a question that reads: "could you please tell me what is the year-end date?" Consequently, the representation of the correct year-end will get rendered and served as a response to our query. We've thus realized that an instance of customDate has state. That state (i.e. the actual value of company's year-end date) is the only state we're interested in when dealing with this software construct.

However, as we've mentioned earlier, this software construct may have 49 other capabilities and states, of which we know nothing. Now, the fundamental principle of software engineering dictates that we are absolutely not required to know anything about any additional, extraneous states and behaviors that a software construct may bring to the table.

Regardless of that prime directive, people who are not well versed in designing software solutions tend to violate this dictum on a daily basis. The way to violate the prime directive would be to introduce certain state/behavior combo that will modify how the question gets interpreted. One can imagine how easy would it be to add a capability to CustomDate which will turn it into a currency conversion utility. This example is admittedly unrealistic and exaggerated, but I chose it to illustrate the foolhardiness of arbitrarily assigning various capabilities to a software construct.

In this example, an overzealous green developer may add a capability to CustomDate that will put it into a "currency conversion" mode. If someone else is using the same instance of CustomDate and puts it into this "currency conversion" mode, that change in its state may modify the behavior of an instance of CustomDate, rendering the response to the

year-end question unintelligible.

Let's now run this hypothetical scenario:CustomDate gets instantiated as a resource on the server

A message arrives from a client asking the resource to convert 100 USD to Canadian dollars

An instance of CustomDate (i.e. customDate) puts itself into the "currency conversion" mode and renders the proper currency conversion

The client then sends a message to customDate asking it for a year-end

The instance renders an answer that corresponds to the value of 100 US dollars at the year-end
The above answer at step 5 comes as a complete shock to the client who asked for a year-end; the client wasn't aware that the instance can be shape-shifting and consequently may not always be returning dates when asked about the year-end.

In other words, what you can say and how it gets interpreted changes based on the state that an instance of the type may be in. A very bad situation, guaranteed to render that particular software program dysfunctional.

Statelessness
We can see from the above example how disastrous it can be to attempt to manage the state of a resource. In our case, we've been managing the state of an instance of

CustomDate, keeping track of when is it a date rendering machine, and when is it a currency conversion machine.

This tracking of the state resulted in the breakage of the working code. If we had abstained from keeping track of the state of the instance, the problems wouldn't have emerged in the first place.

From this we see that the only way to achieve robust and reliable software is to ensure that its constituent components are stateless. No memory of what had transpired during previous conversations should be retained.

However, keep in mind that we must distinguish here two types of states:

Entity state
Conversation state

It is this conversation state that is troublesome. Entity state is perfectly valid, and should be memorized. In this instance, entity state would be the fact that our company's year-end is October 31.

Keeping track of what transpired as clients have interrogated an instance of a software component, and then retaining that state, is always disastrous. And yet that is how most inexperienced software developers tend to architect and design their software.

Coming up

In the next installment, we'll look more closely into how to architect and design stateless software.

Contracts and Coupling in Resource Oriented Architecture

2006-12-25T10:41:00.000-08:00

We've seen that most software developers in operation today are besotted by the vendor model. One of the symptoms of that malaise is the deeply ingrained idea of a contract. Basically, it would appear that, in the minds of typical software developers, there cannot possibly occur an interaction between software components without a predefined contract.

Of course, the contract they are talking about follows (albeit covertly) the implicit revenue model as ingrained in a typical software vendor's business plan. Software vendors make their living by locking into a contract with their customers.

But in cases where the utilization of the software product is not tied with any revenue model, the notion of a contract tends to lose its appeal. Like, for example, in cases where an open source community is bettering the services by utilizing the publicly vetted web standards and protocols.

There are no Unilateral Contracts

The web is a platform that publishes its capabilities in a unilateral fashion. Similar to any other publicly vetted service, the web is calibrated to serve the least common denominator.

For example, in all developed countries in the world there is a publicly vetted service called traffic. Various vehicles are free to enjoy the commodities that the public roads offer, mostly free of charge. However, the reason this service is made so freely available without strings attached lies in the fact that it comes with an extremely pronounced and unbendable bias. That bias states that all vehicles moving along the public roads must keep to the right side of the road.

If this system wasn't calibrated in such a way, it would be impossible to offer the amenities for conducting the public traffic.

So we see that the utmost flexibility of an infinitely complex system, such as public traffic, is only possible if that system is heavily calibrated in a non-negotiable fashion.

And because that calibration is non-negotiable, it does not qualify as being called a contract. In other words, if you're driving your car down the road, you're not holding the traffic contract in your hands that outlines all the details that bind you and the road owner. The road owner (i.e. the public) dictates the rules, and does so unilaterally. You have no say in whether you accept to keep to the right side of the road while driving. You simply must obey the unilateral rule, or suffer the consequences.

To put it more mildly, you must 'get used to it'.

Same principles apply when you're 'driving' around the web. The web specifies upfront the non-bendable rules and regulations, and you have no say in whether you like them, don't like them, whether you're planning to obey them or disobey them. The web doesn't expect you to make your stance known, and the web absolutely doesn't care.

Because of this unilateral set of rules, there cannot be a contract between the sever software on the web and the web client.

The Web is Uncoupled

Because of the lack of a bilateral contract between a web server and a web client, it is not possible to talk about any form of coupling on the web. Tight or loose coupling, which are terms tied with the world of software vendors' applications, lose any meaning on the web.

There are two reasons for this change:

On the web, the server need not know anything about the client
On the web, the server is blissfully unaware of the existence of any other server

All that is known on the web is that it is a sprawling collection of resources; these resources are easily accessible via the publicly vetted protocol (i.e. URI), and these resources know how to represent themselves and how to make their own state transition upon request.

Thanks to this particular bias, the web is the most scalable, the most interoperable, and the most robust, resilient and comprehensive computing platform ever.

Resource Oriented Architecture and Quality

2006-12-23T10:38:00.000-08:00

In his recent post exposing imaginary limitations of the REST paradigm, Udi Dahan laments the lack of explicit dialog that would discuss the benefits of adopting REST. More specifically, he writes:

"What I haven’t heard is how I can use it (i.e. REST) to build large-scale distributed systems that are controlled by complex logic."

Here, I believe he is talking about his definition of quality. Building large-scale distributed systems that are controlled by complex logic sounds like a very costly proposition. Something one would not attempt just for the heck of it, just to kill a rainy weekend.

Because such a project entails a price tag of hundreds of thousands, if not millions of dollars, it is not to be taken lightly. Also, it is not something that happens very often. I would venture out to say that most of us have never been involved in such a project, nor will we ever find ourselves involved in such a thing. It is not very often that businesses embark upon building such complex distributed systems. Yes, eBay and Amazon etc. are such systems, but you could almost count these beasts on the fingers of one hand.

So the question that mystifies me a bit here is: why? Why worry about building such a large-scale distributed complex system? What's the business case for it? How many of those are you planning to build next year? What's your compelling reason for going after such a risky venture?

But assuming that Udi has his valid reasons, and that he has millions of budget dollars to back him up (which, frankly, I seriously doubt), let's engage him in his quest, for the sake of having fun.

What I'm assuming here is that, to Udi, large-scale complex distributed systems spell quality.

So You Want to Build a Multi-million Dollar System?

Why would a system we're building end up costing millions of dollars? There are several possible reasons:

We are using primitive technology, and thus it's taking us too long to build
We don't understand what is the system supposed to be doing, so we're making things up as we go
The scope of the system is on a runaway curve, and so we end up with explosion of features (i.e. featuritis)

In the case of primitive technology, we can imagine trying to dig the Panama canal by giving each worker a wooden spoon. That technology would be considered very primitive for the intended job, and would therefore result in the astronomical cost of the project.

Similarly, if we attempt to build an equivalent of amazon.com or ebay.com using assembler, we'll end up paying astronomical price for using such primitive technology.

In the case of not understanding the goals that the system under construction is supposed to deliver, 'fishing for results' would invariably turn out to be prohibitively expensive.

Finally, in the case of scope runaway, featuritis and bloat are well known for being insanely expensive.

Let's say Udi is unphased after reading this, and still wants to proceed with building his multi-million dollar large-scale complex distributed system. He is now curious how can REST and ROA be of any benefit to him. Let's explore.

Can REST and ROA Help?

My knee-jerk answer is: not really. But let's not give up so easily.

First, let's make two things clear:

If we don't understand what is the system supposed to be doing, then no amount of technology can salvage us from dying a miserable death
If we contract the featuritis disease and start bloating, there equally is no technological cure; we'll be doomed to collapse under our own weight

So let's assume here, for the sake of having fun, that Udi is immune from the two forms of malaise mentioned above. Let's say he has a very clear understanding of the goals of the large-scale distributed complex system he is trying to build, and let's also assume that he is immune from the featuritis and bloat (both assumptions being extremely unlikely, by the way, but let's pretend they've been licked).

Seeing a Problem where there isn't Any

OK, so Udi is concerned about the following:

"There are business rules that control which data should be returned and when. All this has been known for quite a while and has been modeled around transaction isolation levels in the OLTP sense, and needs to be handled at the application level when it comes to intelligent caching. I haven’t heard anything about REST that deals with this."

My question is: why on earth should REST be expected to deal with that? REST and ROA, as their names imply, specialize in dealing with Resources and their Representations. They don't specialize in dealing with the infrastructure that is the underpinning for the resources.

For all we know, that infrastructure might very well be implemented as OLTP, or perhaps as something else. Same as the ability of the RDBMS to index the data stored within it might be implemented using various algorithms. But in reality, who cares? The important thing is that it works and meets our expectations.

Next, Udi asks:

"If, as a result of updating one entity, other entities are affected how is that information propagated back to the caller?"

This is simple: the caller consumes resources by dealing with their representations. This is the essence of REST. The ground rule in ROA is that raw resource can never be touched by the client, or by the caller. All one is allowed to work with is the resource's representation.

Udi talks about updating an entity. I'll try and translate this into ROA speak. Instead of updating an entity, in ROA we speak of requesting a state transition for a resource. The way it works is as follows:

A resource gets identified (via the URI)
That resource then gets located (via the corresponding URL)
A request is then sent to that resource to change its state (i.e. to make a state transition)

At the end of that chain of events, other resources (or, 'entities' in Udi's parlance) may or may not get affected.

His question, then, is: how is that fact (i.e. that other resource got affected during the state transition) propagated back to the caller?

The answer is glaringly simple: it all depends on the use case. If the use case requires that the caller be notified of the state transition, then the appropriate representation of the new state will be presented to the caller in the form of a response.

One really feels like stopping at this point, and asking: what's the big mystery? Why is it that all these highly paid software architects are having such hellish time grasping this extremely simple concept of resources, their states, and their state transitions?

I am indeed forced to conclude that they seem to working really hard on trying to see the problem where there isn't any. Is that why they get paid those big bucks?

Here's the Skinny

Udi even gives us a tiny homework:

"Let’s say I have a simple rule: When the sum of all orders for a customer in the past quarter increases above X, the customer is to become a preferred customer. Do I PUT the new order resource to the customer resource, or should I have a generic orders resource that is “partitioned” by customer Id? Once the change to the customer occurs, should I PUT some change information somewhere so that the initiator can find out where it should go look for the change? And how should I be handling transactions around all these resources?"

Again, he is confusing apples with oranges. Let's break it down gently:

A customer is a resource. We don't know how is that resource implemented behind the firewall, and we really, really don't want to know! This point is crucial to understanding ROA. Please keep in mind that ROA is definitely not about the how.

OK, we now need to understand that a resource (i.e. customer) has state. For example, the resource could be in the state of being a preferred customer, or not being a preferred customer. However, we don't know, we have no idea how is that state implemented. Also, we don't want to know anything about those gory details.

On to how does that state change. Recall that REST stands for "Resource State Transition" (actually, this is incorrect, and this section talks about ROA, not REST; please see the correction notice at the end of this post). So 'state transition' is the crucial phrase here. Obviously, a resource is capable of making a transition from one state to another. Such as making the transition from the state of not preferred customer to the state of preferred customer.

How/when does that state occur, you ask? Well, that transition is driven by a particular event that must be described in the use case scenario. Like in Udi's use case above, in case of an event when the resource's state (such as total value of accumulated orders) reaches a predefined threshold, that resource knows how to make the transition from not being the preferred customer to becoming a preferred customer.

Again, we don't know how is this transition taking place. There are millions of conceivable ways that transition could take place, but the important thing is that it does.

All of Udi's questions surrounding this hypothetical scenario boil down to how, but that's largely irrelevant. Somehow the resource knows the proper way to make the transition from one state to the other state. When the client then makes a request for getting that resource, the resource in question will know how to represent its state, including the newly won 'preferred customer' feather-in-the-cap.

Correction: I've just realized that, in my eagerness to explain the encapsulation that Resource Oriented Architecture offers, I've made a mistake and erroneously ascribed state transition to REST. Actually, REST stands for Representational State Transfer, not Resource State Transition. The latter is a characteristic of ROA, not REST.

Sorry for the inadvertent error.

RESTafarian Jihad

2006-12-21T10:37:00.001-08:00

I've just been accused of waging a RESTafarian Jihad. I feel that this accusation is unjust, and that it is uncalled for. So I need to explain and defend my case here.

Jihad is a strong word in this context, because it came to mean 'holy war' in the modern parlance. And I'm nowhere near waging a holy war on anything, least of on the non-REST platforms.

To be more specific, I am absolutely not interested in converting the Remote Procedure Call (RPC) people to REST. True, approximately 98% to 99% of all software developers I've ever met cannot imagine building a distributed software system without utilizing some form of RPC, so for these people, Resource Oriented Architecture (ROA) is meaningless.

But as I've already explained in one of my recent posts, I view these people as a write-off. Basically, I think there is nothing we can do about that, and that it's time we cut our losses and move on with our lives.

So you see, I'm leaving the RPC crowd (which means almost all software developers in existence today) alone. I'm not waging any Jihads, I'm not plotting to convert anyone.

Instead, I strongly believe that, as it has always been the case, the simpler system, with lower barriers to entry, will somehow win in the end. I don't really know how, but God's ways are mysterious, so we'll see. But same as CORBA had to disappear because it was just too unwieldy, web services and SOA will also be forced to vanish from the scene.

At that point, ROA will take the lead. Right now, we're preparing for that portentous event. It will come in its own sweet time. We're not rushing anything.

We know that, eventually, people will abandon the ways of the past century, and will embrace the twenty first century ways of employing high technology.

Are Software Developers a Writeoff?

2006-12-15T10:35:00.000-08:00

My friend Ted introduced me to the idea that we won't be able to see the real effects of the web until the time when the last person who can remember the days before the web dies out. In his view, if you can remember life before the web, then you're not capable of fully grasping the web.

I tend to agree with him. The web is such a drastic paradigm shift that we're not really capable of fully grasping its significance.

But the young and upcoming generations, who can't remember the days without the web, will be much better equipped to cope with it. Same as I take electricity and indoor plumbing for granted, my children take the web and the perpetual availability of the wireless connectivity for granted.

And that makes them a different kind of persons than I, their father, am. And I find that to be a very fascinating phenomenon.

New Breed of Software Developers

Similar to how my children are a new breed of humans who have a completely different outlook on the web than I do, there is an upcoming new breed of software developers who are diametrically opposite from the existing, entrenched batch of software developers.

The existing developers, as I've been discovering through gruesome series of interviews lately, seem completely oblivious to the incredible powers of the web. All the developers I talk to seem to think that the only way to accomplish something in the software development arena is to use the function calls. They all follow the software vendors' lead, which typically means more complexity, more heavy lifting, less palatable solutions.

Because this tendency is so pervasive and ubiquitous among the existing software professionals, I have reached a point where I'm pretty much ready to throw the towel in. In other words, I'm this close to call all your typical software developers, all those entrenched Java, J2EE, .Net, Oracle etc. developers a big fat writeoff.

I am slowly starting to think that this workforce, as huge as it is, is pretty much useless when it comes to business computing. Some of them may be good for developing certain components of the computing infrastructure. But as soon as we get to the point of utilizing this infrastructure for the benefits of the business, they all turn to be nothing but a writeoff.

And by 'writeoff' I mean more harm than good.

This is why I'm forced to turn my attention to the young and upcoming generation of software developers. We need to ensure that these young people don't get tainted by the tool vendors' agendas and business models. We need to make sure they don't get caught in the remote procedure call hell.

Resource Oriented Architecture and Vendors

2006-12-15T10:34:00.000-08:00

Suppose we get to the point where Resource Oriented Architecture (ROA) becomes prominent and gains wide adoption. Given the ubiquity, pervasiveness and popularity of the web, it is not a far fetched proposition. After all, the native web is strictly resource-based, and a Resource Oriented Architecture that utilizes that orientation seems like a perfectly natural fit.

Now, if this re-orientation happens, what will the vendors do?

Vendor Business Model

Software vendors are in the business of making profit from selling software based solutions. Traditionally, the most efficient solution, from the vendors' profitability point of view, tends to come in the form of a tool. This is why we tend to call these vendors 'tool vendors'.

Because of this orientation, vendors tend to favor system architectures that are complex, convoluted, difficult to fathom and master. Of course, anything can be quite easily made more complicated than it already is. In other words, it is quite easy to muddle the issue, to add complexity, to increase brittleness. You don't have to be very smart of very competent in order to make things more complicated.

As a matter if fact, complicated situations are always a surefire sign of stupidity. If, on the other hand, you find systems that work but are nevertheless very simple and easy to navigate, that invariably means that someone quite intelligent has devised such a solution.

But, since an easy to understand, easy to navigate and use system is non-profitable from the vendors' point of view (i.e. such systems do not require 'tools' in order to be utilized), vendors do not see any value in playing along with them. Vendors prefer murky, complicated architectures. Getting you involved with such ungodly systems and architectures is a very lucrative proposition for the vendors, because then they have a very good justification to approach you and to start selling you their tools.

Do We Need Tools?

Take a practitioner of a well established profession, such as a lawyer, or a mathematician. These professionals deal with extremely elaborate, intricate systems and bodies of knowledge. They are expected to navigate very sophisticated waters of highly formalized systems.

But what kind of tools do they need in order to do their jobs? In reality, very few. For a lawyer may need nothing more than a typewriter, and a mathematician actually needs even less tools -- a paper and a pen will do just fine.

How come? It's simple -- their job is largely conceptual by nature.

Now, the question is: how is that different from software development? Why do we think that software development is not conceptual by nature, in ways similar to mathematical development?

The thing is, software development is not different in any significant ways from mathematical, or legal developments. They are all largely conceptual activities.

And realistically speaking, all these conceptual activities don't need many specialized tools. It is a great fallacy, perpetrated on us by the tool vendors with their business agendas, that software development needs an elaborate set of tools.

Choose the Right Foundation

And since we are dealing with a fairly conceptual activity when developing software, it is of crucial importance that we select the right foundation, the right philosophy, and the right paradigm from where we can operate successfully.

Choosing the right starting point will ensure that we don't get entangled in the morass of complexity, brittleness, analysis paralysis and the like. Otherwise, it would be devilishly easy to slip and end up in the blind alley.

If we now review the philosophy and the paradigms that the tool vendors are offering us, we see that they prefer to side with complexity, heavy lifting and hard work. The vendors are proposing we adopt systems that are architected for ever growing complexity. Theirs is the business model of introducing a big hairy problem, and then selling the solutions that would be pain killers.

So what's the alternative? We propose that instead of aiming at acquiring pain killers from the vendors, we focus our efforts on acquiring vitamins.

And in our vision, vitamins would be the publicly vetted standards upon which the web, as a universal computing platform, is built. The web offers us simple, extensible, scalable, interoperable and linkable architecture. As such, it is very beneficial to us in the sense that it does not introduce the pain of complexity which then needs to be handled with pain killers.

It's Not the How, it's the What

2006-12-06T10:33:00.000-08:00

By now, we all know the intentions are older than implementation. There were times when foolish software developers used to think the reverse was true, but thankfully today most of them have been browbeaten into aligning with the program.

So everyone seems to agree nowadays that articulating the what (i.e. the intention) is much more important than articulating the how (i.e. the implementation). So where's the problem then?

Web Development Challenges

While every prudent software developer had by now learned the lesson in separating the what from the how, they still seem to be applying it at too low a level. Similar to how the budding software developers of the olden days used to build those monstrous huge bloated monolithic programs that ended up being totally unwieldy and unmanageable, today's web developers are falling into the same trap of building similarly huge, bloated, unwieldy web sites.

Or, let me put the question up to you like this: what would you rather build -- a huge monolithic tightly coupled software program, or an open-ended set of smallish, self-contained, loosely coupled software units, that are easy to implement, easy to test, easy to troubleshoot and debug, easy to deploy?

Of course, if you are a software developer worth even a tiny bit of your own salt, you'd go for the latter. Some of us still recoil in horror when the flashbacks of bad old days of monstrous monolithic software programs come back to haunt us. Ugh, not a happy memory!

But it's really funny how these time honored principles of separation of labor, inversion of control, etc., do not seem to translate at all when being ported to the web development. Here, when developing web sites, budding software developers are more than happy to fall into the same old trap of indulging in muddling the concerns and seeking a single point of control. Which ends up in incredibly messy, monolithic, coupled and unmanageable web sites.

This, again, is the outcome of blindly rushing in to grab the how! How do I do such-and-such in Rails? How do I do such-and-such in Ajax?How do I do such-and-such in Flex? And on and on...

People never seem to stop to think what. What do I need to do in order to make this work? Forget about how to do something, and focus on the what.

How you do something is really not a big deal. 99 percent of the time it's just a short google search away. So why sweat the small stuff?

The what is much tougher. Google will not be able to help you there. You'd need your own cool head, your own common sense to figure that one out.

The surefire sign that you're charging down the wrong path is if you find yourself building large monolithic web sites. You know the ones that rely on the function calls, with one central point of control? The ones where it gets progressively difficult to introduce any new capability?

You know the ones I'm talking about. I bet you that right now you're working on one such site.

Well stop! Same as you wouldn't work on a single humongous big ass class, but would rather chop it up into dozens and dozens of smaller, more manageable classes, you must do the same with that web site. Abandon it. Chop it up. Give up the single point of control, give up the irresistible need to shoehorn everything through the single funnel of omniscient point of control.

Think back to your other software development skills. Remember how tough it was until you've learn how to properly delegate?

Well, now's the time to do the same delegating, only not with objects, but with web sites. Treat each web site as a single independent component, that specializes in offering some useful resources. Then open us the traffic gates. Let those independent, standalone 'components' interact, and see what happens.

See, I gave you the what; the how should be a piece of cake for you now.

Web Model for Business Computing

2006-12-05T10:31:00.000-08:00

After reviewing the Vendor Model for Business Computing, it is now time to look into the web model. To refresh your memory for a moment here, the fundamental differences boil down to how each model treats data vs business logic.

As we've seen, vendor model tends to freely expose the data, while jealously hiding the business logic. It operates on the fundamental premises that data is deadwood that can only be animated thanks to their marvelous business logic. Without their magical proprietary processing logic, the data remains useless and thus worthless.

Consequently, vendor model insists that we pay good money to gain access to their beloved processing logic.

Web Model is for Humans

Unlike the vendor model, the web model was built with human beings in mind. Instead of insisting that some cryptic, arcane and privileged proprietary business logic be the animator of the data, web model leaves data animation to the most suitable subjects -- human users.

This is why products based on the web model have nothing to hide when it comes to the processing logic. They don't deem this logic very important. As a matter of fact, such products leave almost all of the decisions as to how one would like the data processed to the human users.

Which is where that decision rightly belongs, after all is being said and done.

Web Model Hides the Data

Quite surprisingly, as much as the web model liberates us from the slavery of being governed by some secretive proprietary processing logic, it is equally adamant that we don't gain access to the data.

Instead, web model insists that we only gain access to the representation of the data. But as to what the actual data looks like, or how does it really feel like, we remain forever mystified.

We'll talk some more in the upcoming series about why is such arrangement a much better thing for all consumers of software.

Web Model vs. Vendor Model

2006-12-04T10:30:00.000-08:00

There are two models of business computing: vendor model and web model. Vendor model is presently the entrenched model, the one that everyone is locked in.

Web model is brand new. Ask any software developer about the web model of business computing, and they wouldn't know what that model is nor how does it work.

But that doesn't mean that the web model is irrelevant. Quite the reverse, it is the most significant thing to ever happen in the world of business computing.

How is the Web Model Different?

Perhaps the best way to understand the web model of business computing is to discuss how is it different from the vendor model.

Before we delve in, let's just quickly recapitulate what are the characteristics of business computing. You may recall that any business computing model must be based on three fundamental characteristics:

Identity
State
Behavior

In the business computing parlance, and for the purposes of our discussion, we can say that the state is equivalent to data, and the behavior is equivalent to the program logic.

Getting back to the difference between the vendor model and the web model of business computing, we will see that the two models fundamentally differ in the way they treat data and programming logic.

Vendor model insists on exposing the data and hiding the programming logic.

Web model, you guessed it, is the exact opposite -- it is based on the architecture that hides the data and exposes the programming logic.

Both models have their use, and in our ongoing series on the Resource Oriented Architecture (ROA, which is the underpinning of the web model), we will be discussing the advantages of the model that hides the data.

Stay tuned (and don't feel shy to ask any questions).

Resource Oriented Architecture -- Growing Pains

2006-12-02T10:21:00.000-08:00

I've already discussed the problems related to how traditional software development workforce seems incapable of grasping Resource Oriented Architecture (ROA). Today, I will tackle another of the misconceptions that seem to inevitably keep popping up in the software development circles.

Someone who very appropriately claims that adding simplicity is an engineering mantra, wrote about the Promise of SOA, REST, and ROA. Here, we will only focus on his view of ROA:

ROA is an intriguing proposition. Applications are freed from worrying about services at all. A resource, which is effectively an entity, exposes the state transitions it is willing to accept. Applications don't even care what the resource is, they simply decide whether they are interested in the state transitions available. The level of omniscient inference with this approach is difficult to even explain. So, if the resource is a tennis court and the state transition is reserve (is that a state change or operation?) an application can decide, without caring about the resource per se, that it wants to reserve it. Of course if this is a seat at a concert you may be committing to a financial transaction. At the very least, the state may change back without the applications knowledge (except it's omniscient so that isn't possible).

Let's pick this apart, to see if we could identify sources of confusion.

Applications and Services

The post quoted above states that: "Applications are freed from worrying about services at all." First off, it is important to understand that ROA is not about applications. This was hinted at in my post on the golden age of software applications.

ROA is, as its name implies, all about resources. Some people claim that it's basically the same as being all about services, but I strongly disagree.

Resources and Entities

He continues: "A resource, which is effectively an entity..." Maybe I should let this slide, but let me just point out that a resource is effectively a resource. No need to drag entity into the picture.

Acceptable State Transitions

Moving along: "[Resource] exposes the state transitions it is willing to accept."

This is absolutely incorrect. Resource only exposes its representation, not the state transitions it is willing to accept.

Applications and the Attempts to Simplify Reality

"Applications don't even care what the resource is, they simply decide whether they are interested in the state transitions available."

He is again talking about applications. And by 'applications' I suspect he means blobs of pre-compiled code.

Again, this is the old, vendor-dictated school. ROA, on the other hand, is the new, vendor-averse school.

Unlike the vendor-giddy applications, ROA does not waste time in the vain attempts to simplify reality. ROA recognizes and realizes that reality is unthinkably complex. And so ROA lives with it, is based on it, grows from it.

So ROA is not interested in any intricacies of the state transitions that are specific to any of the involved resources. The reason for this is that it would be absolutely impossible to be interested in such things, given their astronomical complexity and volatility.

ROA therefore throws the towel in at the very outset. It is non-competitive by nature.

Not Knowing is the Most Intimate

The author goes on: "The level of omniscient inference with this approach is difficult to even explain."

There is a famous Zen koan in which a Zen Master asks a question, and his student responds: "I don't know, Master".

"Not knowing is the most intimate!" replies the Master.

What is a State Transition?

Seems like traditional software developers are clueless when it comes to grasping the state transition model in ROA. For example, the author of the above post continues:

"So, if the resource is a tennis court and the state transition is reserve (is that a state change or operation?) an application can decide, without caring about the resource per se, that it wants to reserve it. "

There is no such thing as a state transition called 'reserve'. In Service Oriented Architecture (SOA), yes, one can envision such state transition. But never in ROA.

Furthermore, here is that pesky application again. Who is this application, on behalf of which vendor is it entering the picture, and how is it deciding what to do?

The State Changes

Finally, there is (at last!) one thing that both the old school and the new school of software development agrees upon: the state may change! Hallelujah!

"At the very least, the state may change back without the applications knowledge (except it's omniscient so that isn't possible)."

The above sentence actually doesn't make any sense. Who is omniscient? The application? Who made this application?

The whole point is that, when talking about resources, the old school developers are actually not talking about resources (the above quoted article illustrates this rather poignantly). Then they wonder out loud how come they don't get ROA?

All that the old school developers seem capable of thinking is applications. This belies their state of being brainwashed by the tool vendors.

I am fairly positive at this point that unless software developers manage to deprogram their brains and escape the tool vendors' programming (i.e. brainwashing), they will remain completely lost for the world of Resource Oriented Architecture.

Just as well.

Resource Oriented Architecture and OLTP

2006-11-30T10:20:00.000-08:00

As the Resource Oriented Architecture (ROA) is slowly but inevitably starting to gain momentum, there will inevitably be loads of misunderstanding popping up from all sides. Today, we'll look at the grievances posted in the Give it a REST article by someone's "Chief Architect" Udi Dahan.

Here is what Udi claims:

If REST wants to conquer the enterprise, it had better get down from its "internet-scale" high-horse and start talking in terms of OLTP.

This is like saying "if OLTP wants to conquer the enterprise, it had better get down from its RDBMS high-horse and start talking in terms of registries and bits and bytes."

There is always something that is at the lower level than the system under study. Just because there is a lower level, doesn't automatically mean we should stoop to it.

The entire point of the REST and the ROA is precisely the liberation from the need to stoop down to the lower level. Using these principles, one gains the necessary discipline to resist the ingrained challenge to stoop to the lower level.

In case this is not clear to you, keep in mind that REST stands for 'representational'. Representational means representing something that is at the lower level.

And since we have the representation to play with, we don't have to worry about the underlying lower level.

So, in this case, you can envision ROA being a representation of an OLTP system. Yes, we know that behind the ROA veneer there is an ugly OLTP system. But we don't want to really go there.

Just as we know that behind the OLTP system, there is an ugly flat file, assembler code system. And we equally don't want to go there.

So try and train yourself not to stoop to the lower level. Enjoy the privileges and the benefits of the freedom that ROA brings to the table.

Rails 1.2 Drives the Final Nail in the Database Coffin

2006-11-24T10:19:00.000-08:00

This morning's announcement that Rails 1.2 is about to hit the streets signals the beginning of the end of the database as we know it. Providing that no major roadblocks be discovered and that the Release 1.2 candidate firms up and becomes a real product, we're looking at the most important advancement in software development since the invention of Ruby 10 years ago.

You Don't Have to Hold Your Breath Anymore

I've been advising my friends, associates and customers to hold off their transition to Rails until Release 1.2. The reason I was doing that is because I didn't want them to be forced to learn something and then unlearn it and relearn a completely different approach. This is now the perfect time to jump into Rails, hook, line and sinker. There is absolutely no reason to hold back anymore.

Resource Oriented Development Changes Everything

The reason I think that Rails 1.2 is a death knell for the databases lies in the fact that it brings full fledged resource oriented development into the mainstream.

To quote "that other guy" from Seinfeld (from The Dealership episode; KRAMER: Listen to me. When that car rolls into that dealership, and that tank is bone dry, I want you to be there with me when everyone says, "Kramer and that other guy, oh, they went further to the left of the slash than anyone ever dreamed!"):

"Things will be different from now on."

Why do I think things will be different from now on? You won't know until you try the resource oriented development for yourself. Same as with Ruby (and LSD), if you haven't tried it, you won't know what is all that fuss that people are talking about.

A very significant side effect of this kerfafel is that databases are from now on going to undergo progressive trivialization. It'll take a lot of time for me to explain why is this process inevitable -- I'll save you the time by suggesting that you go and install Rails 1.2 and see for yourself.

ROA and Microformats

2006-08-12T10:04:00.000-07:00

The most recent feedback I've been getting on my ruminations regarding the Resource Oriented Architecture have been mostly concerned with the programmability of the web. In its vanilla state, the web is very easy to program. Basically, all the computer program needs to know how to do is identify the resource and then send it one of the three or four rigid, predefined messages that apply no matter what. These messages are:

Add this resource
Give me the representation of this resource
Make the resource state transition
Destroy the resource

Works like a charm every time. The beauty of this model is that it is unbreakable. Adhering to this model, one will never be forced to go through the pain of 'upgrading the web'. One's code will keep working no matter what.

Is Simplicity the Problem?

To talk to most web developers, you'd get an impression that this beautiful simplicity is more of a problem than a solution. Basically, it all boils down to the fact that programmers complain how this protocol (we're talking HTTP here) is too plain. It doesn't give them the 'power' they're used to when working with the Java API, or with .NET API and so on.

For example, in Java we have an open-ended world of unlimited custom made, home grown protocols. Anyone in the world is free to create their own monster mash, and invent their own capabilities and name them however they feel like. This is what programmers call 'power'.

But that's what I call 'weakness'. Why? Simply because it's so goddamn confusing. How can a thing that's so confusing be considered powerful?

The Problem of Discoverability

Some developers do recognize this problem (i.e. the problem with open-ended, unlimited world of home grown capabilities). Yes, it may be wonderful to have this vast world of incredibly sophisticated capabilities, but what's the point if no one knows about them? It would be absolutely unrealistic to expect that there be a central control instance that would maintain the world-wide inventory of all the ever growing capabilities that are being added to the web daily.

So instead of abandoning the wild geese chase, these architects suggest we use methods of piecemeal discovery. Various techniques have been proposed to that end: reflection, introspection, Web Services Description Language (WSDL), god knows what else. None of these really work, because even after you've discovered that there is a capability out there you had no idea existed, there isn't anything you can do to use it. This is because, while you may be able to discover the remote procedure call signature of that capability (i.e. how to call it, what types of parameters is it expecting, and what type of parameters is it returning), you still have absolutely no way of deciphering the meaning of that capability. What does it really mean, what does it really do?

You could always assume, but there is inevitably a big ass in every assumption.

It is very hard trying to interpret the intentions that some content conveys by relying on the formally measurable parameters. That would be akin to trying to figure out whether a person likes something or not by measuring that person's pulse, blood pressure, blood sugar level, brain wave activity, etc. Sure, all these things are measurable, but are they really conducive to attaining unambiguous conclusion?

Work from the Known, not from the Assumed

All the RPC methodologies prefer to work from the assumed standpoint. In other words, the RPC client prefers to engage the server in a preliminary conversation. The conversation goes something like this:

Client: "Hi, I am about to request that you render a service for me. Could you please tell me what you're capable of?"

Server: "Hi there, I offer wide variety of top-notch services for your exquisite enjoyment. What would be your pleasure today?"

Client: "Oh, I was hoping that you could help me convert inches to centimeters. Can you do that?"

Server: "Here is the list of things I can do (offers a long list of convoluted names)."

Client: "OK, let's see... (tries to find the name that would resemble the inches-to-centimeters conversion)"

Once the client makes a decision, the real conversation commences, meaning the real data may be exchanged.

In contrast, resource oriented client does not engage the resource in any sort of preliminary chit-chat. The client simply identifies the resource and asks it to send its representation to the client. The client examines the received representation and decides to either give it a miss, ask the resource to make a state transition, or ask it to destroy itself (or perhaps ask it to add a new resource). Simple as that. The conversation between the client and the resource commences right out of the gate. There's no pussyfootin'.

The Problem of Enriching the Protocol

So discoverability didn't really get us anywhere, nor could it ever do so. People are slowly but surely beginning to reach the conclusion that it is much safer and ultimately much better to ask the resource for its representation, than to interrogate it about its dubious capabilities. At least by sticking to the representation model, we know that our request will always get serviced in the predictable way.

But the problem now seems to be that the representation of the resource is not structured enough. What does that mean? Let's go back to my tennis court example for a minute -- if we identify certain tennis court in our town, and request to get its representation, the response will travel to our client and will be rendered for our consumption. We will then be able to read about it in more details. For instance, we may be able to see that this tennis court is not booked on Saturday morning, which is exactly the information we've been looking for (i.e. we've been searching for a tennis court in our town that would be free this coming Saturday morning).

So right there we see that this resource (i.e. tennis court) is endowed with the capability to be in a booked or free state. And that's all we need to know in order to fulfill our goal (and thus we'll find the web site that's hosting this resource to be very useful to us).

Now, most programmers see this situation as being very problematic. Basically, they are complaining that this representation of the resource is only human-friendly, and that machines have been left out of the equation. The highly unstructured content of the resource's representation may be fine for humans, but is all but useless for the machines.

Because of that, they propose that the rock solid HTTP protocol be enriched, ameliorated, and opened up for allowing us to enforce more structure upon the content of the resource's representation.

How do they propose to do that? Microformats is one way that seems to be getting many people's hope quite high. So let's look at how do Microformats propose to enrich the HTTP protocol.

The 80/20 Myth

Microformats offer a very non-intrusive approach to ameliorating the protocol. That approach is based on the more 'organic' view of things. In other words, it's bazaar rather than a cathedral, a garden rather than a crystal palace.

The so-called Zen of Microformats states that it only makes sense to cope with the 80% of the problem space, and leave the remaining 20% of the unsolved portion to take care of itself.

This, of course, is very reasonable. It is rather unacceptable from the engineering standpoint, but we all know by now that software development is as close to engineering as tap dancing is close to Dave Chapelle's Block Party.

In the nutshell, then, Microformats propose to open up the playing field for structuring the wild and woolly content as it is being served on the web as we speak.

Right now, it is possible to see some of the Microformats in action. Plenty of good ideas that definitely add value to the meaning of the structure of the resource representation.

So where's the problem? It's in the unsubstantiated belief that this additional structuring of the resource representation will catch on in approximately 80% of the cases. My hunch is that this expectation is hugely blown out of proportion.

The Selfish Web

One of the fascinating qualities of the web is that it offers one of the most altruistic experiences that emerge out of the most selfish motives. This is called 'harvesting the collective intelligence'. Each individual on the web pursues his/her proprietary, selfish goals, and yet the community tends to benefit vastly from such selfish pursuits.

But it would behoove us to keep in mind that, on the web, work avoidance is the norm. People mostly blurt things out on the web and then go on their merry ways. No one has the time nor any intention to stop and carefully structure their content.

Presently, the content offered on the web is at best structured to offer the following semantics:

HTML head with a half-meaningful title (hopefully)
body with (hopefully) only one H1 (heading one) markup tag
ordered/unordered lists enumerating some collection
divisions with semi-meaningful class names and ids

If one is extremely lucky, one may find an HTML representation of a resource that offers such well-formedness. But in most cases, the representations we do get are even below such extremely lax standards.

How are we then to expect that Microformats will pick up and reach the 80% of all representations? I think it's a pipe dream. I am doubtful that Microformats will ever reach even 20% of the representations out on the web. I hate to say this, but I'm afraid that we're more realistically looking at 2% to 5% rate of adoption.

Only time will tell, as always.

Is the Web Machine-Friendly?

2006-08-10T10:02:00.000-07:00

People would like the web to be machine-friendly. What does that mean? Basically, people would like be able to teach the machines to go out on the web and do the legwork for them.

Now, machines are notoriously very brittle, and tend to easily break. You throw an unexpected piece of information at the machine, and it freaks out.

Humans are different. Humans can cope with irregularities. That's because we are blessed with common-sense.

Is the Web Broken?

We know for a fact that the web is not machine-friendly. But does that mean that it is broken? Some people tend to think yes, since the web cannot offer regular, uniform and predictable experience to the machines, it is broken.

Some other people, myself included, tend to think differently. I don't believe that the web should be made uniform, just so that the machines could traverse it without experiencing any hiccups.

So, in my view, the web is not broken. The web is just fine the way it is. It is the expectation that the web must be machine-friendly that is broken.

Smart Servant does not Imply Automation

Most people make a primordial mistake upon hearing about the Smart Servant and think that it means some really smart piece of automation. But that's very 19th century thinking. Today, in the 21st century, this is not what we're after anymore.

We're really after humanization of the technology. We want machines to learn to bend over backwards and kiss their own ass and serve our human needs. Nothing more.

And for that, we don't need massive automation. We don't need to turn the web into the wasteland of bland uniformity. Let the web be what it already is -- an enormous mass of messy, irregular, wacky and crazy stuff. That's life. That's what human beings thrive on.

We need to harness the technology that will help us participate and contribute to this mess. We don't need technology that will help us clean up and solve this mess. It is up to us, humans, to decide what's a problem and what isn't a problem.

Should the Web be Easy to Manipulate Programmatically?

My answer is: why? Who needs programmatic ways for accessing the content on the web? This is because I want to be in charge. I am the one who's in the driver's seat. Even when I hire a chauffeur to drive me, and am sitting in the back of the car, it is still me who is in the driver's seat.

Same is with the web. I am the consumer, the participator, the contributor on the web. I don't want machines to do that. I don't see any value or benefit in expecting the machines to do that.

How Can ROA Help Us Build Smart Servants?

2006-08-10T10:00:00.000-07:00

Pat Maddox and Abhijit Nadgouda are two clever dudes. They tend to ask tough questions, which is fine by me -- keeps me on my toes. I see people like them being at the forefront of the next wave of software development. People who don't take things at a face value, who keep probing and exploring and are not satisfied until everything becomes crystal clear.

Those who do not see any value in such behavior will be left behind, partying on their Titanic while it gradually and almost imperceptibly continues to sink. Eventually, it will run out of escape boats and life jackets.

All right, onto addressing Pat's questions:

How does this thinking allow us, as developers, to build smart servants?

Bingo! Right smack in the middle, he asks the absolute perfect question. But just to flip it around a bit, I'd like to ask a slightly modified question:

Why Haven't We Been Building Smart Servants?

In other words, what was it that was stopping us from trying to build smart servants? I mean, after more than 40 years of developing all kinds of software, why is it that only now are we starting to talk about smart servants?

The reason for it is quite simple: if you're drowning, and are gasping for air, you don't have enough free time and energy to start reciting poetry. And we've been drowning for the past 40 years or so in the turbulent waters of oppressive computing infrastructure woes.
If 98% of our time must be dedicated to serving the finicky infrastructure, there is really no free time left to think about finer things in life. Such as -- a smart servant.

Computing Infrastructure is Rapidly Losing its Luster

We have reached a point where there aren't any more compelling reasons to be fascinated by the computing infrastructure. Naturally, in previous times, when a single computer used to command a million dollars price tag, there was lots to be fascinated with. But today this infrastructure is dirt cheap, and therefore ceases to be the subject of heated conversations. Similar thing happened to the light bulb, the radio, the TV, etc.

We are therefore standing at the threshold of the realization that we are not to serve this dirt cheap computing infrastructure. We are slowly coming to our senses and are beginning to insist that the computing infrastructure must serve us.

What that means is that we're turning the tables on the computing infrastructure. From now on, instead of spending 98% of our time servicing this infrastructure, we don't want to spend more than 2% of the time doing it. It is a drastic, radical turning point, where we basically turn things upside down.

So if you're planning to continue being engaged in using computing infrastructure the way tool vendors are shoving it down our throats (like Microsoft, IBM, Sun, Oracle, etc.), you know with frightening degree of certainty that 98% of your efforts will continue to be wasted on servicing that infrastructure. That means that 98% of your decisions will add 0% value to your business (and will keep adding 100% value to the vendors' business).

If, however, you make a healthy transition and stop eating junk food and embrace the world of resource-oriented programming, you will be forsaking your love affair with the computing infrastructure and will begin embracing the smart server model.

The Honeymoon is Over

Often times, even though the honeymoon is long gone, people still don't feel ready to acknowledge that fact. This is what's happening with the infrastructure-centric software development. Lots of people have fallen into a bad habit, created by the pushers (i.e. the tool vendors), and are now convinced that they must keep feeding the beast. But in actuality they really don't have to keep feeding the beast. Walking away from your dealer is easier than you might think.

It is hard to consider a divorce if you're convinced that the honeymoon is still in full swing. But the time for filing a divorce has come.

Building Smart Servants Takes Full Attention

It would not be possible to build a smart servant if we cannot have our undivided attention focused on it. So long as we keep using the tool vendors' vision of the computing infrastructure-centric model, we won't have the ability to focus on building the smart servant. There will always be something else that is more pressing, more important than the smart servant project.

But if we switch to the liberating technologies such as resource-oriented programming and embrace principles of radical simplicity, the computing infrastructure concerns will cease popping into our head while we're developing software. That experience is very liberating (as all people who've made an effort to learn Ruby can attest for).

Browsing the Resources

Pat then continues his inquiry:

Right now with a resource-oriented paradigm, the only way to use the resources is through a browser.

You don't have to use the resources via a browser. If you can identify and locate a resource, you can send it an HTTP request in more than one way. Then, you'll receive an HTTP response from that resource, and it's up to you how would you like to handle that response.

This is because, as I said, you still need to know the attributes of a resource in order to do anything with it programatically.

You can do a lot of things programatically to a resource even if you don't know any of its attributes. You can ask it to represent itself to you. You can ask it to destroy itself. You can ask it to make a transition to a different state.

So we can publish useful resources, because humans are smart enough to process them, but how can we consume the resources?

As I've already said, we can consume them by asking them to represent themselves etc.

Even if it’s through a browser, you’ve already proclaimed that the browser is a broken model.

True, but we're not forced to stay with the browser. AJAX is leading the way out of the browsing paradigm.

Should the Purpose of Conversation be Pre-Established?

2006-08-09T09:58:00.000-07:00

In response to my recent post on Resource Oriented Architecture (ROA), Abhijit Nadgouda made the following comment:

Not sure if this meant that the capability was a surprise to the client. If the client did not know of the capability, why did the client engage with the resource? Shouldn't the purpose of conversation be pre-established? Should the client expect any capability from the resource?

Just to clue you in, I was talking about the absence of a need for a client to know the particulars for the resource the client is interacting with. So long as the resource understands the request to represent itself, to make a state transition, and to eventually destroy itself, the client can accomplish its goal.

Now, when the client identifies and locates the resource, it is most natural for the client to expect to receive the representation of that resource. To use the example from my original post, if I, as a client, am looking for a tennis court where I could play with my friends, upon identifying and locating the potential court, I'd like to see its representation.

At that point, the identified tennis court will ship its representation to me. This representation will then be rendered in my browser. By examining the representation I've just received, I should be able to get a better picture about the resource's capability.

Now, Abhijit's question is: is this capability a surprise to me, the client? Well, hard to say, isn't it? I mean, it all depends on what I, as a client, was expecting when engaging in the conversation with the resource.

If, for instance, I was expecting the resource to offer shower facilities and it didn't, then yes, maybe I'd be surprised. But then again maybe not, because many of the public tennis courts in my city do not offer any amenities.

The Web Is About Exploratory Behavior

Another interesting question is this:

If the client did not know of the capability, why did the client engage with the resource?

It's called exploration. And web is all about exploring. Poking around. Does this tennis court have a wall to bounce the ball off of, or not? I don't know, let's explore and find out.

Shouldn't the Purpose of Conversation be Pre-Established?

Why enforce such a constraint? The web is also about freedom. Let me start by an informal chat, and see where it takes us. There's no need to impose an authoritarian, military type of a rigid conversation that mustn't deviate from the very narrow 'norm'.

Should the Client Expect any Capability from the Resource?

Yes, and we've covered that expectation in very many details already. To reiterate, a web client invariably expects any resource on the web to be capable of representing itself, of making the transition of its state, and of destroying itself.

Learn to Walk Before You Try to Run

2006-08-08T09:57:00.000-07:00

It is wonderful that so many people are nowadays serendipitously discovering the power of Resource Oriented Architecture (ROA). However, one of the most problematic aspects of human nature is impatience. I feel qualified to speak about this foible because I am one of the most impatient people around.

And that's always been the downfall in any venture. So it is with rediscovering the world of resources. When I say 'rediscovering', I'm trying to remind the reader that resources have been built into the web from the day one. But somehow, we chose not to notice them. Instead, we chose to focus on remote procedure calls (i.e. the services).

The Follies of Impatience

Humans like to complicate things. Here we have a very robust and simple situation -- an ever growing collection of resources which could be accessed via three simple commands: represent yourself, modify yourself, destroy yourself. But in our blind impatience we've jumped at the conclusion that things are just too simple and that we need a much more complicated protocol in order to make things work. We rushed to invent the dreaded web services.

This impatient over-engineering is so foolish, that one is at a loss when trying to find an equivalent folly in other engineering fields. The closest one I could come up is with the system of traffic lights.

Today, we have this traffic regulation system that consist of the protocol based on the state of the physical semaphores out in the streets. At any point in time, each semaphore could assume one of the three possible states: it could turn turn green, which means 'go', it could turn yellow, which means 'get ready to stop' or it could turn red, which means 'stop' .

That's all there is to it. Very simple very elegant, and serves its purpose perfectly -- to regulate otherwise astronomically complex traffic patterns.

Now imagine if we've given in to the engineering follies of impatience, and let civic engineers take over the asylum and go nuts with their 'solutions'. We'd be then quite easily looking at 64 different colors that a traffic semaphore could take. Mauve would possibly mean 'go, but be forewarned that there's construction ahead', blue would mean 'speed limit around the corner', purple would mean 'food and lodging ahead', brown would mean 'traffic congestion after the next intersection', and so on. The possibilities are endless.

Would that be an improvement? Quite the contrary, it would be a veritable disaster. Not only would the chances of drivers getting totally confused be blown out of all proportions, but we'd probably witness the 'vendor wars' all over again. Different municipalities would strive to come up with their own version of comprehensive traffic regulating color scheme. Drivers would have to learn and memorize numerous color patterns, and to be additionally very aware when they have crossed the boundaries from one jurisdiction to another jurisdiction.

All in all an incredible mess which thankfully got avoided by patiently sticking to the dull but extremely reliable protocol of only three traffic lights.

Same as reducing the number of decisions when driving leads to the more reliable public traffic, reducing the number of decisions when traveling the web leads to the more reliable web software. On the web, all we need to know is that a resource is identifiable and locatable, and that it can represent itself to us, and can respond to our requests to change its state or to destroy itself. And if we keep these radically simple things in mind when developing web software, we'll realize that we can do absolutely anything on the web.

Not only that, but by observing these five simple rules (identify the resource, locate it, ask it to represent itself, ask it to make a state transition, ask it to destroy itself), we open up the world of endless possibilities for other interested parties to join in on the conversation and to ameliorate and augment our business. All that without expecting anyone to learn anything in particular.

This is the world of true democracy, where everyone is welcome and everyone is qualified and is free to feel adequate to contribute their own value.

Contrast that with the world of web services where each and every business is free to invent its own system of 'traffic lights' and then insist that anyone driving through their territory learns their convoluted system and obeys it. It is painfully obvious how web services had regressed us and damaged our credibility big time. It'll take many years before the damage can get undone. In the meantime, we'd do well to really learn how to walk properly on the web, before we try to run and again fall flat on our faces.

Replacing Service Oriented Architecture with Resource Oriented Architecture

2006-08-08T09:53:00.000-07:00

We've already seen that resource is the central abstraction upon which the web is based. The problem in today's mainstream web development is that most developers have been forced to make a quick switch from the mainframe or the desktop or the client/server environments to the web environment. Now, we've seen that in the mainframe or in the desktop etc. environments, other equally powerful abstractions found their way into the collective mindshare. For example, file is a very powerful abstraction that allows software developers to program all kinds of nifty things on the *nix box.

At the same time, object is an equally powerful abstraction that allows software developers to develop all kinds of nifty things in a distributed system.

Also, remote procedure call (RPC) is a very powerful abstraction allowing developers to define all kinds of contracts across the system boundaries.

All these abstractions, as powerful as they are, have proven inadequate in the world of web development. The biggest problem right now is that most web developers seem to be barking up the wrong tree by relying on another strong abstraction -- service. This reliance led to the very popular Service Oriented Architecture (SOA) on the web, which, being inadequate, is slowing web development down and thus incurring a lot of unwanted delays and damages.

What is a Service?

As the name itself implies, a service is basically a rendered capability. This concept actually comes from the world of objects. Objects are often in the world of software referred to as 'bundles of capabilities'. When we set the environment in such a way that a contract could be made between the provider of the service (i.e. an object) and a consumer (i.e. a client), we arrive at the service-oriented architecture.

A service, being a general purpose abstraction, can be anything. Thus, services are extremely diverse, and that creates a huge problem. Any consumer-to-be must keep an inventory of the available services of interest that are on the web. Not only that, but because each service comes with its own arbitrary protocol, the consumer-wannabe must also make sure that the inventory of corresponding protocols is kept up to date.

This places enormous pressure on the consumers. How are they to keep up with the incredibly rich and diverse world of web-based services? It simply isn't possible. Certain systems have been proposed in the past to allow interested parties to discover the availability of the web services and to then engage in learning the ins and outs of the particular service's protocol. But we haven't seen a truly workable solution to that problem. My hunch is that we never will.

The Web is All About Conversation

The nature of the web, as a medium and as an enabling technological platform, is conversational. The web is abuzz with incessant streams of conversations going on around the world. Some private, some public.

Because of that, the web is built on the premise that there always will be plenty of third parties who will be interested in joining the conversation. A classic example -- a group of friends interested in playing tennis may strike a conversation about who, when and where. At the same time, there could be a facilities provider who may be interested in joining the conversation by offering a nice court with all amenities for a very reasonable fee. It's a win-win situation.

But how is the third party (i.e. the tennis court provider) to join in on the conversation?

The Web is All About Relinquishing Control

In the non-web world, where control is the highest order of the day, such 'joining in on a conversation' could typically not happen in a haphazard way. A specialized protocol would have to be developed and all the interested parties would have to be notified beforehand. But often just being notified isn't sufficient. Sometimes the notified parties who would want to join in on the conversation tend to learn that, in order to join, they first need to enroll in some sort of an educational course. High barriers to entry etc. There would therefore be very little possibility for someone to just jump in the middle of a fairly advanced conversation.

And yet that's exactly what the web is all about. The web as a medium is extremely (some even say radically) open, extremely trusting. It places almost no restrictions on who joins in. Not only that, but it also places almost no restrictions on when does the third party join in. So, usually on the web, there's no need to take a university course before we can participate in the conversation.

This is possible because no one owns the web, and consequently no one can control it. The web is everyone's property, and everyone is welcome to join in. It's the world of sharing. The need for control has been relinquished, and it is left to the participants to sort their discrepancies out. The web as a medium and as a technology is not going to offer any assistance in that regard.

Back to the Resources

Because of the uniqueness of the web as a medium, the only abstraction that does it true justice is resource. The web is a collection of resources. These resources are astronomically diverse, and it would be mathematically impossible to maintain any semblance of a reasonable inventory of web resources.

This reality then forces us to give up on trying to control and maintain the inventory. And instead, we turn our attention to the web protocol.

Each resource on the web, no matter how unique and complicated it may be, obeys only one protocol. This protocol has three major aspects to it. In no particular order, these aspect are:

Each resource knows how to represent itself to the consumer
Each resource knows how to make a transition from one state to another state
Each resource knows how to self-destruct

In addition to the above three aspects of the web protocol, there is a protocol that allows clients to add a new, never before existing resource to the web.

All in all, four very simple aspects of a single protocol. And using these four aspects, it is possible to accomplish absolutely anything on the web.

How Does the Web Protocol Work?

If there is a resource out on the web, such as a tennis court, that resource could be identified. Once identified, that resource could be located on the web (using its corresponding URI). Once located, a request, obeying one of the three aspects outlined above, could be sent to the resource.

Typically, a client would initially want the resource (i.e. a tennis court) to respond to the request by shipping its representation. In reality, no one knows where the actual resource sits, nor how is its state implemented. And that's a good thing -- one less thing to worry about, one less decision to make.

The resource decides how to represent itself (based on some sort of business logic, of which we as clients know nothing, and frankly don't want to know). Its representation travels to the requesting client, who renders it and then consumes it.

Once the representation of the resource's state has been consumed, the requesting client may make a decision to modify that state. For example, if the state of the identified tennis court is that it is free on Saturday morning at 10:00 am, the client may decide to change that state to booked. The client sends another request to the resource, this time using a different aspect of the protocol (basically, using the aspect number 2). The resource handles this request, and, obeying its business logic, makes the transition from its original state (free) to its new state (booked).

Keep in mind that in this scenario the client didn't have to know in advance that the resource is endowed with the capability to be in the free or in the booked state. The client merely engaged the resource in the conversation, and upon receiving the representation of the resource, realized that the resource's state could be changed. That realization prompted the client to try and request the resource state transition.

And if the resource didn't find any objection to that request (for example, if the client's request was deemed as authoritative enough to request the state transition), the resource would obey and would transition to its new state (i.e. booked).

Representational Logic

It is very important to realize at this point that we are dealing with representational logic only. What does that mean? Well, when dealing with representations, we should not take things too literally.

For instance, if an authorized person identifies the tennis court and sends it the request to self-destruct, the resource may obey and destroy itself. But just because we may witness this event take place on the web, doesn't necessarily mean that the resource actually did get obliterated from the system. We have no way of knowing how is the resource implemented, and thus have no way of understanding that resource's true destiny.

It may easily be that, upon receiving a request to destroy itself, the resource merely makes a state transition from "represent yourself to the client's request" to "ignore any requests to represent yourself". But in reality the resource would still be there, with all its remaining state intact.

Next time someone requests a representation of that tennis court, they may get an exception response stating how that tennis court does not exists. However, that may not be entirely true. The resource may actually exist, it's just not to be represented to the clients requesting it.

Procedural to Object-Oriented to Resource-Oriented

It is possible to develop object-oriented software using procedural languages (such as C). But it is very impractical to do so. Still, a fully blown object-oriented system is implemented, under the hood, using lower level procedural constructs.

In a similar fashion, we are about to make a transition from object, or service oriented discipline, to resource oriented discipline. But contrary to how some people interpret this transition, it does not imply supplanting object oriented way of doing things. Same as object oriented implies procedural, as its underpinning, resource oriented approach could easily imply the object oriented underpinning.

For example, when a tennis court (a resource we've mentioned above) receives a request to make a state transition, the logic governing that transition may be implemented using a full fledged object oriented system. But the clients making that request absolutely need not know that. Resource oriented systems expect the clients to only know a simple four-pronged protocol.

Would you like to convert a
pdf file to word?
You can accomplish
pdf to word
conversion with software. The

conversions of
PDF to MS Office programs can be done
much easier than you might believe.

Transitioning From Object-Oriented to Resource-Oriented Programming

2006-08-03T09:51:00.000-07:00

Ten years ago I was hired by a fairly sizable corporation to help them transition from procedural to object-oriented software development. At that time, many businesses have been going through the growing pains of adopting object oriented model, which most of them didn't find very palatable.

Fast forward to present time, when we see how most business have made a successful transition to object oriented model of development. But have they made a successful transition to a more productive way of developing software?

Time for a Change (Again!)

Just as everyone is settling into a comfortable groove, there comes another jolt. Unexpected, to be sure, but nevertheless absolutely necessary. We're talking about the resource oriented development.

What is resource oriented development? It's the type of development that adopts one powerful abstraction (i.e. resource) and then enforces it onto any problem area.

This is not the first time that we've been doing this. In Unix world, for example, we had file oriented development. Unix mindset adopted a very powerful abstraction named file, and decided to enforce that abstraction onto any problem that Unix developers might be trying to solve. It proved to be a very powerful way of doing things, as the ongoing success of Unix platform attests to this very day.

Similar thing was happening in other areas with procedure oriented development (also known as procedural development). A central abstraction (i.e. a procedure) was used as a philosopher's stone to be applied to any and every kind of programming problem.

Then object oriented development became the order of the day. This type of development adopted the notion of object. Everything is an object. Thus, enforcing this abstraction onto any problem domain, object oriented development was able to make major inroads in the world of software development.

But today we're finding that objects, as an abstraction, are not cutting the mustard when it comes to challenges of software development. We will explain now why is that.

Objects are an Extensible Protocol

Great thing about objects is that they behave. Unlike files and resources that just passively sit there, object actively behave. That was, and still is, their biggest selling point.

But that's also their biggest shortcoming. By being endowed with the ability to behave, objects have become unpredictable.

If we examine a typical object, such as for example a Customer, we see that it has a public interface. What that means is that we can send messages to it and it will respond to those messages by behaving. In less technical parlance, we usually say that objects are 'bundles of capabilities'.

The collection of these capabilities is what we call a protocol. Protocol exists in order to enable third parties to join the conversation. If a third party does not understand the protocol, or if it isn't even aware that the protocol exists, its attempts to join the conversation will be futile.

So where's the problem? The problem lies in the fact that the object-defined protocol is entirely arbitrary. This means that an innocent bystander (i.e. a third party) stands very little chance of knowing about that arbitrary protocol. It must learn about it, it must somehow discover it.

The very fact that the onus of discovering/learning the protocol is placed on the client (i.e. on the consumer) means that the solution is very expensive. In contrast, any client using the 'file' abstraction in the Unix solution is free from any such expectations. An innocent bystander in the Unix world is only expected to know the standard, immutable protocol (which is by the way quite simple) in order to accomplish anything.

The limitlessly extensible protocol that the world of objects imposes on software developers is very detrimental to achieving elegant, acceptable solutions. Something must change.

Rediscovering the World of Resources

Resources are limitless. So how's that an improvement over the world of objects?

In the resource oriented development, we tamper the infinite and unpredictable variety of resources with the extremely rigid and simple protocol. True, there are possibly countless resources we might encounter, but the way to engage in the conversation is extremely simple and immutable.

Basically, we have adopted the Hypertext Transfer Protocol (HTTP) when dealing with resources. And this protocol defines extremely stringent set of acceptable behavior. In a nutshell, when dealing with resources, we can identify and locate them via the abstraction called Uniform Resource Identifier (URI). Once identified and located, a resource is susceptible to only a handful of possible behaviors. And in practice, this set of acceptable behaviors boils down to four notorious ones:

Deliver the representation of the resource
Modify the resource
Destroy the resource

In addition to the above, it is also possible to add, or create a resource that never existed before.

This protocol is published worldwide, and shouldn't come as a surprise to anyone using the web to accomplish something.

The rigidity and the immutability of this protocol offers a lot of power and robustness to the software written to accommodate it.

How to Use Web Resources

2006-07-01T09:49:00.000-07:00

Web resources are exposed and are accessible via networks. Anyone with sufficient credentials is free to use them and to manipulate them.

But the way these resources usually get manipulated is very problematic.

What or Who?

Basically, there are two ways to accomplish a destructive action:

You can first decide that you want to destroy something (what to do), and only after making that decision will you choose who will be destroyed
You can first decide who needs to be destroyed, and will only then perform the destructive action (what to do)

For example, if there is a resource on the web to be destroyed, what is most likely to happen is that someone will implement it like this:

http://resource/delete?id=x

What we see here is the decision to first switch to the what mode ("I'm now switching to the slash-and-burn mode"), and only after that the decision is made as to who to delete. The mode (i.e. delete) is declared first, and only after that do we get the identity of the resource to be deleted.

This is disturbingly wrong. It is much more advisable to switch the order of doing things, and to first decide on who is it that needs to be modified/deleted. Once the resource has been fully identified, we can perform the desired action, such as delete or update, etc.

So in the above example, we would simply say:

http://resource/x

By doing that, we will first fully identify the resource ("it's the resource x"). Only after doing that (i.e. after obtaining the handle on the resource), will we send it a message to change itself (such as delete).

This results in a much clearer, much less ambiguous implementation.

How to Design Software for Human Consumption

2006-06-30T09:39:00.001-07:00

Most software developers I've worked with so far tend to specialize in knowing how to design software for the machine consumption. The underlying computing machinery (the CPU, the auxiliary data storage, etc.) is notorious for being very finicky, very demanding. In order to appease the incessant tantrums that the machinery usually throws at us, most software people have learned to focus all their efforts on pleasing the machinery.

That leaves human consumers shortchanged. I would like to advocate here the shift away from trying to please the machinery, and to focus our efforts toward trying to please humans. After all, humans are the most expensive resource in this age of knowledge-based economy.

What do Humans Need from Software?

Basically, what we need from software boils down to two things:

An answer to some question
Processing some information

I won't go here into the gaming/entertainment side of software. Excluding these two aspects of software, everything else seems to be covered by the above two facets.

How to get an Answer from Software?

When you approach a software product with the desire to obtain an answer from it, you are basically doing one of the following two things:

You have good reasons to believe that the answer is 'in there'
You are 'fishing for results'

In the first case, you are convinced not only that the answer is in there, but also that you've come to the right place. For example, if I lose my driver's license, I'll go to my local Motor Vehicle Branch and ask for a new one. Not only do I know that all the information necessary for issuing the new one exists in some government office, I also know that I've come to the right place. It wouldn't be of much help if I went to the Revenue and Taxation office, for example.

In the second case, you may not be sure that the answer even exists, let alone that you've come to the right place. But you're fishing for it anyway.

How to get Software to Process some Information?

A software product may not only contain the ability to answer pointed questions, it could also be capable of transforming the information. If we compare a software product with a human servant for a minute, we could say that, just as a human servant can assist someone in obtaining something (such as a glass of water, or an instruction on how to get to the nearest pharmacy), software can do the same type of service. But in addition to that, just as a human servant can modify the existing resource and turn it into something else (like, use wheat flour, yeast, salt and water and bake a loaf of bread), so can a software product modify the existing information and turn it into something else (like, use the supplied information about the customer and the product to create an order to be shipped).

In order to enable software to perform such transformations, we need to make sure that the following three capabilities are present:

Create new information
Modify existing information
Delete existing information

Working in conjunction with the ability to answer questions, software can produce and manipulate information, thus offering the whole gamut of services to its human users.

Take the Rails RIDE

Rails is the best-of-breed technology that offers this kind of simplistic approach to software development today. It is based on the idea that any desired software behavior could be modeled on the principles of irreducibly simple fourfold behavior:

Retrieve
Insert
Delete
Edit

Any situation could be modeled as either being appropriate for retrieving information, inserting new information, deleting existing information, or editing existing information.

Once we acclimatize ourselves to viewing software modeling that way, we are capable of expressing our intentions to Rails without worrying about how will these intentions be implemented under the hood.

Such arrangement finally opens the door for us to start worrying about the human side of the experience, and to stop fretting over the machine side of the equation.