User-contributed Resources

As we have already discussed on more than one occasion, the world wide web is best described as an ever growing collection of resources. What usually doesn't get discussed is how many of these resources are offered by the original authors, vs. how many resources are user-contributed.

By 'original authors' I mean people who have initially built the web site(s) hosting the resources. For example, if I build a web site dedicated to voting on the best sites available on the web, I may initially 'seed' that site by offering some of my best picks. These best picks would be offered as URLs to the original sites that I am offering as candidates to being voted on.

After the initial launch of my 'best of the best' web site, others may join and may start contributing by adding their picks. All these additional URLs will then be user-contributed resources.

In general, as the so-called participatory web keeps gaining momentum, we will start seeing more and more resources being user-contributed. In the early days of the web, most, if not all published resources, have been contributed by the original authors, or creators of the web sites. Today, we see increasing quantity of resources published on the web that are user-contributed (or, sometimes called member-contributed).

Let's examine now how do users, or members, contribute resources. Typically, there are two ways that resources get published on the web:

1. By filling out a form and successfully submitting it

2. By uploading some digitized information

Whichever the case may be, it is interesting to examine the nature of the content that is being published. For the most part, one would expect that the content is pure text. Sometimes, the content is binary, meaning a sound clip or a jpeg file (representing a photo) or a video clip.

Because the web is basically text-driven, we will now turn our attention to the text-based content. At this point, the question becomes: what is this text-based content supposed to represent?

By and large I'm sure we'll all agree that the text-based content is related to the ongoing conversation on the web. But that's not the whole story. The web as the medium isn't suitable only for enticing and recording various conversations. The web can also enable acting upon the ongoing conversations.

In order to understand in what ways can the web enable this acting upon, let's first refresh our memory on what the resources on the web are:

1. Resources are abstractions, or entities, published on the web
   
2. Resources know how to represent themselves upon request
   
3. Resources know how to make a transition from one state to another, upon request
   

When a request reaches a resource on the web (routed via that resource's Uniform Resource Identifier, or URI), that event triggers a response. To provide a response is the responsibility of the resource. And, from the perspective of the consumer who had initially requested the response, it is anyone's guess what form will that response take.

It is customary to expect the resource to respond to the request by sending back a response containing the representation of that resource. Of course, a single resource can represent itself in more than one way. Often times, the mode of representation will depend on the context of the receiving request. If the request is, for example, coming from a human user, the response will most likely be in the form of a hypertext markup, which can be rendered in the user's browser. If, on the other hand, the request is coming from a machine, the response will quite likely be rendered as an XML document (although other machine-readable formats, such as YML or JSON are possible). These are merely the most generic scenarios, and there are of course different circumstances, where the resource may represent itself in a binary format, such as a PDF document, or a JPEG image, or an MP3 audio file, or even as a movie clip and so on.

Another interesting thing that deserves further examination is where is this response going to be coming from? It is almost always blindly assumed that if the request gets sent to the resource that is published in a specific domain (such as bestofthebestsites.com, for example), the response will arrive from that same domain, or URL.

However, there is nothing in the web architecture that may enforce that convention. Let me illustrate this point by providing a specific example: if someone sends a request asking for the number one best web site ever to the bestofthebestsites.com domain, that request may travel to hypothetical URL such as bestofthebestsites.com/top_site. Upon receiving this request, the top_site resource will respond by preparing its representation which will be shipped to the consumer. But in addition to that, another, more elaborate and more involved representation could be readied and sent over to some other resource, which may live in a completely different domain. From there, the original requestor may be notified by receiving yet another representation. There really are no limitations to chaining events in such fashion.

At this point, this discussion deserves a more elaborate illustration. Suppose we launch a web site that allows users to publish their own resources, such as hand-made merchandise that they put up for sale. The resource will typically be published by asking users to log in, fill out the form representing the resource, and then submitting that form. If the submitted form gets properly validated, the resource will be published on our site.

Let us now examine what could one possible representation of the resource 'merchandise' be. This resource may have a name, or a title, a description,  a date when it was created, a date when it was offered for sale (i.e. published on our web site). In addition, it must have asking price (if it's for sale). It may also be offered for bidding between potential buyers, meaning that the merchandise will have an expiry date, after which it gets sold to the highest bidder.

And of course, that resource may also have a pictorial representation, such as one or more JPEG images, or video clips etc.

After filling out the form specifying many or all of the above properties of the resource, the form will be validated and, if correct, the item will get published on the site. At that point, all the information (i.e. content) will get stored on the hosting site, and the state of that resource will be persisted.

From that moment on, this resource will live in its own address (e.g. http://merchandisehost.com/items/123546). Any request arriving to this address will elicit a response from that resource. And, as we've already discussed above, the shape and form of the response will depend on the circumstances -- meaning, is the request coming from a human user or from a machine, and furthermore, are there any other constraints that could govern the specifics of that response?

So in this simplified hypothetical example, the entire interaction between the resource and its consumer is based on the assumption that the resource only knows how to represent itself by shipping the response from the http://merchandisehost.com domain.

The above constraint, however, is largely simplistic and overtly naive. It is presupposing that the resource on the web is only defined by its static content. But a web resource is not limited to only having static properties, such as textual or visual representation. In addition to having state, web resources also exhibit behavior.

The above statement may seem superfluous in the light of our previous understanding that web resources already know how to behave, by knowing how to represent themselves upon request and by knowing how to make a transition from one state to another, also upon request. But the kind of behavior specific to the web resources that hasn't been discussed yet is the user-contributed behavior. That aspect of the web resources could be the most interesting, and yet the least explored opportunity that the web has to offer.

Time for another illustration: let's say I create a series of prints and offer them for sale on http://merchandizehost.com. I supply the title of the print series, the number of available prints for sale, the item price and the photo illustrating the print. Nothing unusual so far; but suppose now the merchandize host allows me to specify what I would like to happen each time someone buys one of my prints. Even that is not unusual, as many websites today allow people to click on a checkbox saying "Send me an email or SMS when someone buys this". But that's not what I'm talking about here; I'm asking you to imagine a host which will allow us to specify the URL which will receive HTTP POST request on event, such as sales purchase.

Now what could that URL be? Why, it could be anything. It is user-contributed. And once the HTTP POST arrives at that URL, it is anyone's guess what happens next. The user who had contributed that URL is now in the driver's seat. Upon receiving the HTTP POST request, that user's code could post a Twitter update containing the tiny URL representing the item just sold. Furthermore, that same code could further elaborate by sending a request to the author's supplier requesting replenishment for the sold item. And so on, the sky is the limit.

The event when someone had purchased an item on the http://merchandizehost.com site can thus easily mushroom into all kinds of user-defined events that could continue percolating throughout the web. The user who had created the resource may then get notified about the purchase from his/her Twitter feed, or from any other source of alerts they care to set up. Also, the customer who had purchased that item may get confirmation not only from the original web site, but also from some other sources, driven by the author-contributed behavior enhancements.

We see from the above hypothetical scenario that user-contributed resources enhance and embellish not only the content on the web, but also the behavior on the web. Resources on the web put end-users in the driver's seat. Thanks to the radically distributed, fully democratized end-to-end nature of the world wide web, humans have finally achieved the ultimate freedom to fully control what happens to the resources, as they create them.

Resource Representation and Discoverability

We now seem to be entering the phase where we are in the process of reclaiming the web. Who are we reclaiming it from? Why, from no one else but the hordes of software developers.

The web has been hijacked by software developers. It is only natural that the people who dedicate their lives to becoming as intimate with the machines as is humanly possible get to be the people who take center stage in building the web. But, there is trouble in paradise. The web is, in its essence, the exact opposite of the machines. The web is all about humans.

Now, software developers are humans, however they are humans who value serving machines more than they value serving other human beings. Because of that, the web is a horrific place right now, fully dedicated to serving the machines.

This terrifying state of affairs must change. And the only way it will change is if humans reclaim the web, pry it out of the slaves-to-the-machine greedy hands, and install it at its proper place -- as a platform for serving the needs of the human mind and the needs of the human social dimension.

Resources and representation

As has been argued elsewhere, resources on the web are mere abstractions. They are not the tangibles that one could indulge in. At best, one can hope to consume the representation of the abstract resources scattered around the web.

It is often erroneously assumed that resources are directly exposed on the web, in their raw form. One example would be the unanimous conviction that URLs are web's native resources. Thus, a URL such as craigslist.org is viewed as a web resource.

Nothing could be further from the truth. A URL (such as craigslist.org) is not a web resource, it is merely a representation of a web resource. There seems to be an abstraction labeled as craigslist.org and hosted somewhere on the web. But what that abstraction (i.e. craigslist.org) really is, we have no way of knowing.

What we can learn about is one or more representations of that resource. For example, one representation of the craigslist.org resource is its URL. Another one could be the list of all the sites offered by craigslist.org. And so on.

What's the use of representation?

Some people fail to see the usefulness of the representation. They'd much rather get their hands on the resource itself, instead of beating around the bush of resource representation. The best way to explain this problem is to employ assistance of some heavy duty science:

Map is not the territory

This famous premise was issued by Alfred Korzybski, one of the seminal thinkers who helped shape the communication and information theory.

A map is the representation of the territory. Without the map, we'd be lost when traveling through the territory.

In a similar fashion, resource representation could be viewed as a map that eases our voyage through the territory (i.e. the resource we're exploring).

Very few people tend to question the usefulness of maps. It is our hope that, similarly, people will learn to embrace the usefulness of the resource representation.

Discoverability

The web is intended for human consumption. Humans are notorious for having fairly constrained short-term memory buffer, and are thus forced to consume information in a piecemeal fashion. The architecture of the web is therefore tailor-built to serve exactly that constraint -- consume the resource representation at a fairly leisurely pace. Expecting humans to consume the resource representation in a single giant gulp would be utterly unrealistic, and so the fundamental architecture of the web is based on the principle of discoverability.

What that means is that, on the web, we are serving byte-sized chunks of resource representations. These byte-sized chunks are intended to be consumed by human users pretty much at a glance. Any need for consuming more, for learning more, gets fulfilled by letting the users explore further, allowing them to discover more intricate details of the resource representation.

Away from the sociopath web

Today's web is mostly built by humans who crave serving the machines. As such, the way resource representations are typically architected on the web today belies the optimization to the way machines consume information. There is typically very little discoverability offered on most web sites, and the consumers of the representation are usually expected to digest insanely vast quantities of intricate information in a single gulp.

It is painfully obvious to even a very casual observer that the web today is not tailored for easy consumption by humans. It is as if some sociopaths, who lack any degree of empathy with fellow human beings, have built most of the web sites in operation today. Come to think of it, sociopaths is the fair characteristic of individuals who value interactions with the machines more than they value interacting with other human beings.

It is high time we start working on getting out of this sociopath hell.

Resource Representation

Software developers seem largely incapable of understanding the difference between resources and their representation. Understanding that difference is absolutely vital in understanding REST and the Resource Oriented Architecture (ROA).

Is it a small wonder then that we are beginning to see articles like this one ("Why is REST so hard to understand?") pop up in the publications devoted to the web architecture? It is absolutely clear that we won't be in the position to advance ROA practices unless we reach a critical mass of people who do possess a solid grasp of what is the difference between a resource and its representations.

Right now, it seems like only an infinitesimally small percentage of software developers are aware of the significance of that difference. For example, if we say that there are ten million software developers worldwide, 9,999,000 of these developers are blissfully ignorant that such a thing as resources and their representation even exists. Which leaves us with only about 1,000 developers who truly understand what it all means. Not a very encouraging statistics. As a matter of fact, the situation today is right down dismal.

We need to do something quickly in order to remedy this catastrophic situation. My small contribution here is to attempt to clarify the difference between resources and their representation. I will use mostly concepts from everyday life for illustration purposes. I will then attempt to draw the parallel between these concepts and the concepts one encounters in the arena of software development.

What is a Resource?

Let's start from the top: a resource is anything that humans find useful. For example, a house might be a resource. Or, money might be a resource. We find houses very useful, because they offer certain level of comfort in our daily living. We also find money very useful, because it's a resource that enables us to get other resources, such as houses.

Ten years ago, when I was buying my first house, I wanted to see what kind of a house can I afford. To that end I met with a mortgage broker, who interviewed me in the attempt to assess my buying power. Among other things, he needed to know how much money can I allocate towards the down payment on the house.

Once I told him how much money I have for the down payment, he was able to plug that information into his spreadsheet and then tell me what kind of mortgage do I qualify for. So I was then all set to go out and shop for the house, right?

Well, not so fast! Just because I've blurted out a dollar figure that I had in mind for the down payment, didn't automatically mean that I fully qualify for that mortgage. Because, you see, my word in these matters was viewed as being largely subjective, that is, prone to miscalculations and all kinds of other aberrations. What the bank needed is a more objective assessment of my buying power. Talk is cheap, and anyone can claim that they have one hundred thousand dollars tucked away in their bank account, ready to be plopped in toward the down payment, but the reality might be somewhat different.

Because of that, the banks have devised a more impartial practice, whereby the applicant must provide what is deemed to be a more objective proof that the money really is in the applicant's bank account.

In this example, the money that has been allocated for the down payment on the house is regarded as a resource.

What is a Representation?

Now, the down payment money we are talking about may indeed be sitting in my bank account. But how am I to truly convince the mortgage broker that it is really there? Well, I use the means of representation, that is to say, I use something else that acts on behalf of that money. For example, I use my words to convey the fact that I do have $100,000 in my bank account. My words then represent my money.

But, as we've seen, that representation (my words uttered in a casual conversation) is apparently not good enough for the mortgage broker. He needs something more 'solid' before he is fully convinced that the money is actually there. Meaning, he needs a different form of representation.

Typically, what the mortgage broker would deem a fairly impartial and objective representation of my money is a piece of paper issued by my bank that claims that it is true that I do have $100,000 in my account. In other words, once the mortgage broker receives that piece of paper, he can 'take it to the bank', so to speak.

That piece of paper is what we call a representation of a resource. The real money is nowhere to be found in that representation, yet in a somewhat superstitious manner, all the parties involved almost religiously obey and agree that a piece of paper is as good as the real thing.

Back on the Web

The exact same protocol abides on the web. There, as in everyday, non-virtual life, we have resources which are tucked away somewhere, and all we get to see, feel and touch are the representations. Resources on the web are merely concepts that are being represented using other concepts. On the web, as in real life, no one ever gets to see, hear, feel and touch a resource. All we get is mere representations.

We are free to attempt to manipulate those representations. And even in the cases when our manipulation yields perceptible differences, we have no grounds to claim the we have actually manipulated the resource itself. All we can conclude is that the resource representation has changed under the influence of our actions, nothing more, nothing less.

For example, if there is a resource such as a volleyball tournament published on the web, I can get its representation by sending a request for it. I may then decide to destroy that tournament by manipulating the tournament's representation. And my action may result in an affirmative representation (i.e. the resource may respond by sending me its representation which states that it has destroyed itself). I may then conclude that the volleyball tournament has, indeed, been destroyed. But I really have no way of ascertaining that. All I can claim is that I have acted upon the initial representation of that volleyball tournament, I have sent it the request to destroy itself, and I have received the confirmation that it is now destroyed. However, the resource itself may still be alive and well, existing somewhere beyond the reach of the resource representation. The only thing that is manifestly certain is that from that moment on, the resource will refuse to render its representation for the incoming requests.

This is similar to how, in the real world, no one would ever suggest that I take the mortgage broker into my bank's safety vault and point him to the pile of cash that's sitting there, and tell him: "Here, here's my $100,000, please count them and let's get on with it!" No one goes straight to the resource itself in real life, and the same holds true in the digital life as it gets distributed on the web (not to mention that even the physical cash is merely a representation of some other concept, etc.; one can never possibly get to the bottom of it, that is, the real resource is nouminal, unreachable by us mere mortals).

Understanding this distinction is vitally important if we are to move forward successfully in building the functional web of information. It is my experience that none of the developers I've ever worked with understand this situation, and that all of them erroneously believe that they are, at all times, working with the 'real thing'. Nothing could be farther from the truth, so I implore all developers to take a moment and try to digest the difference between resources and their representations.