Contracts and Coupling in Resource Oriented Architecture

We've seen that most software developers in operation today are besotted by the vendor model. One of the symptoms of that malaise is the deeply ingrained idea of a contract. Basically, it would appear that, in the minds of typical software developers, there cannot possibly occur an interaction between software components without a predefined contract.

Of course, the contract they are talking about follows (albeit covertly) the implicit revenue model as ingrained in a typical software vendor's business plan. Software vendors make their living by locking into a contract with their customers.

But in cases where the utilization of the software product is not tied with any revenue model, the notion of a contract tends to lose its appeal. Like, for example, in cases where an open source community is bettering the services by utilizing the publicly vetted web standards and protocols.

There are no Unilateral Contracts

The web is a platform that publishes its capabilities in a unilateral fashion. Similar to any other publicly vetted service, the web is calibrated to serve the least common denominator.

For example, in all developed countries in the world there is a publicly vetted service called traffic. Various vehicles are free to enjoy the commodities that the public roads offer, mostly free of charge. However, the reason this service is made so freely available without strings attached lies in the fact that it comes with an extremely pronounced and unbendable bias. That bias states that all vehicles moving along the public roads must keep to the right side of the road.

If this system wasn't calibrated in such a way, it would be impossible to offer the amenities for conducting the public traffic.

So we see that the utmost flexibility of an infinitely complex system, such as public traffic, is only possible if that system is heavily calibrated in a non-negotiable fashion.

And because that calibration is non-negotiable, it does not qualify as being called a contract. In other words, if you're driving your car down the road, you're not holding the traffic contract in your hands that outlines all the details that bind you and the road owner. The road owner (i.e. the public) dictates the rules, and does so unilaterally. You have no say in whether you accept to keep to the right side of the road while driving. You simply must obey the unilateral rule, or suffer the consequences.

To put it more mildly, you must 'get used to it'.

Same principles apply when you're 'driving' around the web. The web specifies upfront the non-bendable rules and regulations, and you have no say in whether you like them, don't like them, whether you're planning to obey them or disobey them. The web doesn't expect you to make your stance known, and the web absolutely doesn't care.

Because of this unilateral set of rules, there cannot be a contract between the sever software on the web and the web client.

The Web is Uncoupled

Because of the lack of a bilateral contract between a web server and a web client, it is not possible to talk about any form of coupling on the web. Tight or loose coupling, which are terms tied with the world of software vendors' applications, lose any meaning on the web.

There are two reasons for this change:

1. On the web, the server need not know anything about the client
2. On the web, the server is blissfully unaware of the existence of any other server

All that is known on the web is that it is a sprawling collection of resources; these resources are easily accessible via the publicly vetted protocol (i.e. URI), and these resources know how to represent themselves and how to make their own state transition upon request.

Thanks to this particular bias, the web is the most scalable, the most interoperable, and the most robust, resilient and comprehensive computing platform ever.

Resource Oriented Architecture and Quality

In his recent post exposing imaginary limitations of the REST paradigm, Udi Dahan laments the lack of explicit dialog that would discuss the benefits of adopting REST. More specifically, he writes:

"What I haven’t heard is how I can use it (i.e. REST) to build large-scale distributed systems that are controlled by complex logic."

Here, I believe he is talking about his definition of quality. Building large-scale distributed systems that are controlled by complex logic sounds like a very costly proposition. Something one would not attempt just for the heck of it, just to kill a rainy weekend.

Because such a project entails a price tag of hundreds of thousands, if not millions of dollars, it is not to be taken lightly. Also, it is not something that happens very often. I would venture out to say that most of us have never been involved in such a project, nor will we ever find ourselves involved in such a thing. It is not very often that businesses embark upon building such complex distributed systems. Yes, eBay and Amazon etc. are such systems, but you could almost count these beasts on the fingers of one hand.

So the question that mystifies me a bit here is: why? Why worry about building such a large-scale distributed complex system? What's the business case for it? How many of those are you planning to build next year? What's your compelling reason for going after such a risky venture?

But assuming that Udi has his valid reasons, and that he has millions of budget dollars to back him up (which, frankly, I seriously doubt), let's engage him in his quest, for the sake of having fun.

What I'm assuming here is that, to Udi, large-scale complex distributed systems spell quality.

So You Want to Build a Multi-million Dollar System?

Why would a system we're building end up costing millions of dollars? There are several possible reasons:

1. We are using primitive technology, and thus it's taking us too long to build
2. We don't understand what is the system supposed to be doing, so we're making things up as we go
3. The scope of the system is on a runaway curve, and so we end up with explosion of features (i.e. featuritis)

In the case of primitive technology, we can imagine trying to dig the Panama canal by giving each worker a wooden spoon. That technology would be considered very primitive for the intended job, and would therefore result in the astronomical cost of the project.

Similarly, if we attempt to build an equivalent of amazon.com or ebay.com using assembler, we'll end up paying astronomical price for using such primitive technology.

In the case of not understanding the goals that the system under construction is supposed to deliver, 'fishing for results' would invariably turn out to be prohibitively expensive.

Finally, in the case of scope runaway, featuritis and bloat are well known for being insanely expensive.

Let's say Udi is unphased after reading this, and still wants to proceed with building his multi-million dollar large-scale complex distributed system. He is now curious how can REST and ROA be of any benefit to him. Let's explore.

Can REST and ROA Help?

My knee-jerk answer is: not really. But let's not give up so easily.

First, let's make two things clear:

1. If we don't understand what is the system supposed to be doing, then no amount of technology can salvage us from dying a miserable death
2. If we contract the featuritis disease and start bloating, there equally is no technological cure; we'll be doomed to collapse under our own weight

So let's assume here, for the sake of having fun, that Udi is immune from the two forms of malaise mentioned above. Let's say he has a very clear understanding of the goals of the large-scale distributed complex system he is trying to build, and let's also assume that he is immune from the featuritis and bloat (both assumptions being extremely unlikely, by the way, but let's pretend they've been licked).

Seeing a Problem where there isn't Any

OK, so Udi is concerned about the following:

"There are business rules that control which data should be returned and when. All this has been known for quite a while and has been modeled around transaction isolation levels in the OLTP sense, and needs to be handled at the application level when it comes to intelligent caching. I haven’t heard anything about REST that deals with this."

My question is: why on earth should REST be expected to deal with that? REST and ROA, as their names imply, specialize in dealing with Resources and their Representations. They don't specialize in dealing with the infrastructure that is the underpinning for the resources.

For all we know, that infrastructure might very well be implemented as OLTP, or perhaps as something else. Same as the ability of the RDBMS to index the data stored within it might be implemented using various algorithms. But in reality, who cares? The important thing is that it works and meets our expectations.

Next, Udi asks:

"If, as a result of updating one entity, other entities are affected how is that information propagated back to the caller?"

This is simple: the caller consumes resources by dealing with their representations. This is the essence of REST. The ground rule in ROA is that raw resource can never be touched by the client, or by the caller. All one is allowed to work with is the resource's representation.

Udi talks about updating an entity. I'll try and translate this into ROA speak. Instead of updating an entity, in ROA we speak of requesting a state transition for a resource. The way it works is as follows:

1. A resource gets identified (via the URI)
2. That resource then gets located (via the corresponding URL)
3. A request is then sent to that resource to change its state (i.e. to make a state transition)

At the end of that chain of events, other resources (or, 'entities' in Udi's parlance) may or may not get affected.

His question, then, is: how is that fact (i.e. that other resource got affected during the state transition) propagated back to the caller?

The answer is glaringly simple: it all depends on the use case. If the use case requires that the caller be notified of the state transition, then the appropriate representation of the new state will be presented to the caller in the form of a response.

One really feels like stopping at this point, and asking: what's the big mystery? Why is it that all these highly paid software architects are having such hellish time grasping this extremely simple concept of resources, their states, and their state transitions?

I am indeed forced to conclude that they seem to working really hard on trying to see the problem where there isn't any. Is that why they get paid those big bucks?

Here's the Skinny

Udi even gives us a tiny homework:

"Let’s say I have a simple rule: When the sum of all orders for a customer in the past quarter increases above X, the customer is to become a preferred customer. Do I PUT the new order resource to the customer resource, or should I have a generic orders resource that is “partitioned” by customer Id? Once the change to the customer occurs, should I PUT some change information somewhere so that the initiator can find out where it should go look for the change? And how should I be handling transactions around all these resources?"

Again, he is confusing apples with oranges. Let's break it down gently:

A customer is a resource. We don't know how is that resource implemented behind the firewall, and we really, really don't want to know! This point is crucial to understanding ROA. Please keep in mind that ROA is definitely not about the how.

OK, we now need to understand that a resource (i.e. customer) has state. For example, the resource could be in the state of being a preferred customer, or not being a preferred customer. However, we don't know, we have no idea how is that state implemented. Also, we don't want to know anything about those gory details.

On to how does that state change. Recall that REST stands for "Resource State Transition" (actually, this is incorrect, and this section talks about ROA, not REST; please see the correction notice at the end of this post). So 'state transition' is the crucial phrase here. Obviously, a resource is capable of making a transition from one state to another. Such as making the transition from the state of not preferred customer to the state of preferred customer.

How/when does that state occur, you ask? Well, that transition is driven by a particular event that must be described in the use case scenario. Like in Udi's use case above, in case of an event when the resource's state (such as total value of accumulated orders) reaches a predefined threshold, that resource knows how to make the transition from not being the preferred customer to becoming a preferred customer.

Again, we don't know how is this transition taking place. There are millions of conceivable ways that transition could take place, but the important thing is that it does.

All of Udi's questions surrounding this hypothetical scenario boil down to how, but that's largely irrelevant. Somehow the resource knows the proper way to make the transition from one state to the other state. When the client then makes a request for getting that resource, the resource in question will know how to represent its state, including the newly won 'preferred customer' feather-in-the-cap.

Correction: I've just realized that, in my eagerness to explain the encapsulation that Resource Oriented Architecture offers, I've made a mistake and erroneously ascribed state transition to REST. Actually, REST stands for Representational State Transfer, not Resource State Transition. The latter is a characteristic of ROA, not REST.

Sorry for the inadvertent error.

RESTafarian Jihad

I've just been accused of waging a RESTafarian Jihad. I feel that this accusation is unjust, and that it is uncalled for. So I need to explain and defend my case here.

Jihad is a strong word in this context, because it came to mean 'holy war' in the modern parlance. And I'm nowhere near waging a holy war on anything, least of on the non-REST platforms.

To be more specific, I am absolutely not interested in converting the Remote Procedure Call (RPC) people to REST. True, approximately 98% to 99% of all software developers I've ever met cannot imagine building a distributed software system without utilizing some form of RPC, so for these people, Resource Oriented Architecture (ROA) is meaningless.

But as I've already explained in one of my recent posts, I view these people as a write-off. Basically, I think there is nothing we can do about that, and that it's time we cut our losses and move on with our lives.

So you see, I'm leaving the RPC crowd (which means almost all software developers in existence today) alone. I'm not waging any Jihads, I'm not plotting to convert anyone.

Instead, I strongly believe that, as it has always been the case, the simpler system, with lower barriers to entry, will somehow win in the end. I don't really know how, but God's ways are mysterious, so we'll see. But same as CORBA had to disappear because it was just too unwieldy, web services and SOA will also be forced to vanish from the scene.

At that point, ROA will take the lead. Right now, we're preparing for that portentous event. It will come in its own sweet time. We're not rushing anything.

We know that, eventually, people will abandon the ways of the past century, and will embrace the twenty first century ways of employing high technology.

Are Software Developers a Writeoff?

My friend Ted introduced me to the idea that we won't be able to see the real effects of the web until the time when the last person who can remember the days before the web dies out. In his view, if you can remember life before the web, then you're not capable of fully grasping the web.

I tend to agree with him. The web is such a drastic paradigm shift that we're not really capable of fully grasping its significance.

But the young and upcoming generations, who can't remember the days without the web, will be much better equipped to cope with it. Same as I take electricity and indoor plumbing for granted, my children take the web and the perpetual availability of the wireless connectivity for granted.

And that makes them a different kind of persons than I, their father, am. And I find that to be a very fascinating phenomenon.

New Breed of Software Developers

Similar to how my children are a new breed of humans who have a completely different outlook on the web than I do, there is an upcoming new breed of software developers who are diametrically opposite from the existing, entrenched batch of software developers.

The existing developers, as I've been discovering through gruesome series of interviews lately, seem completely oblivious to the incredible powers of the web. All the developers I talk to seem to think that the only way to accomplish something in the software development arena is to use the function calls. They all follow the software vendors' lead, which typically means more complexity, more heavy lifting, less palatable solutions.

Because this tendency is so pervasive and ubiquitous among the existing software professionals, I have reached a point where I'm pretty much ready to throw the towel in. In other words, I'm this close to call all your typical software developers, all those entrenched Java, J2EE, .Net, Oracle etc. developers a big fat writeoff.

I am slowly starting to think that this workforce, as huge as it is, is pretty much useless when it comes to business computing. Some of them may be good for developing certain components of the computing infrastructure. But as soon as we get to the point of utilizing this infrastructure for the benefits of the business, they all turn to be nothing but a writeoff.

And by 'writeoff' I mean more harm than good.

This is why I'm forced to turn my attention to the young and upcoming generation of software developers. We need to ensure that these young people don't get tainted by the tool vendors' agendas and business models. We need to make sure they don't get caught in the remote procedure call hell.

Resource Oriented Architecture and Vendors

Suppose we get to the point where Resource Oriented Architecture (ROA) becomes prominent and gains wide adoption. Given the ubiquity, pervasiveness and popularity of the web, it is not a far fetched proposition. After all, the native web is strictly resource-based, and a Resource Oriented Architecture that utilizes that orientation seems like a perfectly natural fit.

Now, if this re-orientation happens, what will the vendors do?

Vendor Business Model

Software vendors are in the business of making profit from selling software based solutions. Traditionally, the most efficient solution, from the vendors' profitability point of view, tends to come in the form of a tool. This is why we tend to call these vendors 'tool vendors'.

Because of this orientation, vendors tend to favor system architectures that are complex, convoluted, difficult to fathom and master. Of course, anything can be quite easily made more complicated than it already is. In other words, it is quite easy to muddle the issue, to add complexity, to increase brittleness. You don't have to be very smart of very competent in order to make things more complicated.

As a matter if fact, complicated situations are always a surefire sign of stupidity. If, on the other hand, you find systems that work but are nevertheless very simple and easy to navigate, that invariably means that someone quite intelligent has devised such a solution.

But, since an easy to understand, easy to navigate and use system is non-profitable from the vendors' point of view (i.e. such systems do not require 'tools' in order to be utilized), vendors do not see any value in playing along with them. Vendors prefer murky, complicated architectures. Getting you involved with such ungodly systems and architectures is a very lucrative proposition for the vendors, because then they have a very good justification to approach you and to start selling you their tools.

Do We Need Tools?

Take a practitioner of a well established profession, such as a lawyer, or a mathematician. These professionals deal with extremely elaborate, intricate systems and bodies of knowledge. They are expected to navigate very sophisticated waters of highly formalized systems.

But what kind of tools do they need in order to do their jobs? In reality, very few. For a lawyer may need nothing more than a typewriter, and a mathematician actually needs even less tools -- a paper and a pen will do just fine.

How come? It's simple -- their job is largely conceptual by nature.

Now, the question is: how is that different from software development? Why do we think that software development is not conceptual by nature, in ways similar to mathematical development?

The thing is, software development is not different in any significant ways from mathematical, or legal developments. They are all largely conceptual activities.

And realistically speaking, all these conceptual activities don't need many specialized tools. It is a great fallacy, perpetrated on us by the tool vendors with their business agendas, that software development needs an elaborate set of tools.

Choose the Right Foundation

And since we are dealing with a fairly conceptual activity when developing software, it is of crucial importance that we select the right foundation, the right philosophy, and the right paradigm from where we can operate successfully.

Choosing the right starting point will ensure that we don't get entangled in the morass of complexity, brittleness, analysis paralysis and the like. Otherwise, it would be devilishly easy to slip and end up in the blind alley.

If we now review the philosophy and the paradigms that the tool vendors are offering us, we see that they prefer to side with complexity, heavy lifting and hard work. The vendors are proposing we adopt systems that are architected for ever growing complexity. Theirs is the business model of introducing a big hairy problem, and then selling the solutions that would be pain killers.

So what's the alternative? We propose that instead of aiming at acquiring pain killers from the vendors, we focus our efforts on acquiring vitamins.

And in our vision, vitamins would be the publicly vetted standards upon which the web, as a universal computing platform, is built. The web offers us simple, extensible, scalable, interoperable and linkable architecture. As such, it is very beneficial to us in the sense that it does not introduce the pain of complexity which then needs to be handled with pain killers.

It's Not the How, it's the What

By now, we all know the intentions are older than implementation. There were times when foolish software developers used to think the reverse was true, but thankfully today most of them have been browbeaten into aligning with the program.

So everyone seems to agree nowadays that articulating the what (i.e. the intention) is much more important than articulating the how (i.e. the implementation). So where's the problem then?

Web Development Challenges

While every prudent software developer had by now learned the lesson in separating the what from the how, they still seem to be applying it at too low a level. Similar to how the budding software developers of the olden days used to build those monstrous huge bloated monolithic programs that ended up being totally unwieldy and unmanageable, today's web developers are falling into the same trap of building similarly huge, bloated, unwieldy web sites.

Or, let me put the question up to you like this: what would you rather build -- a huge monolithic tightly coupled software program, or an open-ended set of smallish, self-contained, loosely coupled software units, that are easy to implement, easy to test, easy to troubleshoot and debug, easy to deploy?

Of course, if you are a software developer worth even a tiny bit of your own salt, you'd go for the latter. Some of us still recoil in horror when the flashbacks of bad old days of monstrous monolithic software programs come back to haunt us. Ugh, not a happy memory!

But it's really funny how these time honored principles of separation of labor, inversion of control, etc., do not seem to translate at all when being ported to the web development. Here, when developing web sites, budding software developers are more than happy to fall into the same old trap of indulging in muddling the concerns and seeking a single point of control. Which ends up in incredibly messy, monolithic, coupled and unmanageable web sites.

This, again, is the outcome of blindly rushing in to grab the how! How do I do such-and-such in Rails? How do I do such-and-such in Ajax?How do I do such-and-such in Flex? And on and on...

People never seem to stop to think what. What do I need to do in order to make this work? Forget about how to do something, and focus on the what.

How you do something is really not a big deal. 99 percent of the time it's just a short google search away. So why sweat the small stuff?

The what is much tougher. Google will not be able to help you there. You'd need your own cool head, your own common sense to figure that one out.

The surefire sign that you're charging down the wrong path is if you find yourself building large monolithic web sites. You know the ones that rely on the function calls, with one central point of control? The ones where it gets progressively difficult to introduce any new capability?

You know the ones I'm talking about. I bet you that right now you're working on one such site.

Well stop! Same as you wouldn't work on a single humongous big ass class, but would rather chop it up into dozens and dozens of smaller, more manageable classes, you must do the same with that web site. Abandon it. Chop it up. Give up the single point of control, give up the irresistible need to shoehorn everything through the single funnel of omniscient point of control.

Think back to your other software development skills. Remember how tough it was until you've learn how to properly delegate?

Well, now's the time to do the same delegating, only not with objects, but with web sites. Treat each web site as a single independent component, that specializes in offering some useful resources. Then open us the traffic gates. Let those independent, standalone 'components' interact, and see what happens.

See, I gave you the what; the how should be a piece of cake for you now.

Web Model for Business Computing

After reviewing the Vendor Model for Business Computing, it is now time to look into the web model. To refresh your memory for a moment here, the fundamental differences boil down to how each model treats data vs business logic.

As we've seen, vendor model tends to freely expose the data, while jealously hiding the business logic. It operates on the fundamental premises that data is deadwood that can only be animated thanks to their marvelous business logic. Without their magical proprietary processing logic, the data remains useless and thus worthless.

Consequently, vendor model insists that we pay good money to gain access to their beloved processing logic.

Web Model is for Humans

Unlike the vendor model, the web model was built with human beings in mind. Instead of insisting that some cryptic, arcane and privileged proprietary business logic be the animator of the data, web model leaves data animation to the most suitable subjects -- human users.

This is why products based on the web model have nothing to hide when it comes to the processing logic. They don't deem this logic very important. As a matter of fact, such products leave almost all of the decisions as to how one would like the data processed to the human users.

Which is where that decision rightly belongs, after all is being said and done.

Web Model Hides the Data

Quite surprisingly, as much as the web model liberates us from the slavery of being governed by some secretive proprietary processing logic, it is equally adamant that we don't gain access to the data.

Instead, web model insists that we only gain access to the representation of the data. But as to what the actual data looks like, or how does it really feel like, we remain forever mystified.

We'll talk some more in the upcoming series about why is such arrangement a much better thing for all consumers of software.

Web Model vs. Vendor Model

There are two models of business computing: vendor model and web model. Vendor model is presently the entrenched model, the one that everyone is locked in.

Web model is brand new. Ask any software developer about the web model of business computing, and they wouldn't know what that model is nor how does it work.

But that doesn't mean that the web model is irrelevant. Quite the reverse, it is the most significant thing to ever happen in the world of business computing.

How is the Web Model Different?

Perhaps the best way to understand the web model of business computing is to discuss how is it different from the vendor model.

Before we delve in, let's just quickly recapitulate what are the characteristics of business computing. You may recall that any business computing model must be based on three fundamental characteristics:

1. Identity
2. State
3. Behavior

In the business computing parlance, and for the purposes of our discussion, we can say that the state is equivalent to data, and the behavior is equivalent to the program logic.

Getting back to the difference between the vendor model and the web model of business computing, we will see that the two models fundamentally differ in the way they treat data and programming logic.

Vendor model insists on exposing the data and hiding the programming logic.

Web model, you guessed it, is the exact opposite -- it is based on the architecture that hides the data and exposes the programming logic.

Both models have their use, and in our ongoing series on the Resource Oriented Architecture (ROA, which is the underpinning of the web model), we will be discussing the advantages of the model that hides the data.

Stay tuned (and don't feel shy to ask any questions).

Resource Oriented Architecture -- Growing Pains

I've already discussed the problems related to how traditional software development workforce seems incapable of grasping Resource Oriented Architecture (ROA). Today, I will tackle another of the misconceptions that seem to inevitably keep popping up in the software development circles.

Someone who very appropriately claims that adding simplicity is an engineering mantra, wrote about the Promise of SOA, REST, and ROA. Here, we will only focus on his view of ROA:

ROA is an intriguing proposition. Applications are freed from worrying about services at all. A resource, which is effectively an entity, exposes the state transitions it is willing to accept. Applications don't even care what the resource is, they simply decide whether they are interested in the state transitions available. The level of omniscient inference with this approach is difficult to even explain. So, if the resource is a tennis court and the state transition is reserve (is that a state change or operation?) an application can decide, without caring about the resource per se, that it wants to reserve it. Of course if this is a seat at a concert you may be committing to a financial transaction. At the very least, the state may change back without the applications knowledge (except it's omniscient so that isn't possible).

Let's pick this apart, to see if we could identify sources of confusion.

Applications and Services

The post quoted above states that: "Applications are freed from worrying about services at all." First off, it is important to understand that ROA is not about applications. This was hinted at in my post on the golden age of software applications.

ROA is, as its name implies, all about resources. Some people claim that it's basically the same as being all about services, but I strongly disagree.

Resources and Entities

He continues: "A resource, which is effectively an entity..." Maybe I should let this slide, but let me just point out that a resource is effectively a resource. No need to drag entity into the picture.

Acceptable State Transitions

Moving along: "[Resource] exposes the state transitions it is willing to accept."

This is absolutely incorrect. Resource only exposes its representation, not the state transitions it is willing to accept.

Applications and the Attempts to Simplify Reality

"Applications don't even care what the resource is, they simply decide whether they are interested in the state transitions available."

He is again talking about applications. And by 'applications' I suspect he means blobs of pre-compiled code.

Again, this is the old, vendor-dictated school. ROA, on the other hand, is the new, vendor-averse school.

Unlike the vendor-giddy applications, ROA does not waste time in the vain attempts to simplify reality. ROA recognizes and realizes that reality is unthinkably complex. And so ROA lives with it, is based on it, grows from it.

So ROA is not interested in any intricacies of the state transitions that are specific to any of the involved resources. The reason for this is that it would be absolutely impossible to be interested in such things, given their astronomical complexity and volatility.

ROA therefore throws the towel in at the very outset. It is non-competitive by nature.

Not Knowing is the Most Intimate

The author goes on: "The level of omniscient inference with this approach is difficult to even explain."

There is a famous Zen koan in which a Zen Master asks a question, and his student responds: "I don't know, Master".

"Not knowing is the most intimate!" replies the Master.

What is a State Transition?

Seems like traditional software developers are clueless when it comes to grasping the state transition model in ROA. For example, the author of the above post continues:

"So, if the resource is a tennis court and the state transition is reserve (is that a state change or operation?) an application can decide, without caring about the resource per se, that it wants to reserve it. "

There is no such thing as a state transition called 'reserve'. In Service Oriented Architecture (SOA), yes, one can envision such state transition. But never in ROA.

Furthermore, here is that pesky application again. Who is this application, on behalf of which vendor is it entering the picture, and how is it deciding what to do?

The State Changes

Finally, there is (at last!) one thing that both the old school and the new school of software development agrees upon: the state may change! Hallelujah!

"At the very least, the state may change back without the applications knowledge (except it's omniscient so that isn't possible)."

The above sentence actually doesn't make any sense. Who is omniscient? The application? Who made this application?

The whole point is that, when talking about resources, the old school developers are actually not talking about resources (the above quoted article illustrates this rather poignantly). Then they wonder out loud how come they don't get ROA?

All that the old school developers seem capable of thinking is applications. This belies their state of being brainwashed by the tool vendors.

I am fairly positive at this point that unless software developers manage to deprogram their brains and escape the tool vendors' programming (i.e. brainwashing), they will remain completely lost for the world of Resource Oriented Architecture.

Just as well.

Resource Oriented Architecture and OLTP

As the Resource Oriented Architecture (ROA) is slowly but inevitably starting to gain momentum, there will inevitably be loads of misunderstanding popping up from all sides. Today, we'll look at the grievances posted in the Give it a REST article by someone's "Chief Architect" Udi Dahan.

Here is what Udi claims:

If REST wants to conquer the enterprise, it had better get down from its "internet-scale" high-horse and start talking in terms of OLTP.

This is like saying "if OLTP wants to conquer the enterprise, it had better get down from its RDBMS high-horse and start talking in terms of registries and bits and bytes."

There is always something that is at the lower level than the system under study. Just because there is a lower level, doesn't automatically mean we should stoop to it.

The entire point of the REST and the ROA is precisely the liberation from the need to stoop down to the lower level. Using these principles, one gains the necessary discipline to resist the ingrained challenge to stoop to the lower level.

In case this is not clear to you, keep in mind that REST stands for 'representational'. Representational means representing something that is at the lower level.

And since we have the representation to play with, we don't have to worry about the underlying lower level.

So, in this case, you can envision ROA being a representation of an OLTP system. Yes, we know that behind the ROA veneer there is an ugly OLTP system. But we don't want to really go there.

Just as we know that behind the OLTP system, there is an ugly flat file, assembler code system. And we equally don't want to go there.

So try and train yourself not to stoop to the lower level. Enjoy the privileges and the benefits of the freedom that ROA brings to the table.

Rails 1.2 Drives the Final Nail in the Database Coffin

This morning's announcement that Rails 1.2 is about to hit the streets signals the beginning of the end of the database as we know it. Providing that no major roadblocks be discovered and that the Release 1.2 candidate firms up and becomes a real product, we're looking at the most important advancement in software development since the invention of Ruby 10 years ago.

You Don't Have to Hold Your Breath Anymore

I've been advising my friends, associates and customers to hold off their transition to Rails until Release 1.2. The reason I was doing that is because I didn't want them to be forced to learn something and then unlearn it and relearn a completely different approach. This is now the perfect time to jump into Rails, hook, line and sinker. There is absolutely no reason to hold back anymore.

Resource Oriented Development Changes Everything

The reason I think that Rails 1.2 is a death knell for the databases lies in the fact that it brings full fledged resource oriented development into the mainstream.

To quote "that other guy" from Seinfeld (from The Dealership episode; KRAMER: Listen to me. When that car rolls into that dealership, and that tank is bone dry, I want you to be there with me when everyone says, "Kramer and that other guy, oh, they went further to the left of the slash than anyone ever dreamed!"):

"Things will be different from now on."

Why do I think things will be different from now on? You won't know until you try the resource oriented development for yourself. Same as with Ruby (and LSD), if you haven't tried it, you won't know what is all that fuss that people are talking about.

A very significant side effect of this kerfafel is that databases are from now on going to undergo progressive trivialization. It'll take a lot of time for me to explain why is this process inevitable -- I'll save you the time by suggesting that you go and install Rails 1.2 and see for yourself.

ROA and Microformats

The most recent feedback I've been getting on my ruminations regarding the Resource Oriented Architecture have been mostly concerned with the programmability of the web. In its vanilla state, the web is very easy to program. Basically, all the computer program needs to know how to do is identify the resource and then send it one of the three or four rigid, predefined messages that apply no matter what. These messages are:

1. Add this resource
2. Give me the representation of this resource
3. Make the resource state transition
4. Destroy the resource

Works like a charm every time. The beauty of this model is that it is unbreakable. Adhering to this model, one will never be forced to go through the pain of 'upgrading the web'. One's code will keep working no matter what.

Is Simplicity the Problem?

To talk to most web developers, you'd get an impression that this beautiful simplicity is more of a problem than a solution. Basically, it all boils down to the fact that programmers complain how this protocol (we're talking HTTP here) is too plain. It doesn't give them the 'power' they're used to when working with the Java API, or with .NET API and so on.

For example, in Java we have an open-ended world of unlimited custom made, home grown protocols. Anyone in the world is free to create their own monster mash, and invent their own capabilities and name them however they feel like. This is what programmers call 'power'.

But that's what I call 'weakness'. Why? Simply because it's so goddamn confusing. How can a thing that's so confusing be considered powerful?

The Problem of Discoverability

Some developers do recognize this problem (i.e. the problem with open-ended, unlimited world of home grown capabilities). Yes, it may be wonderful to have this vast world of incredibly sophisticated capabilities, but what's the point if no one knows about them? It would be absolutely unrealistic to expect that there be a central control instance that would maintain the world-wide inventory of all the ever growing capabilities that are being added to the web daily.

So instead of abandoning the wild geese chase, these architects suggest we use methods of piecemeal discovery. Various techniques have been proposed to that end: reflection, introspection, Web Services Description Language (WSDL), god knows what else. None of these really work, because even after you've discovered that there is a capability out there you had no idea existed, there isn't anything you can do to use it. This is because, while you may be able to discover the remote procedure call signature of that capability (i.e. how to call it, what types of parameters is it expecting, and what type of parameters is it returning), you still have absolutely no way of deciphering the meaning of that capability. What does it really mean, what does it really do?

You could always assume, but there is inevitably a big ass in every assumption.

It is very hard trying to interpret the intentions that some content conveys by relying on the formally measurable parameters. That would be akin to trying to figure out whether a person likes something or not by measuring that person's pulse, blood pressure, blood sugar level, brain wave activity, etc. Sure, all these things are measurable, but are they really conducive to attaining unambiguous conclusion?

Work from the Known, not from the Assumed

All the RPC methodologies prefer to work from the assumed standpoint. In other words, the RPC client prefers to engage the server in a preliminary conversation. The conversation goes something like this:

Client: "Hi, I am about to request that you render a service for me. Could you please tell me what you're capable of?"

Server: "Hi there, I offer wide variety of top-notch services for your exquisite enjoyment. What would be your pleasure today?"

Client: "Oh, I was hoping that you could help me convert inches to centimeters. Can you do that?"

Server: "Here is the list of things I can do (offers a long list of convoluted names)."

Client: "OK, let's see... (tries to find the name that would resemble the inches-to-centimeters conversion)"

Once the client makes a decision, the real conversation commences, meaning the real data may be exchanged.

In contrast, resource oriented client does not engage the resource in any sort of preliminary chit-chat. The client simply identifies the resource and asks it to send its representation to the client. The client examines the received representation and decides to either give it a miss, ask the resource to make a state transition, or ask it to destroy itself (or perhaps ask it to add a new resource). Simple as that. The conversation between the client and the resource commences right out of the gate. There's no pussyfootin'.

The Problem of Enriching the Protocol

So discoverability didn't really get us anywhere, nor could it ever do so. People are slowly but surely beginning to reach the conclusion that it is much safer and ultimately much better to ask the resource for its representation, than to interrogate it about its dubious capabilities. At least by sticking to the representation model, we know that our request will always get serviced in the predictable way.

But the problem now seems to be that the representation of the resource is not structured enough. What does that mean? Let's go back to my tennis court example for a minute -- if we identify certain tennis court in our town, and request to get its representation, the response will travel to our client and will be rendered for our consumption. We will then be able to read about it in more details. For instance, we may be able to see that this tennis court is not booked on Saturday morning, which is exactly the information we've been looking for (i.e. we've been searching for a tennis court in our town that would be free this coming Saturday morning).

So right there we see that this resource (i.e. tennis court) is endowed with the capability to be in a booked or free state. And that's all we need to know in order to fulfill our goal (and thus we'll find the web site that's hosting this resource to be very useful to us).

Now, most programmers see this situation as being very problematic. Basically, they are complaining that this representation of the resource is only human-friendly, and that machines have been left out of the equation. The highly unstructured content of the resource's representation may be fine for humans, but is all but useless for the machines.

Because of that, they propose that the rock solid HTTP protocol be enriched, ameliorated, and opened up for allowing us to enforce more structure upon the content of the resource's representation.

How do they propose to do that? Microformats is one way that seems to be getting many people's hope quite high. So let's look at how do Microformats propose to enrich the HTTP protocol.

The 80/20 Myth

Microformats offer a very non-intrusive approach to ameliorating the protocol. That approach is based on the more 'organic' view of things. In other words, it's bazaar rather than a cathedral, a garden rather than a crystal palace.

The so-called Zen of Microformats states that it only makes sense to cope with the 80% of the problem space, and leave the remaining 20% of the unsolved portion to take care of itself.

This, of course, is very reasonable. It is rather unacceptable from the engineering standpoint, but we all know by now that software development is as close to engineering as tap dancing is close to Dave Chapelle's Block Party.

In the nutshell, then, Microformats propose to open up the playing field for structuring the wild and woolly content as it is being served on the web as we speak.

Right now, it is possible to see some of the Microformats in action. Plenty of good ideas that definitely add value to the meaning of the structure of the resource representation.

So where's the problem? It's in the unsubstantiated belief that this additional structuring of the resource representation will catch on in approximately 80% of the cases. My hunch is that this expectation is hugely blown out of proportion.

The Selfish Web

One of the fascinating qualities of the web is that it offers one of the most altruistic experiences that emerge out of the most selfish motives. This is called 'harvesting the collective intelligence'. Each individual on the web pursues his/her proprietary, selfish goals, and yet the community tends to benefit vastly from such selfish pursuits.

But it would behoove us to keep in mind that, on the web, work avoidance is the norm. People mostly blurt things out on the web and then go on their merry ways. No one has the time nor any intention to stop and carefully structure their content.

Presently, the content offered on the web is at best structured to offer the following semantics:

• HTML head with a half-meaningful title (hopefully)
• body with (hopefully) only one H1 (heading one) markup tag
• ordered/unordered lists enumerating some collection
• divisions with semi-meaningful class names and ids

If one is extremely lucky, one may find an HTML representation of a resource that offers such well-formedness. But in most cases, the representations we do get are even below such extremely lax standards.

How are we then to expect that Microformats will pick up and reach the 80% of all representations? I think it's a pipe dream. I am doubtful that Microformats will ever reach even 20% of the representations out on the web. I hate to say this, but I'm afraid that we're more realistically looking at 2% to 5% rate of adoption.

Only time will tell, as always.

Is the Web Machine-Friendly?

People would like the web to be machine-friendly. What does that mean? Basically, people would like be able to teach the machines to go out on the web and do the legwork for them.

Now, machines are notoriously very brittle, and tend to easily break. You throw an unexpected piece of information at the machine, and it freaks out.

Humans are different. Humans can cope with irregularities. That's because we are blessed with common-sense.

Is the Web Broken?

We know for a fact that the web is not machine-friendly. But does that mean that it is broken? Some people tend to think yes, since the web cannot offer regular, uniform and predictable experience to the machines, it is broken.

Some other people, myself included, tend to think differently. I don't believe that the web should be made uniform, just so that the machines could traverse it without experiencing any hiccups.

So, in my view, the web is not broken. The web is just fine the way it is. It is the expectation that the web must be machine-friendly that is broken.

Smart Servant does not Imply Automation

Most people make a primordial mistake upon hearing about the Smart Servant and think that it means some really smart piece of automation. But that's very 19th century thinking. Today, in the 21st century, this is not what we're after anymore.

We're really after humanization of the technology. We want machines to learn to bend over backwards and kiss their own ass and serve our human needs. Nothing more.

And for that, we don't need massive automation. We don't need to turn the web into the wasteland of bland uniformity. Let the web be what it already is -- an enormous mass of messy, irregular, wacky and crazy stuff. That's life. That's what human beings thrive on.

We need to harness the technology that will help us participate and contribute to this mess. We don't need technology that will help us clean up and solve this mess. It is up to us, humans, to decide what's a problem and what isn't a problem.

Should the Web be Easy to Manipulate Programmatically?

My answer is: why? Who needs programmatic ways for accessing the content on the web? This is because I want to be in charge. I am the one who's in the driver's seat. Even when I hire a chauffeur to drive me, and am sitting in the back of the car, it is still me who is in the driver's seat.

Same is with the web. I am the consumer, the participator, the contributor on the web. I don't want machines to do that. I don't see any value or benefit in expecting the machines to do that.

How Can ROA Help Us Build Smart Servants?

Pat Maddox and Abhijit Nadgouda are two clever dudes. They tend to ask tough questions, which is fine by me -- keeps me on my toes. I see people like them being at the forefront of the next wave of software development. People who don't take things at a face value, who keep probing and exploring and are not satisfied until everything becomes crystal clear.

Those who do not see any value in such behavior will be left behind, partying on their Titanic while it gradually and almost imperceptibly continues to sink. Eventually, it will run out of escape boats and life jackets.

All right, onto addressing Pat's questions:

How does this thinking allow us, as developers, to build smart servants?

Bingo! Right smack in the middle, he asks the absolute perfect question. But just to flip it around a bit, I'd like to ask a slightly modified question:

Why Haven't We Been Building Smart Servants?

In other words, what was it that was stopping us from trying to build smart servants? I mean, after more than 40 years of developing all kinds of software, why is it that only now are we starting to talk about smart servants?

The reason for it is quite simple: if you're drowning, and are gasping for air, you don't have enough free time and energy to start reciting poetry. And we've been drowning for the past 40 years or so in the turbulent waters of oppressive computing infrastructure woes.
If 98% of our time must be dedicated to serving the finicky infrastructure, there is really no free time left to think about finer things in life. Such as -- a smart servant.

Computing Infrastructure is Rapidly Losing its Luster

We have reached a point where there aren't any more compelling reasons to be fascinated by the computing infrastructure. Naturally, in previous times, when a single computer used to command a million dollars price tag, there was lots to be fascinated with. But today this infrastructure is dirt cheap, and therefore ceases to be the subject of heated conversations. Similar thing happened to the light bulb, the radio, the TV, etc.

We are therefore standing at the threshold of the realization that we are not to serve this dirt cheap computing infrastructure. We are slowly coming to our senses and are beginning to insist that the computing infrastructure must serve us.

What that means is that we're turning the tables on the computing infrastructure. From now on, instead of spending 98% of our time servicing this infrastructure, we don't want to spend more than 2% of the time doing it. It is a drastic, radical turning point, where we basically turn things upside down.

So if you're planning to continue being engaged in using computing infrastructure the way tool vendors are shoving it down our throats (like Microsoft, IBM, Sun, Oracle, etc.), you know with frightening degree of certainty that 98% of your efforts will continue to be wasted on servicing that infrastructure. That means that 98% of your decisions will add 0% value to your business (and will keep adding 100% value to the vendors' business).

If, however, you make a healthy transition and stop eating junk food and embrace the world of resource-oriented programming, you will be forsaking your love affair with the computing infrastructure and will begin embracing the smart server model.

The Honeymoon is Over

Often times, even though the honeymoon is long gone, people still don't feel ready to acknowledge that fact. This is what's happening with the infrastructure-centric software development. Lots of people have fallen into a bad habit, created by the pushers (i.e. the tool vendors), and are now convinced that they must keep feeding the beast. But in actuality they really don't have to keep feeding the beast. Walking away from your dealer is easier than you might think.

It is hard to consider a divorce if you're convinced that the honeymoon is still in full swing. But the time for filing a divorce has come.

Building Smart Servants Takes Full Attention

It would not be possible to build a smart servant if we cannot have our undivided attention focused on it. So long as we keep using the tool vendors' vision of the computing infrastructure-centric model, we won't have the ability to focus on building the smart servant. There will always be something else that is more pressing, more important than the smart servant project.

But if we switch to the liberating technologies such as resource-oriented programming and embrace principles of radical simplicity, the computing infrastructure concerns will cease popping into our head while we're developing software. That experience is very liberating (as all people who've made an effort to learn Ruby can attest for).

Browsing the Resources

Pat then continues his inquiry:

Right now with a resource-oriented paradigm, the only way to use the resources is through a browser.

You don't have to use the resources via a browser. If you can identify and locate a resource, you can send it an HTTP request in more than one way. Then, you'll receive an HTTP response from that resource, and it's up to you how would you like to handle that response.

This is because, as I said, you still need to know the attributes of a resource in order to do anything with it programatically.

You can do a lot of things programatically to a resource even if you don't know any of its attributes. You can ask it to represent itself to you. You can ask it to destroy itself. You can ask it to make a transition to a different state.

So we can publish useful resources, because humans are smart enough to process them, but how can we consume the resources?

As I've already said, we can consume them by asking them to represent themselves etc.

Even if it’s through a browser, you’ve already proclaimed that the browser is a broken model.

True, but we're not forced to stay with the browser. AJAX is leading the way out of the browsing paradigm.

Should the Purpose of Conversation be Pre-Established?

In response to my recent post on Resource Oriented Architecture (ROA), Abhijit Nadgouda made the following comment:

Not sure if this meant that the capability was a surprise to the client. If the client did not know of the capability, why did the client engage with the resource? Shouldn't the purpose of conversation be pre-established? Should the client expect any capability from the resource?

Just to clue you in, I was talking about the absence of a need for a client to know the particulars for the resource the client is interacting with. So long as the resource understands the request to represent itself, to make a state transition, and to eventually destroy itself, the client can accomplish its goal.

Now, when the client identifies and locates the resource, it is most natural for the client to expect to receive the representation of that resource. To use the example from my original post, if I, as a client, am looking for a tennis court where I could play with my friends, upon identifying and locating the potential court, I'd like to see its representation.

At that point, the identified tennis court will ship its representation to me. This representation will then be rendered in my browser. By examining the representation I've just received, I should be able to get a better picture about the resource's capability.

Now, Abhijit's question is: is this capability a surprise to me, the client? Well, hard to say, isn't it? I mean, it all depends on what I, as a client, was expecting when engaging in the conversation with the resource.

If, for instance, I was expecting the resource to offer shower facilities and it didn't, then yes, maybe I'd be surprised. But then again maybe not, because many of the public tennis courts in my city do not offer any amenities.

The Web Is About Exploratory Behavior

Another interesting question is this:

If the client did not know of the capability, why did the client engage with the resource?

It's called exploration. And web is all about exploring. Poking around. Does this tennis court have a wall to bounce the ball off of, or not? I don't know, let's explore and find out.

Shouldn't the Purpose of Conversation be Pre-Established?

Why enforce such a constraint? The web is also about freedom. Let me start by an informal chat, and see where it takes us. There's no need to impose an authoritarian, military type of a rigid conversation that mustn't deviate from the very narrow 'norm'.

Should the Client Expect any Capability from the Resource?

Yes, and we've covered that expectation in very many details already. To reiterate, a web client invariably expects any resource on the web to be capable of representing itself, of making the transition of its state, and of destroying itself.

Learn to Walk Before You Try to Run

It is wonderful that so many people are nowadays serendipitously discovering the power of Resource Oriented Architecture (ROA). However, one of the most problematic aspects of human nature is impatience. I feel qualified to speak about this foible because I am one of the most impatient people around.

And that's always been the downfall in any venture. So it is with rediscovering the world of resources. When I say 'rediscovering', I'm trying to remind the reader that resources have been built into the web from the day one. But somehow, we chose not to notice them. Instead, we chose to focus on remote procedure calls (i.e. the services).

The Follies of Impatience

Humans like to complicate things. Here we have a very robust and simple situation -- an ever growing collection of resources which could be accessed via three simple commands: represent yourself, modify yourself, destroy yourself. But in our blind impatience we've jumped at the conclusion that things are just too simple and that we need a much more complicated protocol in order to make things work. We rushed to invent the dreaded web services.

This impatient over-engineering is so foolish, that one is at a loss when trying to find an equivalent folly in other engineering fields. The closest one I could come up is with the system of traffic lights.

Today, we have this traffic regulation system that consist of the protocol based on the state of the physical semaphores out in the streets. At any point in time, each semaphore could assume one of the three possible states: it could turn turn green, which means 'go', it could turn yellow, which means 'get ready to stop' or it could turn red, which means 'stop' .

That's all there is to it. Very simple very elegant, and serves its purpose perfectly -- to regulate otherwise astronomically complex traffic patterns.

Now imagine if we've given in to the engineering follies of impatience, and let civic engineers take over the asylum and go nuts with their 'solutions'. We'd be then quite easily looking at 64 different colors that a traffic semaphore could take. Mauve would possibly mean 'go, but be forewarned that there's construction ahead', blue would mean 'speed limit around the corner', purple would mean 'food and lodging ahead', brown would mean 'traffic congestion after the next intersection', and so on. The possibilities are endless.

Would that be an improvement? Quite the contrary, it would be a veritable disaster. Not only would the chances of drivers getting totally confused be blown out of all proportions, but we'd probably witness the 'vendor wars' all over again. Different municipalities would strive to come up with their own version of comprehensive traffic regulating color scheme. Drivers would have to learn and memorize numerous color patterns, and to be additionally very aware when they have crossed the boundaries from one jurisdiction to another jurisdiction.

All in all an incredible mess which thankfully got avoided by patiently sticking to the dull but extremely reliable protocol of only three traffic lights.

Same as reducing the number of decisions when driving leads to the more reliable public traffic, reducing the number of decisions when traveling the web leads to the more reliable web software. On the web, all we need to know is that a resource is identifiable and locatable, and that it can represent itself to us, and can respond to our requests to change its state or to destroy itself. And if we keep these radically simple things in mind when developing web software, we'll realize that we can do absolutely anything on the web.

Not only that, but by observing these five simple rules (identify the resource, locate it, ask it to represent itself, ask it to make a state transition, ask it to destroy itself), we open up the world of endless possibilities for other interested parties to join in on the conversation and to ameliorate and augment our business. All that without expecting anyone to learn anything in particular.

This is the world of true democracy, where everyone is welcome and everyone is qualified and is free to feel adequate to contribute their own value.

Contrast that with the world of web services where each and every business is free to invent its own system of 'traffic lights' and then insist that anyone driving through their territory learns their convoluted system and obeys it. It is painfully obvious how web services had regressed us and damaged our credibility big time. It'll take many years before the damage can get undone. In the meantime, we'd do well to really learn how to walk properly on the web, before we try to run and again fall flat on our faces.

Replacing Service Oriented Architecture with Resource Oriented Architecture

We've already seen that resource is the central abstraction upon which the web is based. The problem in today's mainstream web development is that most developers have been forced to make a quick switch from the mainframe or the desktop or the client/server environments to the web environment. Now, we've seen that in the mainframe or in the desktop etc. environments, other equally powerful abstractions found their way into the collective mindshare. For example, file is a very powerful abstraction that allows software developers to program all kinds of nifty things on the *nix box.

At the same time, object is an equally powerful abstraction that allows software developers to develop all kinds of nifty things in a distributed system.

Also, remote procedure call (RPC) is a very powerful abstraction allowing developers to define all kinds of contracts across the system boundaries.

All these abstractions, as powerful as they are, have proven inadequate in the world of web development. The biggest problem right now is that most web developers seem to be barking up the wrong tree by relying on another strong abstraction -- service. This reliance led to the very popular Service Oriented Architecture (SOA) on the web, which, being inadequate, is slowing web development down and thus incurring a lot of unwanted delays and damages.

What is a Service?

As the name itself implies, a service is basically a rendered capability. This concept actually comes from the world of objects. Objects are often in the world of software referred to as 'bundles of capabilities'. When we set the environment in such a way that a contract could be made between the provider of the service (i.e. an object) and a consumer (i.e. a client), we arrive at the service-oriented architecture.

A service, being a general purpose abstraction, can be anything. Thus, services are extremely diverse, and that creates a huge problem. Any consumer-to-be must keep an inventory of the available services of interest that are on the web. Not only that, but because each service comes with its own arbitrary protocol, the consumer-wannabe must also make sure that the inventory of corresponding protocols is kept up to date.

This places enormous pressure on the consumers. How are they to keep up with the incredibly rich and diverse world of web-based services? It simply isn't possible. Certain systems have been proposed in the past to allow interested parties to discover the availability of the web services and to then engage in learning the ins and outs of the particular service's protocol. But we haven't seen a truly workable solution to that problem. My hunch is that we never will.

The Web is All About Conversation

The nature of the web, as a medium and as an enabling technological platform, is conversational. The web is abuzz with incessant streams of conversations going on around the world. Some private, some public.

Because of that, the web is built on the premise that there always will be plenty of third parties who will be interested in joining the conversation. A classic example -- a group of friends interested in playing tennis may strike a conversation about who, when and where. At the same time, there could be a facilities provider who may be interested in joining the conversation by offering a nice court with all amenities for a very reasonable fee. It's a win-win situation.

But how is the third party (i.e. the tennis court provider) to join in on the conversation?

The Web is All About Relinquishing Control

In the non-web world, where control is the highest order of the day, such 'joining in on a conversation' could typically not happen in a haphazard way. A specialized protocol would have to be developed and all the interested parties would have to be notified beforehand. But often just being notified isn't sufficient. Sometimes the notified parties who would want to join in on the conversation tend to learn that, in order to join, they first need to enroll in some sort of an educational course. High barriers to entry etc. There would therefore be very little possibility for someone to just jump in the middle of a fairly advanced conversation.

And yet that's exactly what the web is all about. The web as a medium is extremely (some even say radically) open, extremely trusting. It places almost no restrictions on who joins in. Not only that, but it also places almost no restrictions on when does the third party join in. So, usually on the web, there's no need to take a university course before we can participate in the conversation.

This is possible because no one owns the web, and consequently no one can control it. The web is everyone's property, and everyone is welcome to join in. It's the world of sharing. The need for control has been relinquished, and it is left to the participants to sort their discrepancies out. The web as a medium and as a technology is not going to offer any assistance in that regard.

Back to the Resources

Because of the uniqueness of the web as a medium, the only abstraction that does it true justice is resource. The web is a collection of resources. These resources are astronomically diverse, and it would be mathematically impossible to maintain any semblance of a reasonable inventory of web resources.

This reality then forces us to give up on trying to control and maintain the inventory. And instead, we turn our attention to the web protocol.

Each resource on the web, no matter how unique and complicated it may be, obeys only one protocol. This protocol has three major aspects to it. In no particular order, these aspect are:

1. Each resource knows how to represent itself to the consumer
2. Each resource knows how to make a transition from one state to another state
3. Each resource knows how to self-destruct

In addition to the above three aspects of the web protocol, there is a protocol that allows clients to add a new, never before existing resource to the web.

All in all, four very simple aspects of a single protocol. And using these four aspects, it is possible to accomplish absolutely anything on the web.

How Does the Web Protocol Work?

If there is a resource out on the web, such as a tennis court, that resource could be identified. Once identified, that resource could be located on the web (using its corresponding URI). Once located, a request, obeying one of the three aspects outlined above, could be sent to the resource.

Typically, a client would initially want the resource (i.e. a tennis court) to respond to the request by shipping its representation. In reality, no one knows where the actual resource sits, nor how is its state implemented. And that's a good thing -- one less thing to worry about, one less decision to make.

The resource decides how to represent itself (based on some sort of business logic, of which we as clients know nothing, and frankly don't want to know). Its representation travels to the requesting client, who renders it and then consumes it.

Once the representation of the resource's state has been consumed, the requesting client may make a decision to modify that state. For example, if the state of the identified tennis court is that it is free on Saturday morning at 10:00 am, the client may decide to change that state to booked. The client sends another request to the resource, this time using a different aspect of the protocol (basically, using the aspect number 2). The resource handles this request, and, obeying its business logic, makes the transition from its original state (free) to its new state (booked).

Keep in mind that in this scenario the client didn't have to know in advance that the resource is endowed with the capability to be in the free or in the booked state. The client merely engaged the resource in the conversation, and upon receiving the representation of the resource, realized that the resource's state could be changed. That realization prompted the client to try and request the resource state transition.

And if the resource didn't find any objection to that request (for example, if the client's request was deemed as authoritative enough to request the state transition), the resource would obey and would transition to its new state (i.e. booked).

Representational Logic

It is very important to realize at this point that we are dealing with representational logic only. What does that mean? Well, when dealing with representations, we should not take things too literally.

For instance, if an authorized person identifies the tennis court and sends it the request to self-destruct, the resource may obey and destroy itself. But just because we may witness this event take place on the web, doesn't necessarily mean that the resource actually did get obliterated from the system. We have no way of knowing how is the resource implemented, and thus have no way of understanding that resource's true destiny.

It may easily be that, upon receiving a request to destroy itself, the resource merely makes a state transition from "represent yourself to the client's request" to "ignore any requests to represent yourself". But in reality the resource would still be there, with all its remaining state intact.

Next time someone requests a representation of that tennis court, they may get an exception response stating how that tennis court does not exists. However, that may not be entirely true. The resource may actually exist, it's just not to be represented to the clients requesting it.

Procedural to Object-Oriented to Resource-Oriented

It is possible to develop object-oriented software using procedural languages (such as C). But it is very impractical to do so. Still, a fully blown object-oriented system is implemented, under the hood, using lower level procedural constructs.

In a similar fashion, we are about to make a transition from object, or service oriented discipline, to resource oriented discipline. But contrary to how some people interpret this transition, it does not imply supplanting object oriented way of doing things. Same as object oriented implies procedural, as its underpinning, resource oriented approach could easily imply the object oriented underpinning.

For example, when a tennis court (a resource we've mentioned above) receives a request to make a state transition, the logic governing that transition may be implemented using a full fledged object oriented system. But the clients making that request absolutely need not know that. Resource oriented systems expect the clients to only know a simple four-pronged protocol.

Would you like to convert a pdf file to word?  You can accomplish pdf to word conversion with software. The conversions of PDF to MS Office programs can be done much easier than you might believe.

Transitioning From Object-Oriented to Resource-Oriented Programming

Ten years ago I was hired by a fairly sizable corporation to help them transition from procedural to object-oriented software development. At that time, many businesses have been going through the growing pains of adopting object oriented model, which most of them didn't find very palatable.

Fast forward to present time, when we see how most business have made a successful transition to object oriented model of development. But have they made a successful transition to a more productive way of developing software?

Time for a Change (Again!)

Just as everyone is settling into a comfortable groove, there comes another jolt. Unexpected, to be sure, but nevertheless absolutely necessary. We're talking about the resource oriented development.

What is resource oriented development? It's the type of development that adopts one powerful abstraction (i.e. resource) and then enforces it onto any problem area.

This is not the first time that we've been doing this. In Unix world, for example, we had file oriented development. Unix mindset adopted a very powerful abstraction named file, and decided to enforce that abstraction onto any problem that Unix developers might be trying to solve. It proved to be a very powerful way of doing things, as the ongoing success of Unix platform attests to this very day.

Similar thing was happening in other areas with procedure oriented development (also known as procedural development). A central abstraction (i.e. a procedure) was used as a philosopher's stone to be applied to any and every kind of programming problem.

Then object oriented development became the order of the day. This type of development adopted the notion of object. Everything is an object. Thus, enforcing this abstraction onto any problem domain, object oriented development was able to make major inroads in the world of software development.

But today we're finding that objects, as an abstraction, are not cutting the mustard when it comes to challenges of software development. We will explain now why is that.

Objects are an Extensible Protocol

Great thing about objects is that they behave. Unlike files and resources that just passively sit there, object actively behave. That was, and still is, their biggest selling point.

But that's also their biggest shortcoming. By being endowed with the ability to behave, objects have become unpredictable.

If we examine a typical object, such as for example a Customer, we see that it has a public interface. What that means is that we can send messages to it and it will respond to those messages by behaving. In less technical parlance, we usually say that objects are 'bundles of capabilities'.

The collection of these capabilities is what we call a protocol. Protocol exists in order to enable third parties to join the conversation. If a third party does not understand the protocol, or if it isn't even aware that the protocol exists, its attempts to join the conversation will be futile.

So where's the problem? The problem lies in the fact that the object-defined protocol is entirely arbitrary. This means that an innocent bystander (i.e. a third party) stands very little chance of knowing about that arbitrary protocol. It must learn about it, it must somehow discover it.

The very fact that the onus of discovering/learning the protocol is placed on the client (i.e. on the consumer) means that the solution is very expensive. In contrast, any client using the 'file' abstraction in the Unix solution is free from any such expectations. An innocent bystander in the Unix world is only expected to know the standard, immutable protocol (which is by the way quite simple) in order to accomplish anything.

The limitlessly extensible protocol that the world of objects imposes on software developers is very detrimental to achieving elegant, acceptable solutions. Something must change.

Rediscovering the World of Resources

Resources are limitless. So how's that an improvement over the world of objects?

In the resource oriented development, we tamper the infinite and unpredictable variety of resources with the extremely rigid and simple protocol. True, there are possibly countless resources we might encounter, but the way to engage in the conversation is extremely simple and immutable.

Basically, we have adopted the Hypertext Transfer Protocol (HTTP) when dealing with resources. And this protocol defines extremely stringent set of acceptable behavior. In a nutshell, when dealing with resources, we can identify and locate them via the abstraction called Uniform Resource Identifier (URI). Once identified and located, a resource is susceptible to only a handful of possible behaviors. And in practice, this set of acceptable behaviors boils down to four notorious ones:

1. Deliver the representation of the resource
2. Modify the resource
3. Destroy the resource

In addition to the above, it is also possible to add, or create a resource that never existed before.

This protocol is published worldwide, and shouldn't come as a surprise to anyone using the web to accomplish something.

The rigidity and the immutability of this protocol offers a lot of power and robustness to the software written to accommodate it.

How to Use Web Resources

Web resources are exposed and are accessible via networks. Anyone with sufficient credentials is free to use them and to manipulate them.

But the way these resources usually get manipulated is very problematic.

What or Who?

Basically, there are two ways to accomplish a destructive action:

1. You can first decide that you want to destroy something (what to do), and only after making that decision will you choose who will be destroyed
2. You can first decide who needs to be destroyed, and will only then perform the destructive action (what to do)

For example, if there is a resource on the web to be destroyed, what is most likely to happen is that someone will implement it like this:

http://resource/delete?id=x

What we see here is the decision to first switch to the what mode ("I'm now switching to the slash-and-burn mode"), and only after that the decision is made as to who to delete. The mode (i.e. delete) is declared first, and only after that do we get the identity of the resource to be deleted.

This is disturbingly wrong. It is much more advisable to switch the order of doing things, and to first decide on who is it that needs to be modified/deleted. Once the resource has been fully identified, we can perform the desired action, such as delete or update, etc.

So in the above example, we would simply say:

http://resource/x

By doing that, we will first fully identify the resource ("it's the resource x"). Only after doing that (i.e. after obtaining the handle on the resource), will we send it a message to change itself (such as delete).

This results in a much clearer, much less ambiguous implementation.

How to Design Software for Human Consumption

Most software developers I've worked with so far tend to specialize in knowing how to design software for the machine consumption. The underlying computing machinery (the CPU, the auxiliary data storage, etc.) is notorious for being very finicky, very demanding. In order to appease the incessant tantrums that the machinery usually throws at us, most software people have learned to focus all their efforts on pleasing the machinery.

That leaves human consumers shortchanged. I would like to advocate here the shift away from trying to please the machinery, and to focus our efforts toward trying to please humans. After all, humans are the most expensive resource in this age of knowledge-based economy.

What do Humans Need from Software?

Basically, what we need from software boils down to two things:

1. An answer to some question
2. Processing some information

I won't go here into the gaming/entertainment side of software. Excluding these two aspects of software, everything else seems to be covered by the above two facets.

How to get an Answer from Software?

When you approach a software product with the desire to obtain an answer from it, you are basically doing one of the following two things:

1. You have good reasons to believe that the answer is 'in there'
2. You are 'fishing for results'

In the first case, you are convinced not only that the answer is in there, but also that you've come to the right place. For example, if I lose my driver's license, I'll go to my local Motor Vehicle Branch and ask for a new one. Not only do I know that all the information necessary for issuing the new one exists in some government office, I also know that I've come to the right place. It wouldn't be of much help if I went to the Revenue and Taxation office, for example.

In the second case, you may not be sure that the answer even exists, let alone that you've come to the right place. But you're fishing for it anyway.

How to get Software to Process some Information?

A software product may not only contain the ability to answer pointed questions, it could also be capable of transforming the information. If we compare a software product with a human servant for a minute, we could say that, just as a human servant can assist someone in obtaining something (such as a glass of water, or an instruction on how to get to the nearest pharmacy), software can do the same type of service. But in addition to that, just as a human servant can modify the existing resource and turn it into something else (like, use wheat flour, yeast, salt and water and bake a loaf of bread), so can a software product modify the existing information and turn it into something else (like, use the supplied information about the customer and the product to create an order to be shipped).

In order to enable software to perform such transformations, we need to make sure that the following three capabilities are present:

1. Create new information
2. Modify existing information
3. Delete existing information

Working in conjunction with the ability to answer questions, software can produce and manipulate information, thus offering the whole gamut of services to its human users.

Take the Rails RIDE

Rails is the best-of-breed technology that offers this kind of simplistic approach to software development today. It is based on the idea that any desired software behavior could be modeled on the principles of irreducibly simple fourfold behavior:

1. Retrieve
2. Insert
3. Delete
4. Edit

Any situation could be modeled as either being appropriate for retrieving information, inserting new information, deleting existing information, or editing existing information.

Once we acclimatize ourselves to viewing software modeling that way, we are capable of expressing our intentions to Rails without worrying about how will these intentions be implemented under the hood.

Such arrangement finally opens the door for us to start worrying about the human side of the experience, and to stop fretting over the machine side of the equation.