Resource Oriented Architecture and Quality

In his recent post exposing imaginary limitations of the REST paradigm, Udi Dahan laments the lack of explicit dialog that would discuss the benefits of adopting REST. More specifically, he writes:

"What I haven’t heard is how I can use it (i.e. REST) to build large-scale distributed systems that are controlled by complex logic."

Here, I believe he is talking about his definition of quality. Building large-scale distributed systems that are controlled by complex logic sounds like a very costly proposition. Something one would not attempt just for the heck of it, just to kill a rainy weekend.

Because such a project entails a price tag of hundreds of thousands, if not millions of dollars, it is not to be taken lightly. Also, it is not something that happens very often. I would venture out to say that most of us have never been involved in such a project, nor will we ever find ourselves involved in such a thing. It is not very often that businesses embark upon building such complex distributed systems. Yes, eBay and Amazon etc. are such systems, but you could almost count these beasts on the fingers of one hand.

So the question that mystifies me a bit here is: why? Why worry about building such a large-scale distributed complex system? What's the business case for it? How many of those are you planning to build next year? What's your compelling reason for going after such a risky venture?

But assuming that Udi has his valid reasons, and that he has millions of budget dollars to back him up (which, frankly, I seriously doubt), let's engage him in his quest, for the sake of having fun.

What I'm assuming here is that, to Udi, large-scale complex distributed systems spell quality.

So You Want to Build a Multi-million Dollar System?

Why would a system we're building end up costing millions of dollars? There are several possible reasons:

1. We are using primitive technology, and thus it's taking us too long to build
2. We don't understand what is the system supposed to be doing, so we're making things up as we go
3. The scope of the system is on a runaway curve, and so we end up with explosion of features (i.e. featuritis)

In the case of primitive technology, we can imagine trying to dig the Panama canal by giving each worker a wooden spoon. That technology would be considered very primitive for the intended job, and would therefore result in the astronomical cost of the project.

Similarly, if we attempt to build an equivalent of amazon.com or ebay.com using assembler, we'll end up paying astronomical price for using such primitive technology.

In the case of not understanding the goals that the system under construction is supposed to deliver, 'fishing for results' would invariably turn out to be prohibitively expensive.

Finally, in the case of scope runaway, featuritis and bloat are well known for being insanely expensive.

Let's say Udi is unphased after reading this, and still wants to proceed with building his multi-million dollar large-scale complex distributed system. He is now curious how can REST and ROA be of any benefit to him. Let's explore.

Can REST and ROA Help?

My knee-jerk answer is: not really. But let's not give up so easily.

First, let's make two things clear:

1. If we don't understand what is the system supposed to be doing, then no amount of technology can salvage us from dying a miserable death
2. If we contract the featuritis disease and start bloating, there equally is no technological cure; we'll be doomed to collapse under our own weight

So let's assume here, for the sake of having fun, that Udi is immune from the two forms of malaise mentioned above. Let's say he has a very clear understanding of the goals of the large-scale distributed complex system he is trying to build, and let's also assume that he is immune from the featuritis and bloat (both assumptions being extremely unlikely, by the way, but let's pretend they've been licked).

Seeing a Problem where there isn't Any

OK, so Udi is concerned about the following:

"There are business rules that control which data should be returned and when. All this has been known for quite a while and has been modeled around transaction isolation levels in the OLTP sense, and needs to be handled at the application level when it comes to intelligent caching. I haven’t heard anything about REST that deals with this."

My question is: why on earth should REST be expected to deal with that? REST and ROA, as their names imply, specialize in dealing with Resources and their Representations. They don't specialize in dealing with the infrastructure that is the underpinning for the resources.

For all we know, that infrastructure might very well be implemented as OLTP, or perhaps as something else. Same as the ability of the RDBMS to index the data stored within it might be implemented using various algorithms. But in reality, who cares? The important thing is that it works and meets our expectations.

Next, Udi asks:

"If, as a result of updating one entity, other entities are affected how is that information propagated back to the caller?"

This is simple: the caller consumes resources by dealing with their representations. This is the essence of REST. The ground rule in ROA is that raw resource can never be touched by the client, or by the caller. All one is allowed to work with is the resource's representation.

Udi talks about updating an entity. I'll try and translate this into ROA speak. Instead of updating an entity, in ROA we speak of requesting a state transition for a resource. The way it works is as follows:

1. A resource gets identified (via the URI)
2. That resource then gets located (via the corresponding URL)
3. A request is then sent to that resource to change its state (i.e. to make a state transition)

At the end of that chain of events, other resources (or, 'entities' in Udi's parlance) may or may not get affected.

His question, then, is: how is that fact (i.e. that other resource got affected during the state transition) propagated back to the caller?

The answer is glaringly simple: it all depends on the use case. If the use case requires that the caller be notified of the state transition, then the appropriate representation of the new state will be presented to the caller in the form of a response.

One really feels like stopping at this point, and asking: what's the big mystery? Why is it that all these highly paid software architects are having such hellish time grasping this extremely simple concept of resources, their states, and their state transitions?

I am indeed forced to conclude that they seem to working really hard on trying to see the problem where there isn't any. Is that why they get paid those big bucks?

Here's the Skinny

Udi even gives us a tiny homework:

"Let’s say I have a simple rule: When the sum of all orders for a customer in the past quarter increases above X, the customer is to become a preferred customer. Do I PUT the new order resource to the customer resource, or should I have a generic orders resource that is “partitioned” by customer Id? Once the change to the customer occurs, should I PUT some change information somewhere so that the initiator can find out where it should go look for the change? And how should I be handling transactions around all these resources?"

Again, he is confusing apples with oranges. Let's break it down gently:

A customer is a resource. We don't know how is that resource implemented behind the firewall, and we really, really don't want to know! This point is crucial to understanding ROA. Please keep in mind that ROA is definitely not about the how.

OK, we now need to understand that a resource (i.e. customer) has state. For example, the resource could be in the state of being a preferred customer, or not being a preferred customer. However, we don't know, we have no idea how is that state implemented. Also, we don't want to know anything about those gory details.

On to how does that state change. Recall that REST stands for "Resource State Transition" (actually, this is incorrect, and this section talks about ROA, not REST; please see the correction notice at the end of this post). So 'state transition' is the crucial phrase here. Obviously, a resource is capable of making a transition from one state to another. Such as making the transition from the state of not preferred customer to the state of preferred customer.

How/when does that state occur, you ask? Well, that transition is driven by a particular event that must be described in the use case scenario. Like in Udi's use case above, in case of an event when the resource's state (such as total value of accumulated orders) reaches a predefined threshold, that resource knows how to make the transition from not being the preferred customer to becoming a preferred customer.

Again, we don't know how is this transition taking place. There are millions of conceivable ways that transition could take place, but the important thing is that it does.

All of Udi's questions surrounding this hypothetical scenario boil down to how, but that's largely irrelevant. Somehow the resource knows the proper way to make the transition from one state to the other state. When the client then makes a request for getting that resource, the resource in question will know how to represent its state, including the newly won 'preferred customer' feather-in-the-cap.

Correction: I've just realized that, in my eagerness to explain the encapsulation that Resource Oriented Architecture offers, I've made a mistake and erroneously ascribed state transition to REST. Actually, REST stands for Representational State Transfer, not Resource State Transition. The latter is a characteristic of ROA, not REST.

Sorry for the inadvertent error.