Software Development Detox Part 5: Session

In my previous installment (oh-my-god, it's already been a year!), I've discussed the hard-wired need to retain central point of control when building software. Today, I'm going to examine the central software concept that implements this urge for control: session.

Session is a vessel used for retaining the memory of all the events that had occurred during the software interaction. It often gets implemented with the intention of providing continuity during the interaction.

Because of its central position in the overall architectural solution of a networked software, session as a concept is very brittle. It can quickly gain entropy and can end up costing a lot in terms of computing infrastructure.

On the web, however, since the prevailing architecture is stateless, the concept of session is entirely unnecessary. The memory of an interaction that may occur on the web need not be retained by any participant.

Realization of this simple dictum is proving to be extremely difficult for many software developers. They feel that, without having the crutches and training wheels that the concept of session provides, they will have no way of knowing how to make a decision on what to do next. At least when they use the state of the conversation, as recorded in the session, they can write some logic that would help them execute some meaningful actions. But without that knowledge available to them, they feel lost.

This sentiment bellies the lack of understanding of the web architecture. On the web, each request contains all the information necessary for the server code to make a decision on what to do next. There is absolutely no need for the server to keep track of any previous requests. Not only is such a task exorbitantly expensive, it also creates a single point of failure, which is the primary cause of faulty software being deployed live worldwide.

So the advice to all budding software developers is: abandon the idea of a session, and rely on your own wits when building the business logic that drives your site. Going through such software development detox procedure will ensure that you build and deliver robust, lightweight web sites that will be easy to host and maintain.

Software Development Detox Part 4: Tribal Computing

One of the absolutely most heinous types of software intoxication is what I like to call Tribal Computing. This is the form of computing that is tied in with the primitive, proprietary setup and is a remnant of the old days of early adoption of computing by the businesses.

Ontogeny recapitulates Phylogeny

Ernst Haeckel (a German natural historian) wrote in 1868: “Ontogeny, or the development of the individual, is a short and quick recapitulation of phylogeny, or the development of the tribe to which it belongs.” (in this context, ontogeny is the development of the fetus, and phylogeny is the evolution of a species). Haeckel was referring to the way the fetal development of mammals seems to parallel the evolution of all life on earth. The fertilized mammalian egg first resembles a single-celled amoeba, then a multi-celled sponge, then a jellyfish, then an amphibian, then a reptile, then finally becomes recognizable as a mammal.

Applied to the field of computing, one could say that the development of computing seems to parallel the development of human society. At first, the society was segmented into small primitive tribes, which slowly evolved into larger units, such as cities, counties, provinces, regions, countries, nations, and so on.

We are today standing on a threshold of global computing (i.e. the web). But, in certain ways, we still seem deeply entrenched in the primitive, stone age world of tribal computing.

Think Locally

Most software in operation today is severely localized. Meaning, it was built to live on a single box, single CPU. Everything about it is very closed, very proprietary, and extremely local.

Some software allows for certain level of connectivity, whereby other software systems are given an opportunity to connect to the localized software and share/exchange some information.

Only the latest, most recent batch of software products (the so-called social software) has left the world of tribal computing and is reaching out to the global computing space.

Control Locally

Most tribal software was/is built with an engineering frame of mind. Whenever we approach building something with an engineering outlook, we are striving to introduce maximum level of control into the system.

One of the most detrimental side effects of building software with an engineering slant is the temptation to retain the state of the conversation. As we've seen in our first installment (State), the best way to create brittle and buggy software is to insist on retaining the state of the conversation that had transpired during the operation of the software product.

In addition to that, insisting on staying local (i.e. tribal, single box, single CPU etc.) means that the point of control also stays tribal. There is a single authoritative instance that claims to know everything and that controls what can and cannot happen on the system. That instance then becomes a single point of catastrophic failure.

Relinquish the Control Globally

In contrast, non-tribal software exhibits stunning capabilities for growth thanks to relinquishing the rigid engineering attitude. One of the fundamental reasons why web is such a spectacularly successive computing platform lies precisely in this abandoning of the tribal past and moving beyond the need to control and retain the state of the conversation.

By deciding to not care about the state of the systems engaged in the conversation, the non-tribal, globally oriented software is free to grow in any direction and to scale to any level of complexity.

We will see in the next installment what are the most optimal ways to achieve that level of robustness. Stay tuned.

Software Development Detox Part 3: Join the Conversation

So far we've established that the unsophisticated software solutions tend to be brittle and unreliable due to the following erroneous decisions:

1. Keeping track of the state of the conversation between the client and the server
2. Choosing to expose services through messaging interfaces (a.k.a. Remote Procedure Calls)

Today we're going to look into the minimal requirements for establishing the platform for enabling successful implementation of the business computing. We will see how important it is to separate the state of the conversation, that may have transpired between two or more business parties, from the transformations of the state of the participating parties. Also, we'll look into the advantages of abandoning the messaging interfaces when conducting business conversations.

Sustainable Business Strategies

Abandoning the world of software for a moment, let us examine the strategies that facilitate success in the world of business. Experience has shown that short term success is certainly achievable by exclusivity (i.e. by locking customers into some sort of a proprietary business model). However, for achieving any level of sustainability, businesses must, willy-nilly, open up and introduce free choices and embrace diversity.

In order to attain happy equilibrium of a sustainable business growth, one component of the business interaction must be optimized -- conversation. Businesses thrive on open-ended conversation. What that means is that while initially only two privileged parties may begin business conversation, other interested parties should be able to join and participate at any point.

For that to happen successfully, the barriers to entering the stream of conversation must be placed extremely low. In practical terms, anyone interested in joining the conversation should not be expected to learn any arbitrary set of rules, any new language, or any new protocol.

In addition to this fundamental requirement, parties joining the conversation midstream should also be able to easily examine the history. Anyone joining the conversation should be able to immediately gain access to the 'minutes' of all the previous conversations. That way, an interested party could, at their own convenience, examine the minutes and learn about some vital stats of the ongoing business transactions.

These two (the ability to join the conversation at any point in time and the ability to examine what had transpired up until that moment) are the most vital and essential requirements for conducting successful, sustainable business transactions.

Sustainable Business Computing Strategies

When it comes to achieving sustainable levels of business computing, the challenges tend to be even greater. Computing infrastructure and practices, that are being placed between various business parties interested in joining the conversation, serve to raise the barriers to entry.

Still, the promise of information technology is to make those barriers even lower than they tend to be for the non-digital business interactions. And in all truthfulness, that really is the true calling of this technology. So, the question then is: what went wrong?

The simplistic answer to the above question is that someone along the lines made a couple of wrong decisions (see the erroneous decisions listed above) and created a complicated situation resulting in businesses being unable to join interesting and potentially profitable conversations that are happening online.

Our job, then, is to remedy this situation, and to bring the technology back to the level where pretty much any interested business party is capable of easily joining the conversation. Not only that, but any party should qualify for easily examining the history of the conversation by replaying the 'minutes' of what had transpired while they were away.

Simplifying the Medium

In the world of information processing, bits and bytes could represent anything. But in the world of business computing, the medium that is of most interest is text. So in most situations, what is of particular business interest is content that is rendered as text.

Of a much lesser significance, but still figuring rather prominently in the world of business computing, are images. Less prominent media types would be audio and video.

So the viable computing platform that would fully facilitate successful business conversations is mostly focused on the content rendered as text. That content should be marked up in order to achieve certain level of semantic order. But the markup shouldn't be blown out of proportions.

Simplifying the Protocol

Once the medium for exchanging business content is sufficiently simplified, we must make sure that the channels for communicating that content are also fully simplified. Instead of expecting any interested party to learn the intricacies of a foreign language that is unilaterally enforced by the vendor, we must offer a severely restricted list of possible actions that are publicly vetted and extremely non-volatile.

For any resource that the businesses may be interested in, we can safely establish that only four actions would suffice when it comes to maintaining that resource:

1. Add resource
2. Remove resource
3. Modify resource
4. Fetch resource

The above four-pronged protocol is sufficient for conducting any business conversation. No further elaboration on the business computing protocol should ever be required (there is no need for any kind of messaging interfaces).

Joining the Conversation

When an interested party learns about the ongoing business conversation and wants to join in, all it has to do is fetch the state representation of the resource in question. From that point on, the interested party is free to propose any transition of the state of that resource. In addition to that, the interested party is also free to replay the conversation as it had unfolded prior to joining the conversation.

The above scenario is universal, and it applies across the board. Following this model, we can ensure that any business parties will have a successful transition to participating and contributing to the ongoing stream of business conversations.

Software Development Detox Part 1: State

In the first installment of the software development detox program, I am going to review misconceptions and misunderstandings of one of the most fundamental software concepts -- state.

The problem with understanding state in the context of information processing/software development arises when people fail to recognize and acknowledge that there are two distinct aspects of state in the arena of software development. By bundling up these two aspects, people end up projecting an incorrect picture and consequently paint themselves into a corner by choosing the unsuitable architecture.

I am now going to (temporarily) abandon metaphors (such as 'paint oneself into a corner' etc.) and switch to using simple, albeit somewhat exaggerated examples.

Software Types

In this example, I am going to review a common occurrence of a typical software construct, such as date. Date is an abstraction devised to encapsulate and express human concept of time. In the world of information processing, we use software constructs, such as types, to encapsulate and express abstractions such as calendar date.

Suppose someone offers a software abstraction (i.e. type) called CustomDate. This abstraction is supposedly capable of doing accurate date calculations, and is endowed with certain conveniences. One such convenience being the ability to express, in calendar terms, such human concepts as 'tomorrow', 'yesterday', 'next week', etc.

So we see that this type is capable of certain behavior (such as being able to answer the question 'what date is tomorrow?', etc.) But, in addition to discernible behavior, software types typically also possess state. For example, our CustomDate may possess a state of knowing what date is year-end.

This state may change (different corporations have different year-end dates). And the instance of the type is expected to remember the changed state.

What can you say and how it gets interpreted

Upon acquiring a new software type, such as CustomDate, we will be expected to learn about its capabilities. We are not expected to understand how it is working. We're not even expected to understand all of its capabilities. We are free to pick and choose.

For example, if the CustomDate possesses 50 different capabilities, and all we want from it is the ability to tell us what date is the year-end, we should be able to safely ignore the remaining 49 capabilities.

To violate this basic agreement would result in creating brittle, unreliable software. Here is one fictitious example that illustrates this problem:

If we instantiate CustomDate and assign that instance a handle such as customDate, we should then be able to talk to that instance. If we are only interested in learning about our company's year-end date, we can send a message to our customDate instance, as follows:

customDate.year-end

In response to receiving that message, the customDate instance will return the actual year-end date to us.

The above described scenario should always yield the same behavior. There shouldn't be any surprises in how an instance of customDate behaves upon receiving the year-end message. If there is even a slightest possibility that the established message may render different, unexpected behavior, our software is not only brittle, but extremely buggy.

By now you may be wondering how could there be a possibility that the above scenario ever yields any different behavior than expected? Let me explain with another example:

We've learned so far that, when dealing with an instance of customDate, we can say year-end and it will be interpreted as a question that reads: "could you please tell me what is the year-end date?" Consequently, the representation of the correct year-end will get rendered and served as a response to our query. We've thus realized that an instance of customDate has state. That state (i.e. the actual value of company's year-end date) is the only state we're interested in when dealing with this software construct.

However, as we've mentioned earlier, this software construct may have 49 other capabilities and states, of which we know nothing. Now, the fundamental principle of software engineering dictates that we are absolutely not required to know anything about any additional, extraneous states and behaviors that a software construct may bring to the table.

Regardless of that prime directive, people who are not well versed in designing software solutions tend to violate this dictum on a daily basis. The way to violate the prime directive would be to introduce certain state/behavior combo that will modify how the question gets interpreted. One can imagine how easy would it be to add a capability to CustomDate which will turn it into a currency conversion utility. This example is admittedly unrealistic and exaggerated, but I chose it to illustrate the foolhardiness of arbitrarily assigning various capabilities to a software construct.

In this example, an overzealous green developer may add a capability to CustomDate that will put it into a "currency conversion" mode. If someone else is using the same instance of CustomDate and puts it into this "currency conversion" mode, that change in its state may modify the behavior of an instance of CustomDate, rendering the response to the year-end question unintelligible.

Let's now run this hypothetical scenario:

1. CustomDate gets instantiated as a resource on the server


2. A message arrives from a client asking the resource to convert 100 USD to Canadian dollars


3. An instance of CustomDate (i.e. customDate) puts itself into the "currency conversion" mode and renders the proper currency conversion


4. The client then sends a message to customDate asking it for a year-end


5. The instance renders an answer that corresponds to the value of 100 US dollars at the year-end

The above answer at step 5 comes as a complete shock to the client who asked for a year-end; the client wasn't aware that the instance can be shape-shifting and consequently may not always be returning dates when asked about the year-end.

In other words, what you can say and how it gets interpreted changes based on the state that an instance of the type may be in. A very bad situation, guaranteed to render that particular software program dysfunctional.

Statelessness

We can see from the above example how disastrous it can be to attempt to manage the state of a resource. In our case, we've been managing the state of an instance of CustomDate, keeping track of when is it a date rendering machine, and when is it a currency conversion machine.

This tracking of the state resulted in the breakage of the working code. If we had abstained from keeping track of the state of the instance, the problems wouldn't have emerged in the first place.

From this we see that the only way to achieve robust and reliable software is to ensure that its constituent components are stateless. No memory of what had transpired during previous conversations should be retained.

However, keep in mind that we must distinguish here two types of states:

• Entity state
• Conversation state

It is this conversation state that is troublesome. Entity state is perfectly valid, and should be memorized. In this instance, entity state would be the fact that our company's year-end is October 31.

Keeping track of what transpired as clients have interrogated an instance of a software component, and then retaining that state, is always disastrous. And yet that is how most inexperienced software developers tend to architect and design their software.

Coming up

In the next installment, we'll look more closely into how to architect and design stateless software.