Long ago there was a palace with two towers. In the West tower, the King had his office where he would meet with his Ministers and other staff to rule the country. In the East tower, the Queen had her office from where she would regulate the Palace staff, take care of the Royal Heirs and to take care of all social State Affairs. A hallway connects the two towers and has some side rooms for e.g. the Prime Minister and other important State people and guests to reside.

This will be a discussion about messaging systems in the computer world, using a monarchy as an example. The King would be a main process on a single machine, either virtual or real. The Queen is another process, most likely running on a second machine and meant to keep the main system healthy. (The heir would replace the King if something would happen to him.) The other rooms would be other processes, most likely on other computers that the King or Queen would need to get data for their decisions. And the hallways are the network, connecting all these processes.

If the King and Queen and all Ministers and guests would be in the same room, communication between them would be very simple. But when you take into account the size of their staff and have to fit it all in a single room, you’ll soon discover that it won’t fit. The same applies with having all business logic within a single process or on a single computer. The amount of memory or disk space is limited on computers and you’re risking that you’ll run out of space soon enough. You would have to reduce the size of the processes, meaning getting rid of some ministers and guests or (in computer terms) data. But it would be more practical to divide it all over more rooms, which will allow the staff to expand more, thus contain more data.

So the royalty divides itself over several rooms in the Palace, with the King and Queen having the best and most practical rooms: the towers. This does make communication between them more difficult. Still, if the King wants to tell something to the Queen, he could go down his tower, walk through the hallway, climb up the Queen’s tower and then tell her what he wanted to say. This trip could take him 15 minutes and once he’s done talking, he will have to go back again. Thus the King can’t do his own work during at least 30 minutes.

In computer terms, this is equal to synchronous communications. And to be honest, it was quickly decided that this would never work. You can’t move a process in the context of another process on a different machine, let them work together and then bring the process back again. (Well, you could, but it’s just too complex.) So this has been solved by the use of messengers. Or, in TCP/IP terms: messages.

So, the King would write what he wants to say and puts the letter in an envelope. He calls the messenger and gives him the envelope. The messenger then runs to the Queen and gives her the message. The Queen then starts to write an answer, put it in an envelope and will use the same messenger to send it to the King again. Meanwhile, the king knows the trip will take 30 minutes and calculates the Queen will need 10 minutes to write the answer. Thus he will wait 40 minutes for the messenger to return before he can send out the next message. If the time is up, he will have to assume there’s an assassin in the hallway who killed his messenger and will just call in a new messenger.

When the messenger returns, the King picks up a gun and kills the messenger! He doesn’t need any messenger until his next message so why waste food or attention to this messenger? The body can go down the garbage chute and the garbage collection will deal with it.

This is a basis for the synchronous communication which you will see in many Internet-based applications. It’s not really synchronous, since the process can still continue to do other tasks that don’t need anything that will be in the answer from the Queen. Basically, a thread is started to send a message to the other process and then wait for a reply. If the thread returns with an answer, the thread can be closed, destroyed and will be picked up by the garbage collector to free resources. If it does not return in time, the process will have to assume something happened to the message, thus an alert should be raised and actions will have to be taken to discover why the message failed. These discussions can include checking the network for errors or just sending the message again with a new messenger.

It’s not really a synchronous process, since the King can decide to do other things, but the King can send only one messenger at once, so he will have to wait before he can send another message. Still, killing the messenger seems to be a bit “troublesome”.

So the King decides to have a collection of messengers ready to send messages. He knows he needs about 4 minutes to write each message while the Queen needs about 10 minutes. Thus he needs at least 10 messengers to continuously send messages to the Queen. When the messengers return, he can put them back in his collection, thus re-using his resources instead of wasting them.

In computer terms, this translates to a thread pool. Simply put, the system creates an area where it can keep threads that it don’t need for now, so it doesn’t have to spend time on creation or destruction of these threads. The process is still synchronous, since the King expects messengers to return. And if messengers are delayed, he might run out of messengers so he would need a few more, just in case. He can always order one more messenger, but it takes some time to assign this job to some random citizen who happens to be in the Palace at that moment. It is effective, since it allows the King to continue working and to continue sending messages to the Queen. He only gets in trouble when he needs an answer from the Queen before he can continue working. But while he hasn’t received the answer, he could still work on other tasks.

But what if the Queen also wants to say things to the King? Does she wait until the King sends her a message and add her sayings to the response? That would not be practical, since she never knows when a new messenger will arrive. What if the King doesn’t have anything to say? So the Queen will also have her share of messengers, just like the Ministers and the guests. Thus, they can all send messages when they like and wait for the responses. And how much messages they need depends on the distance to the receiver and the amount of time the receiver needs to write the answer.

For example, the Prime Minister might have a room next to the West tower. Thus, his messengers would only need to travel 5 minutes to the king and then wait for 4 minutes, since the king writes really fast. If the Prime Minister needs about 7 minutes to write his messages, then he just needs 3 messengers to be able to continuously send messages to the King. But if he also wants to communicate with the Queen, he will need more messengers since that distance is longer and the Queen writes slower.

In all these cases, we’re dealing with a request/response system, which is basically what most websites are. The browser sends a request to the server and waits a specific amount of time for a response. If a response doesn’t return in the proper amount of time, the user will be told that there’s a technical problem. There’s no way of knowing if the request failed to arrive or if the response failed. What, for example, if the Queen was assassinated? Dead Queens can’t reply. And a new Queen would be needed to get all operations running smooth again. But making a new Queen takes time. And in the Real world, the King would have to approve the new Queen, but fortunately on the Internet, the user would never notice the replacement of the web server, except for the temporary time during which they won’t get responses. Kings might be mourning the loss of their Queen, but visitors of your website won’t care if you’ve replaced your server, as long as they get their responses…

You could, of course, save downtime by having a second Queen prepared and ready when the first one dies. Basically, you would build a second East tower and have a sign telling the messenger which tower he should take to visit the Queen. You could even have two Queens at the same time, letting messengers decide by random which Queen they will visit. Unfortunately, since the Queens don’t know what the other Queen has said or responded, the King might receive some strange messages. Thus the Queens would need more messengers to send carbon copies of every message sent or received to the other Queen so they both know what’s going on.

Still, in this system we wait for responses after every message. What if we decide to stop waiting?

So, we start anew. The West tower holds the King with a collection of messengers. A bunch of citizens are in the area, who can all be promoted to messengers if there’s a need for more messengers. At the east side, we now have two towers, each holding a Queen and each holding a collection of messengers and more civilians who can be promoted. The same is true for the Ministers and the guests.

So, the King writes a message and tells a messenger to deliver it to the Queen. The King then starts working on other tasks, delaying those for which he needs an answer from the Queen. He basically forgets about the messenger and won’t wait. He just calls for another citizen to be promoted to messenger The messenger decides to take the left entrance for the East towers and gives his message to the Queen. Then he’s downgraded to citizen again, free to walk around. Most likely his proximity to the Queen means he’ll be promoted to messenger real soon again.

The Queen writes her message in reply to the King and gives it to her messenger. Her messenger is on its way and a citizen is promoted to messenger while the Queen continues her work again. When the messenger arrives at the West tower, the King remembers the task again that he had delayed. Since he now has the data he needs, he can continue this task again as long as no other task has a higher priority.

This is a pure asynchronous process. It’s advantage is that you don’t have to wait anymore. All you do is delay tasks until the moment that you can actually do something again for this task, which generally occurs once you have a response. until then, the King just can say that he can’t finish the task, since he doesn’t have data.

Of course, there are drawbacks. The King will never know if his message arrived. Does he need to? If he just wrote “I love you.” to the Queen, does he even need a response? Yeah, it would be nice if she responded with “I hate you and only married you for your money.” but it’s just optional. (And no, this is no love story!) The same is true for messages in the opposite direction. The Queen might send messages to keep the King up-to-date without expecting even a thank-you note. An asynchronous system is ideal in situations where you’re not really interested in responses, just regular status updates. Status updates are needed just to know the other sides are still alive.

But let’s make things more interesting… The King decides to have multiple Queens and marries about 40 different women. Every Queen gets a tower on the east side. This way, the King can produce plenty of heirs and knows that the social parts of his court are well-tended for. He will still send messages to the Queen, but the messenger can just pick one at random. If one Queen is unavailable, the messenger just goes to the next one.

This is a practical advantage of asynchronous messages. You don’t care who picks up the message, as long as it’s picked up. You’re not waiting for a reply either. If the Queens are supposed to reply in some way, it doesn’t even matter which Queen replies, as long as there’s a reply. This would be a system where a service is replicated on several different machines in several different processes. It allows the system to continue working without the King even noticing that half of his Queens are assassinated… Or if half the machines are down for an upgrade.

The drawback is, of course, that the Queens don’t know anything about the messages the others have sent or received. This might be troublesome, since they might receive a message from the King as part of a longer discussion, but they just don’t know what the discussion is about or what has been discussed before. So, every Queen will have to make carbon copies of every message, sent or received, and forward these to all other Queens. With 40 Queens, this would make the hallways between the East towers very busy with lots of messengers.

Another solution would be if every Queen would send a carbon copy to a “Mother-Queen”, a Queen to rule all other Queens. This way, the Mother-Queen would know about all discussions and there won’t be 40 messengers needed for every message to the Queens. Now you need only two. One that the King has sent and one the Queen sends to the Mother-Queen.

The Mother-Queen will collect all incoming copies and once in a while send these to the other Queens as updates. This way, each messenger would carry dozens of messages every time to keep the other Queens updated. It does increase the risk that a Queen receives a message from the King without knowing the previous discussion but all she would have to do is wait for the Mother-Queen to send the update so she will know what the discussion is about. besides, the King isn’t expecting an immediate answer so any reply could wait.

Something similar can happen in an asynchronous system. A message could be received by a listener that’s part of a longer discussion. The listener would then just have to wait until it received the previous parts of the discussion. Since asynchronous systems don’t reply, the tasks that are needed to be executed in response to these messages can be delayed.

So, how does this translate into computer processes? Say, for example, that every Queen is a process containing a copy of data from a database. As long as all systems have the same copy, it doesn’t matter which one you will ask to insert a new record. Process 21 might receive it and then tell the Master process to do the same update. Once in a while, the Master process will send a complete list of all updates received to all other processes and they will all be synchronized again.

Things do become more complex when you’re sending a request for data to one of these processes. Because now you’re going to synchronize a request with a response. If process 21 receives the request, it can respond with data including the newly inserted record. But if process 36 receives the request, it still doesn’t know about the insert, thus you will receive outdated data.

Basically, this means that you will have to share updates between processes real fast. You could, for example, send out 40 messengers with carbon copies of the insert message, or you will send a single messenger with 40 carbon copies and let him go past all other Queens systems. Hopefully, all systems will receive the update in time, else the King might receive responses that don’t really match with one another. This synchronisation of incoming updates make things difficult when you have many processes that all do exactly the same job.

So, let’s focus on the Ministers instead. You have one for Foreign Affairs, one for Healthcare, one for Internal Affairs, one for Military purposes et cetera. Basically, they all have different jobs, although they do share some common knowledge. Here, you can send a messenger to a random Minister and the Minister will have to look at the envelope to see if he can do something with the message. If he can, the messenger becomes a citizen again and the Minister starts working. If not, he will tell the messenger to try the next Minister.

This too is a design pattern in the computer world. Basically, these are messages where trial-and-error is used to find a specific system which will be able to reply. With too many listeners there will be a serious delay until the message finally receives the proper place but then again, no one is expecting a reply to the message so there’s no real hurry.

So, when to use which system? Generally, when you’re dealing with users of your software, you have users who expect responses to most of their actions. But not all! For example, if a user requests a web page, they expect a page. But if they save a file to disk, they don’t want a response telling them it was successful. They’d only expect a response if something failed.

But when you design web sites, you enter a world which basically has mostly asynchronous systems. Asynchronous systems which have a synchronisation layer on top of them to make usage easier, but which will also cause delays. If speed is important, you would prefer to use asynchronous processes because many actions might not need an immediate response. Basically, this would allow you to send several messages to other processes, which in turn send new messages to more processes until one or more will start sending answers back so you can respond to the user.

And if you don’t get responses back in time, you can tell the user that you could not collect all data he wanted and possibly even tell him what data you’re still missing. The user will not be waiting forever, but the notification to the user could be that you’re processing his request and that you will tell him later when you do have the required data.

And this is practical when your user is asking for complex things like large reports in PDF format. It takes time to collect all data and then convert it to readable text. In an asynchronous system you would tell the different groups of data processes to collect the required data and to send it to the data-processing processes. Those would then convert the incoming data to practical tables and other overviews and send them to the report builder process. The report builder would then create pages for all processed data and turn them into pages of the report, and when all data has been received it can send the report back to the master server. With a bit of luck, the report can be part of the response. If not, the report will have to be stored until the system has a chance to tell the user that there’s a report available.

Fortunately, with modern web browsers that support HTML 5 you can use Web Sockets. With a web socket the web page can open a communication channel with the web server, thus when the web server receives the report, it can tell the user that the report is ready to be downloaded. With older browsers, you will have to fall back on polling mechanisms, where the web site makes continuous requests to the web server, asking “Is it there yet?” Annoying, but polling is the only solution with older browsers or servers that don’t support web sockets.

Still, many people don’t realise that even a web page uses asynchronous messages for its communication. It’s just that there’s a synchronisation layer on top of those messages which makes it seem synchronous. But if the server doesn’t respond in time, the browser will still tell you that it didn’t receive a reply thus it has nothing to show. The browser is just delaying its task while waiting for a response.

This is something you can see more clearly when you open a page with multiple images. In general, the browser will ask for the page, which will result in a response. But the browser notices it needs to load several images now, so it starts requesting those images, and while it’s waiting to download those images, the user will just see the “broken image” icon but he’s still able to use it.

The same is true when you include JavaScript files or stylesheets in your web page. The browser downloads your page but will notice that it needs to download more files before it can render the page for the user to view. It will try to allow the user to do something as soon as possible, yet it has to wait for all those responses or just generate a time-out when it runs out of time.

Like the King and Queens who talk with one another through asynchronous messages, so does the Internet communicate between servers. Systems will send messages and then wait until they know they have received the data they need or a specific time limit has passed. Keep in mind that there’s nothing really synchronous about the Internet, except that most users will think it is a synchronous process.