Every web developer should know about the Secure Sockets Layer and it’s successor, the Transport Layer Security. (SSL and TLS.) These techniques are nowadays a requirement to keep the Internet secure and to keep private matters private. But while this technique is great, the implementation of it has some flaws.
Flaws that aren’t easy to resolve, though. But to understand, you will need to understand how this technique works when you use it to protect the communication between a visitor and a website.
This is a very long post as it is a complex topic and I just don’t want to split it in multiple posts. So I add several headers to separate parts…
What is SSL?
When you communicate with a website, you’re basically have a line with two endpoints. You on one side and the web server on the other side. But this isn’t a direct line but goes from your computer to your router/modem, to your provider, over some other nodes on the Internet, to the provider of the host, the router of the host to the computer where the host is running the website on. And this is actually a very simplified explanation!
But it shows a problem. The communication between the two endpoints goes over various nodes and at every node, someone might be monitoring the communication, listening for sensitive information like usernames, passwords, social security numbers, credit card numbers and whole lot more. And to make it even more harder, you don’t know if your information will travel over nodes that you can trust! So, you need to make sure that the information you send over between both endpoints is secure. And that’s where SSL is used.
SSL is an encryption technique with asynchronous keys. That means that the host has a private key and a public key. It keeps the private key hidden while giving everyone the public key. You can generally use one key to encrypt a message but you would need the other key to decrypt it again. And that’s basically how security works! You have the public key and you encrypt your data with it before sending it to me. No one on those nodes have the private key, except me, so I’m the only one who can decrypt it. I can then encrypt a response with the private key and send that back to you, as you can decrypt it again with the public key.
Unfortunately, everyone on those nodes can too, as they would all know this public key as I had just sent it to you. So things need to be a little more secure here. Which fortunately is the case. But in general, you should never encrypt sensitive data with a private key as anyone with the public key would be able to read it!
Session keys…
But the Internet uses a better trick. Once you receive the public key from the web server, your browser will generate a session key, which is a synchronous encryption key. This session key can be used by anyone who knows it and right now, you would be the only one. But as you have my public key, you would use my public key to encrypt the session key and send it to me. Now I know it too and we can have a communication session using this session key for security. And once the session is done, the session key can be discarded as you would make a new one for every session.
Most people think the encryption is done using the SSL certificates but that’s not the case! SSL is only used to send a session key to the server so both can communicate safely. That is, as long as no one else knows that session key. Fortunately, session keys are short-lived so there isn’t much time for them to fall in the wrong hands…
The main weakness…
So, this seems pretty secure, right? My site sends you a public key, you create a session key and then we communicate using this session key and no one who is listening in on us can know what we are talking about! And for me it would be real easy to make the SSL certificate that would be used as most web servers can do this without any costs, and generally within just a few minutes. So, what can go wrong?
Well, the problem is called the “Man in the Middle” attack and this means that one of the nodes on the line will listen in on the communication and will intercept the request for secure communications! It will notice that you ask for a public key so it will give its own public key to you instead that of the host. It also asks for the public key of the host so it can relay all communications. You would then basically set up a secure line with the node and the node does the same with the host and will be able to listen in to anything that moves between you, as it has to decrypt and then encrypt each and every message. So, it can listen to sensitive data without you realizing that this is happening!
Authorities…
So the problem is that you need to know that the public key I gave you is my public key, and not the key of this node. How do you know for sure that this is my key? Well, this is where the Certificate Authorities (CA) have a role.
The CA has a simple role of validating the certificates that I use for my website. I want to secure my host so I make a certificate. I then ask the CA to sign this certificate for me. The CA then checks if I really am the owner of the domain at that specific moment or have at least some control over the domain. And when they believe that I’m the owner, they will sign my certificate.
Then, when you receive my public key then you can check the credentials of this certificate. It should tell you if it is for the specific domain that you’re visiting and it should be signed by the CA who issued the certificate. If it is signed correctly then the CA will have confirmed that this certificate is linked to my host and not that of some node between you and me.
Trust…
But the problem is that when your connection isn’t secure because some node is trying to listen than checking if my certificate is properly signed by the CA won’t work, as you would be requesting the CA to validate it over the same unsafe connection. The node will just claim it is so that option won’t work. No, you already need to know the public key of the CA on your system so you can decrypt my signature. And you need to have received the Ca’s certificate from some secure location. Otherwise, you can’t trust if my public key is the real thing.
So most web browsers have a list of public keys as part of their setup. When they install themselves, they will also include several trustworthy public keys from the more popular CA’s. Basically, the CA’s they deem reliable. So your browser will validate any public key from my site with the public keys it knows and trusts and if everything is okay, you will be notified that the connection is secure and the session key can be generated for further secure communications.
Otherwise, your browser will give you a warning telling you what’s wrong with the certificate. For example, it might be outdated or not meant for the specific domain. In general, you should not continue as the connection with the host has security problems!
Distrust!…
But here’s the thing… The list of trusted CA’s in your browser can be modified and to be honest, it sometimes gets modified for various reasons. Some are legitimate, others are not.
For example, this list is modified when a specific CA is deemed unreliable. This happens regularly with smaller CA’s but once in a while, some major scandal happens. For example, in 2011 it was discovered that the company DigiNotar had a security breach which had resulted in several certificates being falsified. Most likely, the Iranian Government was behind this in an attempt to check all emails that their citizens were sending through GMail. The fake certificates allowed them to listen in on all emails using the man in the middle technique. DigiNotar went bankrupt shortly afterwards, as all the certificates they had issued had become worthless.
Similar problems occurred at StartCom, a CA that actually gave away free certificates. The Israeli company was purchased by a Chinese company and some suspicious behavior happened soon afterwards. The fear was that this Chinese company (and perhaps even the Chinese government) would use this trust that StartCom had to make fake certificates to listen in on all communications in China. Both Mozilla and Google started to raise questions about this and didn’t get satisfying answers so they decided to drop the StartCom certificates. This CA had become controversial.
And then there’s Symantec. Symantec is a company that has been making software for decades that all relate to security. It is an American company and has been trustworthy for a long time. And in 2010 Symantec acquired Verisign’s authentication business unit which includes releasing SSL certificates for websites. But in 2015 it was discovered by Google that Symantec had issued several test certificates for impersonating Google and Opera. Further research has led Google to believe that Symantec has been publishing questionable certificates for over 7 years now and thus they announced that they will distrust Symantec SSL certificates in the near future. In April 2018, all Symantec certificates will be useless in Google Chrome and other browsers might follow soon.
Also interesting is that Symantec is selling their SSL business to DigiCert. This could solve the problem as DigiCert is still trusted. Or it makes things worse when browser manufacturers decide to distrust DigiCert from now on also.
But also: Telecom!
But there are more risks! Many people nowadays have mobile devices like tablets and phones. These devices are often included with a subscription to services of some mobile phone company. (T-Mobile and Vodafone, for example.) These companies also sell mobile devices to their customers and have even provided “free phones” to new subscriptions.
However, these companies will either provide you with a phone that has some of their software pre-installed on your new device or will encourage you to install their software to make better use of their services. The manufacturers of these mobile devices will generally do similar things if given a chance. And part of these additions they make to your Android or IOS device is to include their own root certificates with the others. This means that they are considered trustworthy by the browser on your device.
Is that bad? Actually, it is as it allows these companies to also do a man in the Middle attack on you. Your telecom provider and the manufacturer of your phone would be able to listen to all your data that you’re sending and receiving! This is worse, as local government might require these companies to listen in on your connection. It is basically a backdoor to your device and you should wonder why you would need to trust your provider directly. After all, your provider is just another node in your connection to the host.
Did you check the certificate?
So the problem with SSL is that it’s as reliable as the Certificate Authorities who made those certificates. It’s even worse if your device has been in the hands of someone who wants to listen in on your secure connections as they could install a custom trusted certificate. Then again, even some malware could install extra public keys in your trusted certificates list without you noticing. So while it is difficult to listen to your secure conversations, it is not impossible.
You should make a habit of checking any new SSL certificate that you see pop up in your browser and it would be a good idea if browsers would first ask you to check a certificate whenever they detect that a site has a new one. It would then be up to the user to decide to trust the certificate or not. And those certificates would then be stored so the browser doesn’t need to ask again.
Unfortunately, that would mean that you get this question to trust a certificate very often when you’re browsing various different sites and users will tend to just click ‘Ok’ to get rid of the question. So, that’s not a very good idea. Most users aren’t even able to know if a certificate is trustworthy or not!
For example, while you’re reading this blog entry, you might not have noticed that this page is also a secured page! But did you check the certificate? Did you notice that it is signed by Automattic and not by me? That it’s a certificate issued by “Let’s Encrypt“? Well, if the certificate is showing this then it should be the right one. (But when my host Automattic changes to another CA, this statement becomes invalid.)
ACME protocol…
And here things become interesting again. “Let’s Encrypt” gives away free certificates but they are only valid for a short time. This makes sense as domains can be transferred to other owners and you want to invalidate the old owner’s certificates as soon as possible. It uses a protocol called ACME which stands for “Automatic Certificate Management Environment. It’s basically a toolkit that will automate the generation of certificates for domains so even though the certificates are only valid for a short moment, they will be replaced regularly. This is a pretty secure setup, although you’d still have to trust this CA.
Problem is that “Let’s Encrypt” seems to prefer Linux over Windows as there is almost no good information available on how to use ACME on Windows in IIS. But another problem is that this protocol is still under development and thus still has some possible vulnerabilities. Besides, it is very complex, making it useless for less technical developers. The whole usage of certificates is already complex and ACME doesn’t make things easier to understand.
Also troublesome is that I tried to download the ACME client “Certify the Web” for Windows but my virus scanner blocked the download. So, now I have to ask myself if I still trust this download. I decided that it was better not to trust them, especially as I am trying to be secure. Too bad as it seemed to have a complete GUI which would have made things quite easy.
Don’t ignore security warnings! Not even when a site tells you to ignore them…
Additional problems?
Another problem with SSL is that it is an expensive solution so it is disliked by many companies who are hosting websites. It’s not the cost for the certificates, though. It’s the costs for hiring an expert on this matter and making sure they stay with the company! A minor issue is that these security specialists do have access to very sensitive material for companies so you need to be sure you can trust the employee.
Of course, for very small companies and developers who also host websites as a hobby, using SSL makes things a bit more expensive as the costs are generally per domain or sub domain. So if you have three domains and 5 subdomains then you need to purchase 8 certificates! That’s going to easily cost hundreds of euros per year. (You could use a Multi-Domain (SAN) Certificate but that will cost about €200 or more per year.)
Plus, there’s the risk that your CA does something stupid and becomes distrusted. That generally means they will have to leave the business and that the certificates you own are now worthless. Good luck trying to get a refund…
But another problem is that the whole Internet is slowly moving away from insecure connections (HTTP) to secure (HTTPS) connections, forcing everyone to start using SSL. Which is a problem as it starts to become a very profitable business and more and more malicious people are trying to fool people into buying fake or useless certificates or keep copies of the private key so they can keep listening to any communications done with their keys. This security business has become highly profitable!
So, alternatives?
I don’t know if there are better solutions. The problem is very simple: Man in the Middle. And the biggest problem with MITM is that he can intercept all communications so you need something on your local system that you already can trust. Several CA’s have already been proven untrustworthy so who do you trust? How do you make sure that you can communicate with my server without any problems?
There is the Domain Name Service but again, as the MITM is intercepting all your transactions, they can also listen in on DNS requests and provide false information. So if a public key would be stored within the DNS system, a node can just fake this when you request for it. The MITM would again succeed as you would not be able to detect the difference.
Blockchain?
So maybe some kind of blockchain technology? Blockchains have proven reliable with the Bitcoin technology as the only reason why people have lost bitcoins is because they were careless with the storage of their coins. Not because the technique itself was hacked. And as a peer-to-peer system you would not need a central authority. You just need to keep the blocks on your system updated at all times.
As we want the Internet to be decentralized, this technology would be the best option to do so. But if we want to move security into blockchains then we might have to move the whole DNS system into a blockchain. This would be interesting as all transactions in blockchains are basically permanent so you don’t have to pay a yearly fee to keep your domain registered.
But this is likely to fail as Bitcoin has also shown. Some people have lost their bitcoins because their disks crashed and they forgot to make a copy of their coins. Also, the file size of the Bitcoin blockchain has grown to 100 GB of data in 2017 which is quite huge. The whole DNS system is much bigger than Bitcoin is so it would quickly have various problems. Your browser would need to synchronize their data with the network which would take longer and longer as the amount of data grows every day.
So, no. That’s not really a good option. Even though a decentralized system sounds good.
Conclusion?
So, SSL has flaws. Then again, every security system will have flaws. The main flaw in SSL is that you need to trust others. And that has already proven to be a problem. You have to be ultra-paranoid to want to avoid all risks and a few people are this paranoid.
Richard Stallman, for example, is a great expert on software yet he doesn’t use a mobile phone and avoids using the Internet directly. Mobile phones are “portable surveillance and tracking devices” so he won’t use them. He also avoids key cards and other items that allow people to track wherever he goes. And he generally doesn’t access the Web directly, as this too would allow people to track what he’s doing. (He does use Tor, though.) And maybe he’s on to something. Maybe we are putting ourselves in danger with all this online stuff and various devices that we have on our wrists, in our pockets and at our homes.
Thing is that there is no alternative for SSL at this moment so being paranoid is useful to protect yourself. Everyone should be aware of the risks they take when they visit the Internet. This also depends on how important or wealthy you are, as poor, boring people are generally not interesting for malicious people. There isn’t much to gain from people with no money.
Still, people are still too careless when they’re online. And SSL isn’t as secure as most people think, as events from the past have already proven…
I recently received an email from a company that’s doing questionnaires. And well, I subscribed to this and did some of their questionnaires before, so I wanted to do this new one too. Unfortunately, the page loaded quite slow, only to return a very nasty error message. A message that told me that this organisation is using amateurs for developers and administrators.
Let me be clear about one thing: errors will happen. Every developer should expect weird things to happen, but this case is not an error but evidence of amateurs. So, let’s start with analyzing the message…
Server Error in ‘/’ Application.
Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
So, what’s wrong with this? Users should never see these messages! When you develop in ASP.NET you can just tell the system to just keep these error messages only when the user is connected locally. A remote user should see a much simpler message.
This is something the administrator of the website should have known, and checked. He did not. By failing at this simple configuration setting the organisation is leaking some sensitive information about their website. Information that’s enough for me to convince they’re amateurs.
This error is also a quite common error message. Basically, it’s telling me that the system is having too many database connection open. One common cause for this error is when the code fails to close a connection after opening them. Keep that in mind, because I will show that this is what caused the error…
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
This is a standard follow-up message. The fact that users of the site would see this stack trace too is just bad.
Exception Details: System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
A timeout error. A reference to the connection pool and the max pool size. This already indicates that there are more connections are opened than closed and the system can’t handle that correctly. There are frameworks for .NET that are better suited for this to prevent these kinds of errors. That’s because these errors happened to be very common with ASP.NET applications. And with generic database applications written in .NET.
Basically, the top of the error message is just repeating itself. Blame Microsoft for that since this is a generic message from ASP.NET itself. Developers can change the way it looks but that’s not very common. Actually, developers should prevent users from seeing these kinds of messages to begin with. Preferably, the error should be caught by an exception handler which would write it to a log file or database and send an alert out to the administrator.
Considering that I received this error on a Friday afternoon, I bet the developer and administrators are already back home, watching television like I do now. Law & Order is just on…
Source Error:
Line 1578:
Line 1579: cmSQL = New SqlCommand(strSQL, cnSQLconfig)
Line 1580: cnSQLconfig.Open()
Line 1581:
Line 1582: Try
This is interesting… The use of SqlCommand is a bit old-fashioned. Modern developers would have switched to e.g. the Entity Framework or other, more modern solution for database access. But the developers of this site are just connecting to the database in code, probably to execute a query and collect the data and then should close the connection again. The developers are clearly using ADO.NET for this site. And I can’t help but wonder why. They could have used more modern techniques instead. But probably they just need to keep up an existing site and aren’t they allowed to use more modern solutions.
But it seems to me that closing the database is not going to happen here. There are too many connections already open thus this red line of code fails. The code has an existing connection called cnSQLConfig which is already open. It then tries to open and execute an SQL command that fails. Unfortunately, opening that command happens outside a try-except block and if this fails, it is very likely that the connection won’t be closed either.
If this happens once or twice, then it still would not be a big problem. The connection pool is big enough. But here it just happened too often.
Another problem is that the ADO.NET technique used here is also vulnerable for SQL Injection. This would also be a good reason to use a different framework for database access. It could still be that they’re using secure code to protect against this but what I see here doesn’t give me much confidence.
A few interesting, other facts. First of all, the code was written in Visual Basic. That was already clear from the code but this just confirms it. Personally, I prefer C# over Visual Basic, even though I’ve developed in both myself. And in a few other languages. Language should not matter much, especially with .NET, but C# is often considered more professional than BASIC. (Because the ‘B’ in BASIC stands for ‘Beginners’.)
Second of all, this piece of code has over 1580 lines of code. I don’t know what the rest of the code is doing but it’s probably a lot of code. Again, this is an old-fashioned way of software development. Nowadays, you see more usage of frameworks that allow developers to write a lot less code. This makes code more readable. Even in a main index of a web site, the amount of code should be reasonable low. You can use views to display the pages, models to handle the data and controllers to connect both.
Yes, that’s Model-View-Controller, or MVC. A technique that’s practical in reducing the amount of code, if used well enough.
And one more thing is strange. While I replaced the name of the site with ‘example.com’, I kept the word ‘beta’ in front of it. I, a user, am using a beta-version of their website! That’s bad. Users should not be used as testers because it will scare them off when things go wrong. Like in this case, where the error might even last the whole weekend because developers and administrators are at home, enjoying their weekend.
Never let users use your beta versions! That’s what testers are for. You can ask users to become testers, but then users know they can expect errors like these.
Stack Trace:
[InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.]
System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection) +4863482
System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory) +117
System.Data.SqlClient.SqlConnection.Open() +122
_Default.XmlLangCountry(String FileName) in E:\wwwroot\beta.example.com\index.aspx.vb:1580
_Default.selectCountry() in E:\wwwroot\beta.example.com\index.aspx.vb:1706
_Default.Page_Load(Object sender, EventArgs e) in E:\wwwroot\beta.example.com\index.aspx.vb:251
System.Web.UI.Control.OnLoad(EventArgs e) +99
System.Web.UI.Control.LoadRecursive() +50
System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +627
And that’s the stack trace. We see the site loading its controls and resources and the ‘Page_Load’ method is called at line 251. At line 1706 the system is apparently loading country-information which would be needed to set the proper language. Then it returns to line 1580 where it probably opens some table based on information from the language file.
Again, this is a lot of code for basically loading the main page. I even wonder why it needs to load data from the database based on the country information. Then again, I was about to fill in a questionnaire so it probably wanted to load the questionnaire in the proper language. If the questionnaire is multi-lingual then that would make sense.
Version Information: Microsoft .NET Framework Version:2.0.50727.3655; ASP.NET Version:2.0.50727.3658
And here’s one more bad thing. This site still uses .NET version 2.0 while the modern version is 4.5 and we’re close to version 5.0… It would not surprise me if these developers still use Visual Studio 2005 or 2008 for this all. If that’s the case then their budget for development is probably quite low. I wonder if the developers who are maintaining this site are even experts at software development. It’s not a lot of information that I can base this upon but in short:
The administrator did not prevent error messages to show up for users.
The use of ADO.NET adds vulnerabilities related to the connection pool and SQL injection.
The use of VB.NET is generally associated to less experienced developers.
The amount of code is quite long but common for sites that are developed years ago.
Not using a more modern framework makes the site more vulnerable.
Country information seems to be stored in XML while the questionnaire is most likely stored inside the database.
The .NET version has been out-of-date for a few years now.
My advice would be to just rewrite the whole site from scratch. Use the Entity Framework for the database and MVC 4 for the site itself. Rewrite it in C# and hire more professional developers to do the work.
When I was a kid, there just wasn’t any internet. If you wanted to speak with someone else, you’d had to pick up the phone or just go visit them. Being social was complex because it involved plenty of travel to meet others. And even when the Internet was born, being social was still something that people did in real life, not behind a computer screen. Still, things slowly changed about 15 years ago, when people started to use the Internet for all kinds of fun things. It also helped that proper internet tools became more popular. (And free!) The increased speed and the change from the 33k6 modem to ADSL or Cable also helped a lot. And now, just one generation further, being social is something we do online, with bits and bytes.
But enough history. And no, I won’t explain what social media are because now, you’re reading stuff I wrote on such a social media website. (Yeah, a Hosted WordPress site, but I could have used Blogger or Tumblr too..) This discussion is about the complexity of all those social media, not their history.
Most people will be familiar with both Twitter and Facebook. On Twitter you post a message that you’ve just pooped and on Facebook you post the picture of the result. And if you’re a professional, you might also post it on LinkedIn, if you’ve pooped during office hours. Since you can connect these three together, you will start to build a practical resource with all kinds of personal information about you online. Twitter will be used to send small but important updates about yourself, your company or your products to every subscriber while Facebook is practical to connect with the consumers at home. But if you’re looking for a new job or need to hire or find some experts, you use LinkedIn for your search.
Search? That reminds me. There’s also Google Plus although not many people use it as a social platform. Still, people like it because you can use your Google Plus account to log in many other websites. (Facebook, LinkedIn and Twitter also support this.) Google also provides email accounts and document management tools, plus plenty of online storage, so it’s a very attractive site to use, even if people still are less social on Google Plus than they are elsewhere.
Yahoo also used to be a great social media center, but the competition with other sites has lessened its influence considerably. Many things that Yahoo offers is also available on other sites. Yahoo also used to be great with their email services until they decided to drop support for email through POP/SMTP, just when Google decided to start increasing their email services. By doing so, Yahoo lost much if it’s influence and never really managed to get some back, although their photo-service Flickr still holds plenty of value. (But here too, the competition becomes murderous.)
Pinterest, for example, can also be used to share photo’s with others, although Pinterest is mostly used to share pictures from others, to promote those people. Basically, it’s a site for fans. DeviantArt is a bigger challenge for Flickr and has a huge amount of graphics. Especially cartoons and CGI next to pictures. But DeviantArt is missing an easy way to connect your other social media to your DeviantArt account.
So Behance is another interesting photo site where you can build your gallery and, more importantly, allow people to contact you and offer you jobs and other career opportunities. It also connects better with other social media and if it was free, it would definitely kill Flickr. Unfortunately, the free version has limitations and the commercial version is a bit expensive if you just want to share a bit of your work. Or maybe you’d prefer Bitpine.
Then again, if you’re into the art of images and photo’s, you might like to try to make some profit by selling merchandise. Cafepress is known for this and allows you to upload pictures and put it on all kinds of things, including the cape for your dog or panties for your girlfriend. There are plenty of other sites that allow simpler merchandise like t-shirts but Cafepress just has a huge collection of things you don’t need but which still look nice with your picture on them.
There are more social media sites, of course. Including sites that will combine all your social media sites into a single reference for all your friends to know where you hang around. About.me will combine your bio, your résumé and all kinds of social media connections. Mine tends to have plenty of connections. Connect.me is also practical to connect with other people and allows you to build up your online reputation. TrustCloud is another medium that links people you know to your account. (Or mine.) Or go to Visify and tell others how active you are online.
An oldie is Reddit which is more like an online forum. However, it has so many users that all discussions go very fast. Vimeo can be used to share videos, just like YouTube. Or use GitHub if you’re a software developer and want to share your code with others. Or Society3 for those who need social media for their marketing strategies. Or, the simples one: FourSquare, where you can tell where you are and where you went.
Well, I’ve mentioned plenty of social media sites and it’s all great to share your personal information with the World and get your 15 minutes of fame. And they all connect to one another, often via ID providers from Google, Facebook, Twitter or LinkedIn and lately also from Adobe. (Although Adobe is mostly using its ID provider to have others connect to the Creative Cloud.) If you’re connected to even a third of these sites, then there’s a lot of information about you online. And this is where it starts to become creepy and dangerous.
First of all, the amount of personal information that people share is huge. The joke I started with that people tell others on Twitter that they’ve just pooped isn’t just a joke. It happens! But when people are on a holiday, they also tend to use Twitter, FourSquare and TwitPic to tell the World where they are. With more information on Facebook, thieves can try to find where those people live and rob those empty homes. They might also check LinkedIn to see if someone might have some interesting stuff at home. For example, a CEO of a company who’s on holiday in Italy is a more interesting target than a teacher visiting his aunt in Almelo. And this is just a few different media that can be abused by others without the need to hack anything.
So beware of your privacy and avoid sharing sensitive information online. Or at least be less interesting than the other online people.
But getting robbed is just one risk. You can protect your home, make sure there’s at least one person there when you’re on holiday. The problem is that all these media are connected to one another. And in general, you have given them permission to combine their information. And systems are as strong as their weakest links.
Take, for example, Facebook. Many websites use your Facebook ID to let you log in to those websites. Thus, if someone hacks your Facebook account, they also have access to those other websites. And if one of those sites has your credit card information, your bank account information or your PayPal information. They might not even need this information to make purchases in your name, simply because those connected sites remember this internally. I checked which all I use that are connected to Facebook and it turns out that I’m connected with over a hundred other websites! I know a few friends of mine have an average of around 40 other sites connected to their Facebook account and it’s easy to increase that number since plenty of sites want to connect to Facebook.
Fortunately, I have created several websites that connect to Facebook so several of those connected apps are actually my own sites. Still, it’s a lot. It means that you have to be aware that anyone who hacks my Facebook account will be able to use these other sites. What they can do on those sites depends on how those other sites have implemented their security. And the same applies with apps connected to Google Plus, Twitter or LinkedIn.
If you use Flickr or Yahoo then you might have connected that account with Facebook or Google Plus. Since Yahoo is used as ID provider for even more websites, you can see a complete chain fall down once your Facebook account is taken over. This makes Yahoo less reliable than the others. With Facebook, Twitter, LinkedIn and Google you can try to add more security. For example, only copy the ID key from the provider plus the email address and force the user to generate a new password for your site. Thus, if Facebook is hacked, they still need a password for your site.
Which causes another problem. When people have a few dozens of social media accounts, they start having troubles remembering all the passwords. I use an email alias for every site. Websites tend to allow visitors to log in with email address and password so I can use the same password for many sites, because the user email address is different for every site. (I still use different passwords too, though.) Most people just use the same address and password for many sites, though. And that’s a big risk, because if one of the sites is hacked, the hackers will be able to use that information for all the other sites.
The bigger websites do have a proper security. At least, that’s what most people think. However, both Adobe and LinkedIn have had some serious trouble with their user databases and users of both websites have received a notice in the past urging them to change their password immediately, because of the hacks. And these were just the bigger sites who dared to publicly admit they’ve been hacked. Smaller social media sites can be a bigger risk if their security isn’t strong enough. Which is why it’s actually better that they use ID providers from the bigger sites instead of implementing their own systems.
Developers often ignore security, thinking that what they’re making isn’t very interesting for hackers. But I can’t say it often enough and remind people that social media are just chained together. One weak link exposes all.
When you want to build your own social media website then be very aware of the security. Don’t build your own version unless you have an expert in your team. And even then have the code audited by another expert. Since social media chain together, a weak link in this chain will take it all down. Which reminds me of this xkcd comic:
xkcd
When you create your own ID provider, you’re just adding to the competing standards that already exist. What would make your system better than those others? Your site will be more secure by using an existing provider but if that provider has a weakness, your site will fall too unless you require more information.
My suggestion would be that people should be able to log in using Google Plus, Facebook, Twitter or LinkedIn but combine it with some extra security. You know, for example, the IP address from the visitor thus you can remember it. As long as it’s the same as in your history, it’s unlikely that the account is hacked. Once it changes, you should ask for one more extra piece of information like a separate password. The visitor should know this, since he might have had to enter it during registration.
Another option would be by asking the visitor for his mobile phone number during registration so you can send an SMS message as part of the authentication process. Thus, if a user is using a different computer, you can send an SMS with a security code. The user will have to enter that code and then you know you can trust that system. Add it to the list of trusted computers for this user and you can keep the visitor safe. (Microsoft is doing something like this with Windows Live.)
So, a long story just to start a discussion about the best way to secure social media, reminding everyone that there are actually a lot of sites chained together through all of this.
Long ago there was a palace with two towers. In the West tower, the King had his office where he would meet with his Ministers and other staff to rule the country. In the East tower, the Queen had her office from where she would regulate the Palace staff, take care of the Royal Heirs and to take care of all social State Affairs. A hallway connects the two towers and has some side rooms for e.g. the Prime Minister and other important State people and guests to reside.
This will be a discussion about messaging systems in the computer world, using a monarchy as an example. The King would be a main process on a single machine, either virtual or real. The Queen is another process, most likely running on a second machine and meant to keep the main system healthy. (The heir would replace the King if something would happen to him.) The other rooms would be other processes, most likely on other computers that the King or Queen would need to get data for their decisions. And the hallways are the network, connecting all these processes.
If the King and Queen and all Ministers and guests would be in the same room, communication between them would be very simple. But when you take into account the size of their staff and have to fit it all in a single room, you’ll soon discover that it won’t fit. The same applies with having all business logic within a single process or on a single computer. The amount of memory or disk space is limited on computers and you’re risking that you’ll run out of space soon enough. You would have to reduce the size of the processes, meaning getting rid of some ministers and guests or (in computer terms) data. But it would be more practical to divide it all over more rooms, which will allow the staff to expand more, thus contain more data.
So the royalty divides itself over several rooms in the Palace, with the King and Queen having the best and most practical rooms: the towers. This does make communication between them more difficult. Still, if the King wants to tell something to the Queen, he could go down his tower, walk through the hallway, climb up the Queen’s tower and then tell her what he wanted to say. This trip could take him 15 minutes and once he’s done talking, he will have to go back again. Thus the King can’t do his own work during at least 30 minutes.
In computer terms, this is equal to synchronous communications. And to be honest, it was quickly decided that this would never work. You can’t move a process in the context of another process on a different machine, let them work together and then bring the process back again. (Well, you could, but it’s just too complex.) So this has been solved by the use of messengers. Or, in TCP/IP terms: messages.
So, the King would write what he wants to say and puts the letter in an envelope. He calls the messenger and gives him the envelope. The messenger then runs to the Queen and gives her the message. The Queen then starts to write an answer, put it in an envelope and will use the same messenger to send it to the King again. Meanwhile, the king knows the trip will take 30 minutes and calculates the Queen will need 10 minutes to write the answer. Thus he will wait 40 minutes for the messenger to return before he can send out the next message. If the time is up, he will have to assume there’s an assassin in the hallway who killed his messenger and will just call in a new messenger.
When the messenger returns, the King picks up a gun and kills the messenger! He doesn’t need any messenger until his next message so why waste food or attention to this messenger? The body can go down the garbage chute and the garbage collection will deal with it.
This is a basis for the synchronous communication which you will see in many Internet-based applications. It’s not really synchronous, since the process can still continue to do other tasks that don’t need anything that will be in the answer from the Queen. Basically, a thread is started to send a message to the other process and then wait for a reply. If the thread returns with an answer, the thread can be closed, destroyed and will be picked up by the garbage collector to free resources. If it does not return in time, the process will have to assume something happened to the message, thus an alert should be raised and actions will have to be taken to discover why the message failed. These discussions can include checking the network for errors or just sending the message again with a new messenger.
It’s not really a synchronous process, since the King can decide to do other things, but the King can send only one messenger at once, so he will have to wait before he can send another message. Still, killing the messenger seems to be a bit “troublesome”.
So the King decides to have a collection of messengers ready to send messages. He knows he needs about 4 minutes to write each message while the Queen needs about 10 minutes. Thus he needs at least 10 messengers to continuously send messages to the Queen. When the messengers return, he can put them back in his collection, thus re-using his resources instead of wasting them.
In computer terms, this translates to a thread pool. Simply put, the system creates an area where it can keep threads that it don’t need for now, so it doesn’t have to spend time on creation or destruction of these threads. The process is still synchronous, since the King expects messengers to return. And if messengers are delayed, he might run out of messengers so he would need a few more, just in case. He can always order one more messenger, but it takes some time to assign this job to some random citizen who happens to be in the Palace at that moment. It is effective, since it allows the King to continue working and to continue sending messages to the Queen. He only gets in trouble when he needs an answer from the Queen before he can continue working. But while he hasn’t received the answer, he could still work on other tasks.
But what if the Queen also wants to say things to the King? Does she wait until the King sends her a message and add her sayings to the response? That would not be practical, since she never knows when a new messenger will arrive. What if the King doesn’t have anything to say? So the Queen will also have her share of messengers, just like the Ministers and the guests. Thus, they can all send messages when they like and wait for the responses. And how much messages they need depends on the distance to the receiver and the amount of time the receiver needs to write the answer.
For example, the Prime Minister might have a room next to the West tower. Thus, his messengers would only need to travel 5 minutes to the king and then wait for 4 minutes, since the king writes really fast. If the Prime Minister needs about 7 minutes to write his messages, then he just needs 3 messengers to be able to continuously send messages to the King. But if he also wants to communicate with the Queen, he will need more messengers since that distance is longer and the Queen writes slower.
In all these cases, we’re dealing with a request/response system, which is basically what most websites are. The browser sends a request to the server and waits a specific amount of time for a response. If a response doesn’t return in the proper amount of time, the user will be told that there’s a technical problem. There’s no way of knowing if the request failed to arrive or if the response failed. What, for example, if the Queen was assassinated? Dead Queens can’t reply. And a new Queen would be needed to get all operations running smooth again. But making a new Queen takes time. And in the Real world, the King would have to approve the new Queen, but fortunately on the Internet, the user would never notice the replacement of the web server, except for the temporary time during which they won’t get responses. Kings might be mourning the loss of their Queen, but visitors of your website won’t care if you’ve replaced your server, as long as they get their responses…
You could, of course, save downtime by having a second Queen prepared and ready when the first one dies. Basically, you would build a second East tower and have a sign telling the messenger which tower he should take to visit the Queen. You could even have two Queens at the same time, letting messengers decide by random which Queen they will visit. Unfortunately, since the Queens don’t know what the other Queen has said or responded, the King might receive some strange messages. Thus the Queens would need more messengers to send carbon copies of every message sent or received to the other Queen so they both know what’s going on.
Still, in this system we wait for responses after every message. What if we decide to stop waiting?
So, we start anew. The West tower holds the King with a collection of messengers. A bunch of citizens are in the area, who can all be promoted to messengers if there’s a need for more messengers. At the east side, we now have two towers, each holding a Queen and each holding a collection of messengers and more civilians who can be promoted. The same is true for the Ministers and the guests.
So, the King writes a message and tells a messenger to deliver it to the Queen. The King then starts working on other tasks, delaying those for which he needs an answer from the Queen. He basically forgets about the messenger and won’t wait. He just calls for another citizen to be promoted to messenger The messenger decides to take the left entrance for the East towers and gives his message to the Queen. Then he’s downgraded to citizen again, free to walk around. Most likely his proximity to the Queen means he’ll be promoted to messenger real soon again.
The Queen writes her message in reply to the King and gives it to her messenger. Her messenger is on its way and a citizen is promoted to messenger while the Queen continues her work again. When the messenger arrives at the West tower, the King remembers the task again that he had delayed. Since he now has the data he needs, he can continue this task again as long as no other task has a higher priority.
This is a pure asynchronous process. It’s advantage is that you don’t have to wait anymore. All you do is delay tasks until the moment that you can actually do something again for this task, which generally occurs once you have a response. until then, the King just can say that he can’t finish the task, since he doesn’t have data.
Of course, there are drawbacks. The King will never know if his message arrived. Does he need to? If he just wrote “I love you.” to the Queen, does he even need a response? Yeah, it would be nice if she responded with “I hate you and only married you for your money.” but it’s just optional. (And no, this is no love story!) The same is true for messages in the opposite direction. The Queen might send messages to keep the King up-to-date without expecting even a thank-you note. An asynchronous system is ideal in situations where you’re not really interested in responses, just regular status updates. Status updates are needed just to know the other sides are still alive.
But let’s make things more interesting… The King decides to have multiple Queens and marries about 40 different women. Every Queen gets a tower on the east side. This way, the King can produce plenty of heirs and knows that the social parts of his court are well-tended for. He will still send messages to the Queen, but the messenger can just pick one at random. If one Queen is unavailable, the messenger just goes to the next one.
This is a practical advantage of asynchronous messages. You don’t care who picks up the message, as long as it’s picked up. You’re not waiting for a reply either. If the Queens are supposed to reply in some way, it doesn’t even matter which Queen replies, as long as there’s a reply. This would be a system where a service is replicated on several different machines in several different processes. It allows the system to continue working without the King even noticing that half of his Queens are assassinated… Or if half the machines are down for an upgrade.
The drawback is, of course, that the Queens don’t know anything about the messages the others have sent or received. This might be troublesome, since they might receive a message from the King as part of a longer discussion, but they just don’t know what the discussion is about or what has been discussed before. So, every Queen will have to make carbon copies of every message, sent or received, and forward these to all other Queens. With 40 Queens, this would make the hallways between the East towers very busy with lots of messengers.
Another solution would be if every Queen would send a carbon copy to a “Mother-Queen”, a Queen to rule all other Queens. This way, the Mother-Queen would know about all discussions and there won’t be 40 messengers needed for every message to the Queens. Now you need only two. One that the King has sent and one the Queen sends to the Mother-Queen.
The Mother-Queen will collect all incoming copies and once in a while send these to the other Queens as updates. This way, each messenger would carry dozens of messages every time to keep the other Queens updated. It does increase the risk that a Queen receives a message from the King without knowing the previous discussion but all she would have to do is wait for the Mother-Queen to send the update so she will know what the discussion is about. besides, the King isn’t expecting an immediate answer so any reply could wait.
Something similar can happen in an asynchronous system. A message could be received by a listener that’s part of a longer discussion. The listener would then just have to wait until it received the previous parts of the discussion. Since asynchronous systems don’t reply, the tasks that are needed to be executed in response to these messages can be delayed.
So, how does this translate into computer processes? Say, for example, that every Queen is a process containing a copy of data from a database. As long as all systems have the same copy, it doesn’t matter which one you will ask to insert a new record. Process 21 might receive it and then tell the Master process to do the same update. Once in a while, the Master process will send a complete list of all updates received to all other processes and they will all be synchronized again.
Things do become more complex when you’re sending a request for data to one of these processes. Because now you’re going to synchronize a request with a response. If process 21 receives the request, it can respond with data including the newly inserted record. But if process 36 receives the request, it still doesn’t know about the insert, thus you will receive outdated data.
Basically, this means that you will have to share updates between processes real fast. You could, for example, send out 40 messengers with carbon copies of the insert message, or you will send a single messenger with 40 carbon copies and let him go past all other Queens systems. Hopefully, all systems will receive the update in time, else the King might receive responses that don’t really match with one another. This synchronisation of incoming updates make things difficult when you have many processes that all do exactly the same job.
So, let’s focus on the Ministers instead. You have one for Foreign Affairs, one for Healthcare, one for Internal Affairs, one for Military purposes et cetera. Basically, they all have different jobs, although they do share some common knowledge. Here, you can send a messenger to a random Minister and the Minister will have to look at the envelope to see if he can do something with the message. If he can, the messenger becomes a citizen again and the Minister starts working. If not, he will tell the messenger to try the next Minister.
This too is a design pattern in the computer world. Basically, these are messages where trial-and-error is used to find a specific system which will be able to reply. With too many listeners there will be a serious delay until the message finally receives the proper place but then again, no one is expecting a reply to the message so there’s no real hurry.
So, when to use which system? Generally, when you’re dealing with users of your software, you have users who expect responses to most of their actions. But not all! For example, if a user requests a web page, they expect a page. But if they save a file to disk, they don’t want a response telling them it was successful. They’d only expect a response if something failed.
But when you design web sites, you enter a world which basically has mostly asynchronous systems. Asynchronous systems which have a synchronisation layer on top of them to make usage easier, but which will also cause delays. If speed is important, you would prefer to use asynchronous processes because many actions might not need an immediate response. Basically, this would allow you to send several messages to other processes, which in turn send new messages to more processes until one or more will start sending answers back so you can respond to the user.
And if you don’t get responses back in time, you can tell the user that you could not collect all data he wanted and possibly even tell him what data you’re still missing. The user will not be waiting forever, but the notification to the user could be that you’re processing his request and that you will tell him later when you do have the required data.
And this is practical when your user is asking for complex things like large reports in PDF format. It takes time to collect all data and then convert it to readable text. In an asynchronous system you would tell the different groups of data processes to collect the required data and to send it to the data-processing processes. Those would then convert the incoming data to practical tables and other overviews and send them to the report builder process. The report builder would then create pages for all processed data and turn them into pages of the report, and when all data has been received it can send the report back to the master server. With a bit of luck, the report can be part of the response. If not, the report will have to be stored until the system has a chance to tell the user that there’s a report available.
Fortunately, with modern web browsers that support HTML 5 you can use Web Sockets. With a web socket the web page can open a communication channel with the web server, thus when the web server receives the report, it can tell the user that the report is ready to be downloaded. With older browsers, you will have to fall back on polling mechanisms, where the web site makes continuous requests to the web server, asking “Is it there yet?” Annoying, but polling is the only solution with older browsers or servers that don’t support web sockets.
Still, many people don’t realise that even a web page uses asynchronous messages for its communication. It’s just that there’s a synchronisation layer on top of those messages which makes it seem synchronous. But if the server doesn’t respond in time, the browser will still tell you that it didn’t receive a reply thus it has nothing to show. The browser is just delaying its task while waiting for a response.
This is something you can see more clearly when you open a page with multiple images. In general, the browser will ask for the page, which will result in a response. But the browser notices it needs to load several images now, so it starts requesting those images, and while it’s waiting to download those images, the user will just see the “broken image” icon but he’s still able to use it.
The same is true when you include JavaScript files or stylesheets in your web page. The browser downloads your page but will notice that it needs to download more files before it can render the page for the user to view. It will try to allow the user to do something as soon as possible, yet it has to wait for all those responses or just generate a time-out when it runs out of time.
Like the King and Queens who talk with one another through asynchronous messages, so does the Internet communicate between servers. Systems will send messages and then wait until they know they have received the data they need or a specific time limit has passed. Keep in mind that there’s nothing really synchronous about the Internet, except that most users will think it is a synchronous process.
Wim ten Brink 