How Yahoo has failed.

As many people have already read, Yahoo had a severe data leak in the past which resulted in ALL YAHOO ACCOUNTS being leaked to hackers. The hack includes sensitive personal information and includes an MD5 hash of the password you’ve used with Yahoo. This is a very serious issue as Yahoo has told me today in an email. It says:

Yahoo
UPDATED NOTICE OF DATA BREACH
Dear Yahoo User,
We are writing to update you about a data security issue Yahoo previously announced in December 2016. Yahoo already took certain actions in 2016, described below, to help secure your account in connection with this issue.
What Happened?On December 14, 2016, Yahoo announced that, based on its analysis of data files provided by law enforcement, the company believed that an unauthorized party stole data associated with certain user accounts in August 2013. Yahoo notified the users it had identified at that time as potentially affected. We recently obtained additional information and, after analyzing it with the assistance of outside forensic experts, we have determined that your user account information also was likely affected.
What Information Was Involved?

The stolen user account information may have included names, email addresses, telephone numbers, dates of birth, hashed passwords (using MD5) and, in some cases, encrypted or unencrypted security questions and answers. Not all of these data elements may have been present for your account. The investigation indicates that the information that was stolen did not include passwords in clear text, payment card data, or bank account information. Payment card data and bank account information are not stored in the system we believe was affected.
What We Are Doing

In connection with the December 2016 announcement, Yahoo took action to protect users (including you) beyond those identified at that time as potentially affected. Specifically:

  • Yahoo required potentially affected users to change their passwords.

  • Yahoo also required all other users who had not changed their passwords since the time of the theft to do so.

  • Yahoo invalidated unencrypted security questions and answers so they cannot be used to access an account.

We are closely coordinating with law enforcement on this matter, and continue to enhance our systems that detect and prevent unauthorized access to user accounts.

What You Can Do

While Yahoo already has taken action to help secure your account, we encourage you to consider the following account security recommendations:

  • Change your passwords and security questions and answers for any other accounts on which you used the same or similar information used for your Yahoo account.

  • Review your accounts for suspicious activity.

  • Be cautious of any unsolicited communications that ask for your personal information or refer you to a web page asking for personal information.

  • Avoid clicking on links or downloading attachments from suspicious emails.

Additionally, please consider using Yahoo Account Key, a simple authentication tool that eliminates the need to use a password on Yahoo altogether.
For More Information

For more information about this issue and our security resources, please visit the Yahoo 2013 Account Security Update FAQs page available at https://yahoo.com/security-update.

We value the trust our users place in us, and the security of our users remains a top priority.

Sincerely,
Chris Nims
Chief Information Security Officer

And yes, that’s bad… it’s even worse as the hack occurred in 2013 and it has taken Yahoo 4 years to confess everything about the hack. Well, everything? I’m still not sure if we’ve heard everything about this case. Worse, as Verizon recently took over Yahoo for a large sum of money, it could even have an impact for anyone using the Verizon services.

But there is more as people might not realise that the sites Tumblr and Flickr are also part of the Yahoo sites. We know that Yahoo is hacked but how about those other two sites? As I said, we might still not know everything…

Yahoo

About to drown by failing security.

Well, assume the worst. While we might be arming ourselves properly against any of these kinds of hacks, we also chain ourselves to the security provided by companies like Yahoo. And those security measures might not protect us against everything.

Fact is that Yahoo has become great and is becoming even bigger now they’re part of Verizon. As a result, all those 3 billion accounts are now owned by Verizon and we better hope that Verizon will use better security than Yahoo ever did. If not, anyone who ever used Yahoo, Tumblr, Flickr or Verizon might soon drown in security problems as their accounts have been hacked and they will continue to hack those.

Is there a solution to this problem? That’s a good question as there are many other companies that we rely upon for our security. Twitter, Google and Facebook are a few popular sites that are also popular targets for hackers. However, as long as these large corporations immediately notify all users if there’s a serious data breach and immediately respond by increasing security, the risks should be acceptable. What Yahoo did was wrong as it took 4 years before they finally admitted the truth!

So in my opinion, Yahoo has to disappear. It is unacceptable that any company with such a major role on the Internet regarding security is trying to hide the truth and keep people vulnerable instead of responding immediately. So instead of following Yahoo’s advise and change your password, I suggest everyone just close their Yahoo account. Permanently! You might still keep your Flickr and Tumblr account as those might not be involved in this hack but Yahoo should go.

And let’s hope that someone will improve the security on both Tumblr and Flickr as these services are highly popular all over the World.

The riddle of the Holy Men…

I generally stay far away from religion but I do like discussions about religion. And recently, I got involved in a discussion with some fanatic who “knew” he was right and I was wrong, as his Religious counselors have told him so. So I came up with a simple riddle which he could not answer.

It starts with three Holy Men and a large chest. And as I’m talking about religion in general, I make them of three different religions so we have a rabbi, a priest and an imam. But if you talking to a fanatic, make them all Holy Men for his specific religion.

Azra Yilmaz with Chest.png

The rabbi starts and opens the chest to look inside. Once he has seen what is inside, he closes the chest and tells you there’s a statue of a golden calf inside, with precious jewels for his eyes. He describes it in details and he sounds very convincing.

Next, the priest opens the chest and looks inside. After he has seen the content he closes it and tells you what is inside. It is a wooden cross with long, silver nails and a golden hammer. And he too describes it in a lot of details and sounds very convincing.

Last, the imam opens the chest and looks inside. He has seen the content and closes the chest and tells you what is inside. It is a marble statue of a horse with a crescent moon next to it made from pure silver. And it is described in details and also sounds very convincing.

Now, who do you believe? How do you know who is telling the truth?

It’s actually a simple riddle. But for some people, it is still too complex. When they’re from a christian background they are more likely to believe the priest. Jewish people would favor the rabbi. And muslims will believe the imam. But all we know is that at least two of these Holy Men are lying. And maybe all three of them are lying.

When you’re not biased by any religion, it becomes more challenging and you would try to find out how reliable these Holy Men are. You talk with them and look at their actions. Have they committed crimes in the past? Have they been caught lying in the past? And what are they describing exactly? And how about their family? Are they from families that have used lies and crimes in the past?

It takes some time to evaluate who is speaking the truth and as all three of them are very convincing you could just decide to believe all three. However, you know that two of them must have been lying, as not all three objects could have been in the chest.

Or could it? Could it be a magical chest which shows the content that the viewer wants to see? If so, all three of them would have spoken the truth but it requires the chest to be very special. And it makes the content of the chest questionable in value, as what worth does an illusion have?

Or you just don’t care about what’s in the chest. That’s actually the easiest solution. However, all three holy Men tell you that you have to believe one of them as your eternal soul would be in danger if you don’t. All three threaten you that you will go to Hell if you don’t believe in him or believe any of the other two. And that makes it even more challenging as you don’t want to spend your eternal life in Hell.

But if you don’t believe in a soul and eternal life then not even that matters. So the holy Men are telling you to believe them or else they will end your life. You have to believe one or choose death. And if you pick one, the other two will want to kill you, but the Holy Man whom you believe will try to protect you. With the threat of violence and danger for your eternal choice, you will be in big problems and will be forced to make a decision.

So, you wouldn’t make a choice on whom you believe, but you choose the one who is most beneficial for you and your family. You have no choice as you need protection and each Holy Man can provide protection. But when you chose one, the other two will become your enemies.

So you have a big dilemma. You are forced to choose one and will have to believe what he said about what’s in the chest. And this is the same problem as you see with religion. You have basically thousands of “holy men” claiming they, and only they, know the Truth and that you have to believe them, or else. And instead of having just two enemies you would end up with potentially thousands of enemies if you chose to believe one of them. They they will also be enemies if you don’t believe any of them. However, when you follow just one then he will likely protect you against the others.

Or not, as many holy men will actually use those who believe in him to convince them that the others are lying and thus deserve to die. They need to die because they don’t believe in what he is claiming. He knows the truth as he has looked into the chest. And you believe him because you think he’s trustworthy. Right?

Generations later, your children will continue to follow the children of this holy Man, as you’ve educated them to believe in what he told so they believe in what you told them. However, the Holy Man now has three children. And each child tells the story about the content of the chest differently again. One repeats what his grandfather has told. The second one adds a collection of gems and golden coins to the content. The third mentions flasks of wine as part of the content in the chest. And your children will have to make the same choice again as what you had to do. They have to solve the Riddle of the Holy Men.

And this explains why we now have so many different forms of religions. Why we have so many different opinions and don’t know what’s in the chest, as we don’t know whom we really can trust. This is a riddle that will repeat itself every generation, as new Holy Men will be born and each of them claim something different in the content of the chest. All these problems simply because we have to chose whom to believe.

Or do we? The solution of the riddle is actually quite simple. You yourself go to the chest and you yourself will open the chest and look inside. Then you will see what is inside. And when you want others to know what is inside, you will invite them to also take a look inside and keep the chest open. That way, every person can learn what is inside the chest without relying on what others claim they saw.

And that’s what’s science is! In science, you don’t present any beliefs but you show everyone the clear facts and allow them to evaluate the reality themselves by telling them what to do to see those facts. And you allow people to think for themselves and determine for themselves what they are seeing. You suggest theories and others might believe those theories or not. If they want, they can make up their own theories and that’s just fine. Everyone can see inside the chest and see the content.

So the answer of finding the truth is simple. You just have to look for yourself!

Nieuwe ABN-AMRO phishing email!

(Dutch warning about a phishing email targeting ABN-AMRO customers. As it targets Dutch people, I write it in Dutch. Sorry…)

2017-06-16.png

Vandaag weer een spam-bericht in mijn spambox ontvangen waarin men weer probeert om mensen op een link te laten klikken. Ik heb het maar meteen als “Phishing” aangemerkt maar het is een beetje onbegrijpelijk dat mensen hier soms toch intrappen want als je goed oplet zie je dat er niets van klopt!

Eerst en vooral komt de email binnen op een account die ik niet gebruik voor deze bank, hoewel ik er wel een account heb. Dit toont maar weer eens aan hoe praktisch het is om je eigen domeinnaam te hebben met een catch-all mailbox zodat je een oneindig aantal email adressen kunt aanmaken.

Andere waarschuwingen zijn de spaties in de datum, de titel “Trouwe Cliënt” en enkele andere taal- en stijlfouten in de tekst. Zo klinkt “betaal kaart” best raar als het om een betaalpas gaat. Duidelijk een gevalletje Google Translate.

Ook het verhaal erachter is vreemd want de bank heeft problemen in hun IT systemen en daardoor moet de klant opeens actie ondernemen? En zolang dat niet gebeurt is de account geblokkeerd?

Interessanter wordt het als je de bron van de email beter gaat controleren. De afzender maakt gebruik van een sub-domein van sodelor.eu en mogelijk is dit gehele domein een phishing-site. In ieder geval heeft het sub-domein een phishing pagina waarin het PayPal nabootst. Sowieso zou je PayPal als afzender verwachten, maar goed. Sommige mensen zijn idioten…

De email bevat ook een URL die verwijst naar een Russische website en dat verbaast mij niets. Russische domeinnamen worden vaak door hackers misbruikt omdat deze vaak eenvoudig te hacken zijn.

Als je verder de bron nakijkt zie je dat deze via de Duitse kundenserver.de worden verstuurd. Dit domein is ondertussen al op diverse blacklists geplaatst wegens de grote hoeveelheid spam die ermee wordt verzonden.

Maar goed, de meest duidelijke detectie dat dit spam is, is omdat het in mijn spam-folder zit.

Delicious spam!

Once more, a post about spam. Why? Because I have one more interesting email in my spam-box, sent by someone who clearly is confused by the whole topic. So, here’s the email, with some annotations:

spam-1486041897301

Why is it spam? Because Google Apps/GMail says it is. And google is often right in these things. And as I don’t know Adam Collier, nor see any name of his company, it clearly seems like spam to me too, from some wannabe web developer in India looking for customers without understanding the rules.

Why  from India? Well, the English writing is more British than American. The writing style is similar to how Indian spam is generally written, with only single-line paragraphs. The skill set used is also very common among Indian developers. The extreme politeness in the writing also is similar to what you see in mostly Asian countries, as people there are generally more polite. Then of course, it mentions India in the email too so that wasn’t difficult.

First of all, this email was sent from a genuine, free email address like those offered by Outlook, Gmail and Yahoo. I’m not going to say if it’s Outlook or not as I allow this guy some anonymity, even though his name is probably fake and the address already closed for sending spam. But for me that’s the first sign of spam. If it is sent from a free mail provider then you should make sure you know the sender before continuing! As usual, check the sender first for every email you receive!

Next is the address to where it was sent. While it seems to be my “info” account, it just isn’t! It was received by the account I used for my registrar and used in my domain registration where it is visible in the WhoIs information, including my name and some other details. The “info” address happens to be the address of some other website, who has also received this email. My address was actually part of the BCC header so other recipients would not see that I had received it. Smart, but it is to be expected from mass mailers as they would really piss off a lot of people if they only use the TO or CC fields, as many people tend to ‘Reply to all’ on spam messages, making even more spam.

So they got my address from the WhoIs database. So they should have known my name too! They just can’t use it as this is a mass email that’s probably sent to hundreds or even more people.As this spammer doesn’t seem to use any mass mailer application, I suspect that he just collected a lot of email addresses from interesting-looking domains and just mailed to them all from Outlook so the amount of recipients is likely to be hundreds, maybe thousands. Not the millions that more experienced spammers will use.

Interesting is how he’s called a webmanager in his email address while calling himself an online marketing manager in the email. No name for his business so maybe he doesn’t even have a real business. This could be a simple PHP developer who is trying to make a freelance web development business and is hoping to get some customers so he can expand his business. He might have a few friends who are also doing development and likely is a student at Computer Science classes in India who wants to put his lessons to the Test. This doesn’t look like a hardcore spammer, even though he is spamming. He’s more a lightweight spammer.

The prices he mentions are very reasonable. Then again, he basically uses standard frameworks like WordPress, Joomla, Magento and Drupal to build those sites which is generally not too much work. I call these “Do not expect too much from us” prices.

There is one major alert in all this, though. The grey line mentions a “Payment Gateway” which you should immediately distrust! Why? Because this developer is probably setting up this payment gateway and might have control over it later on. He could be siphoning off some of the payments made through it or even at one point empty all the money collected and put it in his own bank account! Good luck getting your money back!

Well, he could be honest but you should not take that risk to begin with…

It is interesting to see that he also provides Android and IOS applications. He seems to be specialized in PHP so he would need to know Swift or Objective-C to do the IOS development and Java for the Android development. Or have some other programming environment that allows him to develop for both platforms. He might be using Visual Studio with Xamarin which would allow him to focus on different platforms. Or he has friends who specialized in app development.

At the bottom of his email he tells you that this isn’t spam and that he actually hates spam. So if you aren’t interested you should just reply to him so he can confirm that your email address exists and is in use so he won’t be sending emails to it. Wait… Why does he need that? People who aren’t interested generally won’t respond! So he might actually be collecting confirmations for other purposes…

Anyways, it shows that many spammers are generally amateurs, not knowing what they’re doing. Some might work for some business and think they can promote it this way while others are just freelance developers trying to find a work in the current market. Both will generally learn that these kinds of emails are spam and generally end up being blacklisted or loose their free email account. The problem is not that they really want to spam people, but they are misguided in thinking that you can just send emails to everyone as part of their marketing strategy!

Unfortunately, it doesn’t work that way! If you send these kinds of messages unsolicited then you are spamming. If you seek new customers then you should start by registering your own domain name and provide proper information about yourself. Use your own domain name for sending emails and not some free provider and more important: use mailer software where people can subscribe and unsubscribe and only mail people who have subscribed! Also provide a simple web-based solution to unsubscribe as a link in your email. People might still consider it spam but at least the risks of being blacklisted becomes less as you’re conforming to the anti-spamming rules.

If you want to do proper business online then you need to be familiar with the rules. You should know about spam and how to avoid to becoming a spammer. You should have a clear profile of your business online, preferably under your own domain name. And you need to know about the legislations of the countries that you’re targeting like the cookie-laws and privacy laws in Europe. Thing is, if your site and services are targeting foreign nations then you are operating under their laws also! Never forget that!

Marianne In Office.png

And with that, this lesson ends…

Donald Trump is NOT my President!

Bianca Delmonde for Shapeways_0001.png

That’s because I’m Dutch and still rolling on the floor over this past election result from the USA, showing the utter madness that a Democracy can be, sometimes. Then again, the people in the USA didn’t have much to choose from, did they?

Many people were actually surprised that Trump got elected, which is strange as both the Democratic Party and the Republican Party are both about equally popular. The USA is basically a binary system, as there aren’t many alternative choices. Well, not voting is a choice, albeit a very bad one as you won’t get anything to say afterwards. But judging by the huge amounts of protests from the US population and all the bad things being said about Trump, it still is a bit of a surprise.

But again, binary system! The population of the USA is roughly divided into just two groups and many of these voters are unlikely to switch sides. There are, of course, many swing voters who can go either way but in general, both parties have their own loyal supporters. With no alternative choice than basically two bad candidates, many voters didn’t have any reason to switch sides, although Hillary Clinton did have a past history as First Lady and some Democrats might have wanted to keep her husband out of the White House.

So, today, Donald Trump will be crowned as the New President of the USA. So, utter chaos next?

No, I don’t think so. The Republican Party has been in power many times before and even had some very good Presidents in the past. Lincoln was a Republican. So was Eisenhower and Reagan. Reagan was an important factor in the end of the Cold War, even! Sure, he used to be an actor but in the 1984 elections, he pulverized his Democratic opponent Walter Mondale.

But this election was very close again. Clinton won the Popular Vote but in the USA that isn’t important. Trump just won the most electoral votes.

Then again, many people are also ignoring the fact that Faith Spotted Eagle also won one electoral vote during this election, thus becoming the first Native American to receive one such vote. Well, thanks to a faithless elector in Washington who was supposed to vote for Clinton…

Several other faithless electors decided not to vote for Clinton, which clearly tells me that she wasn’t a favorite among her own party. And that also explains why Trump got elected. Hillary Clinton just wasn’t the best candidate to pick for the Democrats.

First of all, Hillary Clinton would have been the first female POTUS if she was elected. But as the USA business world is still strongly male-oriented, I think her gender was already costing her some votes. She is also known as a former FLOTUS while her husband Bill was in office. And Bill Clinton had smoked a bad cigar while in office so it is understandable that some people would not like to see him return to the White House. Then there were some scandals about emails and an embassy in Libya and something involving the sexual abuse of minors and more fake news that put her in a bad spotlight, but that also happened to Donald Trump. Fact is, this election saw so much fake news that most people stopped believing all of it. They just relied on things they knew from the past.

I think Bernie Sanders would have been a much better choice anyway. Still, the Democrats choose Clinton and they’re allowed to make such mistakes. Considering that Hillary won the Popular Vote, it still wasn’t that bad.

So, what will happen next? Trump is a businessman and as such, he’s known to try and make companies more profitable. Sure, some of his companies went broke but overall, he has been reasonably successful with his businesses. Maybe with the bit of fraudulent support but still, he has collected quite a bit of wealth in his lifetime. Lost a lot too, though.

First of all, his tax plans don’t seem to be that bad. It seems that low-income Americans won’t even have to pay any taxes. And as he wants to reduce the number of brackets from 7 to 3, income tax should become a bit simpler for everyone.

He does want to repeal the Obamacare tax, though. Also called the Affordable Care Act, the Obamacare is considered extremely valuable for many Americans as it means that each and every one of them will at least have some minimum health care so people can get sick without going broke. But repealing the tax doesn’t repeal the act itself. In fact, Trump might find some other solution to fund this care, maybe change a thing or two, have it renamed to Trumpcare and POOF! Obamacare is gone! Yet the system would have barely changed and all Americans would still have basic health care.

Immigration is also a hot topic and Trump wants to build a wall to keep Mexicans out of the USA. And probably have Mexican companies doing most of the building of this wall as the Mexican government might want to subsidize the extra employment for their own population. A more strict policy on illegal immigrants might be useful as it could actually increase the wages of the legal immigrants! So, we’ll just have to wait and see what will happen…

Of course, Donald Trump has his site full with interesting plans that all sound quite nice. He’s a Master Merchant at this, knowing how to sell his ideas to the public, as he has done all his life with all his businesses. Trump is a Master at Selling and we can only hope that he will keep to his promises. But in the end, it isn’t Trump who is in Power. It is the Republican Party that is in Power and Trump is just their main spokesperson. He can’t do much that his party won’t approve, as they would just vote against him and probably even force him to step down, if need be. Barack Obama had similar problems as he had Great Plans, but could not get all of them executed as his own party and the Republican Party shot most plans down again. Donald Trump will have the same problem as the Democratic Party will try to resist any of his plans. So he needs the support of all the Republicans, else they just tip the balance against him.

The USA is just a binary system which makes it difficult to rule and have a lot of changes. Not many Presidents have managed to make a lot of changes to the whole system. Trump does have the advantage that the Senate is mostly Republican also so for a while, he will be able to execute some of his better plans. But in 2018 we will see the next Senate Election and as the Democrats won 3 seats in the past election, chances are that the Democrats will soon be in control over the Senate and thus stop whatever Trump has planned.

So Trump will have less than 2 years to prove his success as the next POTUS. And while many people hope he will fail, I realise that Trump failing as President will just bring more Chaos in the USA. So I’m expecting a few errors from Trump and a few successes and in 4 years, there will be new elections and by then, the Democrats will likely pick a better candidate.

Would be nice if Faith Spotted Eagle became the next POTUS, though. A Native American Female President! How cool would that be?

The Binary Search problem

Many developers will have to learn all kinds of algorithms in their lives so they can write highly optimized code. Many of these algorithms have long histories and are well-tested. And one of them is the binary search method.

The binary search is a fast algorithm to find a record in a sorted list of records. For most people, this is a very familiar algorithm if you had to ever guess a value between 1 and 10, or 1 and 100. The principle is quite simple. You have an x amount of records and you pick the record in the middle of the list. For guessing between 1 and 100, you would pick 50. (100/2) If it is the correct value, you’ve won. If it is too high, you now know the value must be between 1 and 50 so you guess again with value 25. Too low and you pick the value between 50 and 100, which would be 75. You should be able to guess the value in up to 8 tries for values between 1 and 100.

Actually, the binary search is actually the easiest explained as bit-wise checking of values. A single byte will go from 00000000 to 11111111 so basically all you do is a bitwise compare from the highest bit to the lowest. You start with value 10000000 (128) and if the value you search for is higher, you know that first bit is 1, else it needs to be 0.

Your second guess would either be 11000000 (192) or 01000000 (64) and you would continue testing bits until you’ve had all bits tested. However, your last test could also indicate that you guessed wrong so the maximum number of guesses would be equal to the number of bits plus one.

And that’s basically what a binary search is. But it tends to be slightly more complicated. You’re not comparing numbers from 0 to some maximum value but those numbers are generally a kind of index for an array, and you compare the value at the position in the array. You basically have a value X which could basically be any data type and even be a multi-field record and you have an array of records which has all data sorted for some specific index. And these arrays can be reasonably large. Still, the binary search will allow you some very quick search.

Now, the biggest problem with the binary search is how people will calculate the index value for the comparison. I already said that you could basically check the bits from high to low but most developers will use a formula like (floor+ceiling)/2 where floor would be the lowest index value and ceiling the highest index value. This can cause an interesting problem with several programming languages because there’s a risk of overflows when you do it like this!

So, overflow? Yes! If the index is an unsigned byte then it can only hold a value of 11111111 (255) as a maximum value. So as soon when you have a floor value of 10000000 (128) and a ceiling of at least (10000001) then the sum would require 9 bits. But bytes can’t contain 9 bits so an overflow occurs. And what happens next is difficult to predict.

For a signed byte it would be worse, since value 1000000 would be -128 so you would effectively have 7 bits to use. If the 8th bit is set, your index value becomes negative! This means that with a signed byte, your array could never be longer than 64 records, else this math will generate an overflow. (64+65 would be 129, which translates to -127 for signed bytes.)

Fortunately, most developers use integers as index, not bytes. They generally have arrays larger than 256 records anyways. So that reduces the risk of overflows. Still, integers use one bit for the sign and the other bits for the number. A 16-bit integer thus has 15 bits for the value. So an overflow can happen if the number of records has the highest bit value set, meaning any value of 16384 and over. If your array has more than 16384 records then the calculation (floor+ceiling)/2 will sometimes generate an overflow.

So, people solved this by changing the formula to floor+((ceiling-floor)/2) because ceiling-floor cannot cause an overflow. It does make the math slightly more complex but this is the formula that most people are mostly familiar with when doing a binary search!

Yet this formula makes no sense if you want a high performance! If you want a binary search, you should actually just toggle each bit for the index until you found the value. To do so, you need to know how many bits you need for the highest value. And that will also tell you how many guesses you will need, at most, to find the value. But this kind of bitwise math tends to be too complex for most people.

So, there is another solution. You can promote the index value to a bigger one. You could use a 32-bit value if the index is a 16-bit value. Thus, you could use (int16(((int32)floor+(int32)ceiling)/2) and the overflow is gone again. And for a 32-bit index you could promote the math to a 64-bit integer type and again avoid any overflows.

It is still less optimal than just toggling bits but the math still looks easy and you can explain why you’re promoting the values.

But what if the index is a 64-bit value? There are almost no 128-bit values in most programming languages. So how to avoid overflows in those languages?

Well, here’s another thing. As I said, the index value is part of an array. And this array is sorted and should not have any duplicate values. So if you have 200 records, you would also need 200 unique values, with each value being at least 1 byte in size. If the index is a 15-bit signed integer then the values in the array must also be at least 15-bits and would generally be longer. Most likely, it would contain pointers to records elsewhere in memory and pointers are generally 32-bits. (In the old MS-DOS era, pointers were 20 bits, so these systems could manage up to 1.048.576 bytes or 1 megabyte of memory.)

So, let’s do math! For an overflow to occur with an index as a signed 16-bit integer you would need to have at least 16384 records. Each record would then be at least 2 bytes in size, thus you would have at least 32 kilobytes of data to search through. Most likely even more, since the array is probably made up by pointers pointing to string values or whatever. But 21 KB would be the minimum to occur when using a 16-bit signed index.

So, a signed 32-bit index would at least have bit 30 set to 1 before an overflow can occur. It would also need to contain 32-bit values to make sure every value is unique so you would have 4 GB of data to search through. And yes, that is the minimum amount of data required before an overflow would occur. You would also need up to 31 comparisons to find the value you’re searching for, which is becoming a bit high already.

So, a signed 64-bit index would have records of at least 8 bytes in size! This requires 36.893.488.147.419.103.232 bytes of data! That’s 33.554.432 terabytes! 32.768 petabytes! 32 exabytes! That’s a huge number of data, twice the amount of data stored by Google! And you need more data than this to get an overflow. And basically, this is assuming that you’re just storing 64-bit integer values in the array but in general, the data stored will be more complex.

So, chances of overflows with a 32-bit index are rare and on 64-bit indices it would be very unlikely. The amount of data required would be huge. And once you’re dealing with this much data, you will have to consider alternate solutions instead.

The alternate solution would be hash tables. By using a hash function you could reduce any value to e.g. a 16-bit value. This would be the index of an array of pointers with a 16-bit index so it would be 256 KB for the whole array. And each record in this array could be pointing to a second, sorted array of records so you would have 65536 different sorted arrays and in each of them you could use a binary search for data. This would be ideal for huge amounts of data, although things can be optimized better to calculate to an even bigger hash-value. (E.g. 20 bits.)

The use of a hash table is quite easy. You calculate the hash over the value you’re searching for and then check the list at that specific address in your hash table. If it is empty then your value isn’t in the system. Otherwise, you have to search the list at that specific location. Especially if the hash formula is evenly distributing all possible values then a hash table will be extremely effective.

Which brings me to a clear point: the binary search isn’t really suitable for large amounts of data! First of all, your data needs to be sorted! And you need to maintain this sort order every time when you add or delete items to this record, or when you change the key value of a record! Hash tables are generally unsorted and have a better performance, especially with large amounts of data.

So, people who use a 32-bit index for a binary search are just bringing themselves in trouble if they fear any overflows. When they start using floor+((ceiling-floor)/2) for their math, they’re clearly showing that they just don’t understand the algorithm that well. The extra math will slow down the algorithm slightly while the risk of overflows should not exist. If it does exist with a 32-bit index then you’re already using the wrong algorithm to search for data. You’re at least maintaining an index of 4 GB in size, making it really difficult to insert new records. That is, if overflows can occur. The time needed to sort that much data is also quite a lot and again, far from optimal.

Thing is, developers often tend to use the wrong algorithms and often have the wrong fears. Whenever you use a specific algorithm you will have to consider all options. Are you using the right algorithm for the problem? Are you at risk of having overflows and underflows? How much data do you expect to handle? And what are the alternative options.

Finally, as I said, doing a binary search basically means toggling bits for the index. Instead of doing math to calculate the half value, you could instead just toggle bits from high to low. That way, you never even have a chance of overflows.

So, you want to be a software developer? Part 5.

In the first part I talked about resources. In the second post, I mentioned the need of logic and visualization. In the third post I talked about various platforms and the need to pick one to start with. In the fourth post I talked about modeling techniques, the development process and the need for documentation. And now I want to start talking about the first programming language that you should learn, plus the book that will teach you the basics.

This first language should be the Standard C language. Also known as ANSI C or ISO C, or just C. Developed by Dennis Ritchie, who died in 2011 but who also pioneered at developing a programming language meant to create operating systems. Or actually, just UNIX. And the reasoning behind this programming language was that this single language should be able to compile application for all operating systems and computer platforms and a piece of code should do exactly the same on all those platforms. And C supports so many processors and operating systems, making it one of the most important programming languages ever.

The book you will need is “The C Programming Language” by Brian Kernighan and Dennis Ritchie and is less than 300 pages with only 8 chapters and two appendices. And that’s another reason why you should start with C. This language is just plain and simple, yet contains the most important things you will need to learn about programming.

And remember that I have said that programming languages themselves are not important. Using the right technique is. You use languages like tools to solve a problem by making a product. This is also true for C. You should learn it but chances are that you won’t use this language that often. However, many principles in C are also common in other languages. And the C syntax has been borrowed by various other languages too so if you’re familiar with C, you will be able to handle quite a few other programming languages.

If you spend a day on each chapter and take two extra days for the appendices then you’d need 10 days to go through the whole book! In a normal work week, that would be just two weeks to learn a new programming language. The most complex part will actually be working with pointers and memory, which is just something you should know about before moving to other languages. You should understand the principles behind it.

Programming isn’t complex anyways. You’re generally just dealing with three things. You have basic statements, which e.g. assign a value to a variable or display a text on the screen. You have conditions where you will decide which step you will take next based on some value. And you have loops, which allows you to repeat a group of statements.

And there’s something that you can call a compound statement, which is basically a combination of statements, conditions and loops grouped together to create a “new” statement. In C, those are called functions but in other languages they’re also called procedures or methods. And you use these to add structure to your code.

And understanding structure is important in programming. Any data that your code will process will be in a specific data structure. And the code you write will also be in some structure. And even your computer in connection with other hardware will have a specific structure. And as a programmer you will have to use a lot of structures.

So, the C language is generally quite simple. But it is made more complex because the language allows programmers to create very badly structured code and with a lack of documentation, such code becomes extremely difficult to read. Still, C is extremely popular and it is likely that you will have to deal with projects that were developed in C.

For example, most operating systems. Linux, BSD, UNIX, OS X, IOS, Android and Microsoft Windows are all operating systems where the developers used C to write these. And a graphical library like OpenGL is also based on the C language. And these are all extremely complex projects but they have all histories that go back several decades. Even the old MS-DOS was a mixture of both C code and Assembly.

A game like NetHack is also written in C because it has a high performance and because it allows the game to run on basically any platform.

But I’m not telling you that you should learn C to use it, but to better focus on the basics of programming itself. Learn it, then focus om other programming languages, including the various Object Oriented programming languages.

After learning C you could consider C++, Java, C# or even languages like FreePascal, PHP, Python, Visual Basic and many others. But those languages all are way more complex than plain old Standard C. Especially the addition of OO in languages like C++ and Java will add quite a bit of extra complexity.

But C tends to scare off some people just because such complex projects have been created with it. And because some programmers have actually written cryptic C code to confuse others who try to read it. You can actually obfuscate your C code by removing spaces and line breaks and by using various shorthand notations and little tricks. There’s actually a competition to create obfuscated C code, held yearly since 1984!

Don’t be scared to learn C! Once you learn it you will realise that it’s actually quite simple. Don’t try to immediately write complex applications and especially don’t try to make complex GUI applications or 3D render engines with OpenGL because that is way too complex to start with. Start simple by creating console applications like Hangman, where you type letters to guess a word. Use a plain text as user interface since you will learn to make complex GUI applications in the future, once you’ve improved your programming skills! Focus on the C language first.

So, you want to be a software developer? Part 4.

So I’ve mentioned the various resources you can use. I’ve also said that you need to train to think more logical and to visualize your designs. And I’ve told you to pick a target platform for which you will be developing at first. (You can always pick more platforms later.) So if you’d expect that I would start focusing on programming languages then you’re wrong!

Even though I did mention C as a very practical language since it is supported by a lot of platforms, I will still have to focus on a very important process in software design, which is basically the whole development cycle.

As I said, I learned programming from my father. My father, who was a software developer for the ING, a large bank in the Netherlands, has had quite a few trainings in software development. This included the System Development Methodology that was developed by a company in the Netherlands called PANDATA. (Now known as CapGemini.) This was a very important development for managing large projects and focuses on planning and organisation of new projects.

With SDM, several phases are identified that can still be applied to modern development. While it originated as a Waterfall model because you would move from phase to face, yet the same model can also be applied to Agile software development but on a smaller scale. SDM isn’t just the process of turning an idea into a final product but also to adding changes to existing projects. And you can do this in a few big steps and thus have a waterfall, or you do this in a lot of small steps and thus be Agile.

So let’s look at the phases defined by SDM and how you would use them for your own projects.

IP: Information planning

In general, each new project of addition to an existing project starts with an idea that needs to be evaluated. For example, you want to create an online address book for your employees with information about valuable clients. There are plenty of projects that already do this but I just need an example.

You would have to start to look at what you want to solve by this project. How much will this project improve your business. Who will be using it and what does everyone expect to get out of this project?

These questions also occur in an Agile environment because you will often see new feature requests that you will have to consider if they should be implemented and what people would expect from this implementation.

In general, this is when you would also sign a contract with the client for whom you’re going to make this once both sides agree on what the idea will be. And no, this does not mean that you will set a deadline for when it will be finished but you are agreeing on the steps that need to be taken to turn the idea into a final product. As a result, you will need some good documentations about this idea before you continue with it.

Thing is, once documented you will have to stick to whatever you’ve agreed with the customer unless things are impossible to implement. Once the project is done, it should match this documentation. But you are allowed to change the documentation if your client agrees to those changes!

DS: Definition study

Once you have an idea of what everyone expects, you could start looking at the needs to implement this. Do you need extra resources for data storage? Do you need extra security? Are there any possible legal issues with storing data of clients in your database? (Some countries do have strict privacy laws!) Which techniques will have to be used and how important will this be?

In an Agile environment, this would basically mean that you create a new work item into the system which will be split in smaller steps in the next phases. You will have to check if you have the required resources to create and publish the project. For example, your customer might have a Linux web server while you use Windows to develop your PHP code. This means that you need at least one Linux environment similar to what the customer uses for testing purposes and additional experiments. That would be a resource that you need to get.

BD: Basic design

During this phase you will consider how the project will look like when it is functional. Basically, this means creating a functional design with images of what the user interface will look like. You will also need to consider how the various parts of your project will work together without being too detailed.

This doesn’t need to take much time, but you need your customer to agree upon how things will look. For one project I’ve worked on, the customer wanted an export button as part of a desktop application that would send a selection of data as XML to a web service. So I used Microsoft Paint to just draw an extra button on a screenshot and mailed this to the customer together with a short description of the data fields that we would sent. It can be that simple and often people don’t even notice this phase during the design.

But in general, you will have to discuss the look & feel of the project with your customer. Even when you’re using Agile methods, your client still needs to be informed about how it would look. Thing is, if the customer disagrees with the new GUI layout, you’d better know this before you’ve written all the code than afterwards.

These functional designs and screenshots would become part of the child work items for your project. In an agile environment, this could mean that you have a dozen or more small designs that are put on the backlog to be picked up.

DD: Detailed design

The next step is going into more details. Here you would think about the database structure, generic architecture of the project, additional libraries that you might need and basically all technical requirements that you will need for each work item from the previous phase.

In my case with the export button, I needed to know which web service I needed to call and what data structure it needed for me to send data to it. I also needed the data model of the application itself so I could make a plan for collecting the data and sending it to the web service.

It is likely that you will have multiple child work items for each work item from the BS phase. For this button I would have a work item for adding the button to the form, a work item to get access to the web service and a work item from mapping the data from the database to whatever the web service needs.

I could also have added just one child work item but it is good to be detailed in the things you do, so you can check if you haven’t forgotten anything. And even though these are all small tasks to do, it is useful to know which part will get you stuck for whatever reason. In this case, it could be that the web service that I try to connect to has problems. When others are examining the work items, it is much clearer when the item “Connect to web service” is still open than if “Add button” is still open. With the latter, it would be more unclear why things got stuck.

R: Realization

This is when you do the actual coding! You create the database, create all the forms, add all the logic to connect it all and you’re basically going through all work items in the system until the root work item is telling you that everything is done.

In a waterfall environment, this often means that you might have a few hundreds of pages with text, containing both functional and technical designs. In an Agile environment it is more likely that you’re just dealing with a few hundred work items that all need to be solved, although even Agile can create large documents for any new project.

But if you’ve done the Agile stuff properly, those documents should have been already translated into many small work items so the progress of the project can be measured more exactly.

I: Implementation

Once the coding is done, the project should be finished but in general it will have a lot of flaws and issues that are not conform the agreements with the customer. These could be critical errors like the system losing data when you click a specific button to trivial ones like two buttons which aren’t aligned properly.

I once had a customer who disliked the fact that the [OK] and [Cancel] button were the same size on a form, even though both words have various lengths. He didn’t like the white space on the [OK] button and it took some time to convince the customer to still agree with two buttons of the same size.

Another customer needed to be convinced that Comic Sans is not a good font for a serious desktop application, no matter how much he liked it.

But testing of your project is extremely important and this is where Agile methods have an advantage! When you’re doing Agile development then you can build weekly or even daily builds of the latest source code so the testers can start testing all things that have been implemented and the customer can take a peek at how far the current progress already is and provide feedback, if need be.

This feedback would generally mean going a few phases back for the current work item and would generally not have a huge impact on the other items. So in an Agile environment all phases tend to be mixed together, since the customer might already be making change requests while the testers are testing the project and you yourself would still be writing code.

In a Waterfall, the testers would need to wait until all your work is done. And feedback from the customer generally has to wait until even the testers are done. The waterfall has a clearer division of the various phases but tends to add more work when the design needs to change during development, simply because you have to go back one or more phases.

O&S: Operation and Support

Once the project is done and well-tested it is time to deliver it to the customer. Web applications are stored on a web server and desktop applications are packaged in a special installer. App Store applications are sent to the proper App Store and for mainframe applications you would generally send some developer over to the mainframe with a floppy disk or USB drive containing the source code to compile the project on the mainframe itself!

And with the Arduino, you’d upload the application onto the processor, add the additional electronics and case and ship it to the customer.

So, what would you need?

What you need is creativity to come up with new ideas for interesting projects. You need to be able to describe them and you need to be able to draw images of the design. You might even need to draw database models and programming structures and more. Knowledge of UML would provide a bonus but you have to be careful with such modeling principles.

Making models and documentation doesn’t need to be perfect! You can misuse UML any way you like for as long as you can get the idea across to your team members, your customer and the users.

UML is like a programming language: just a tool to change an idea into a product. It just isn’t that important although it is very useful.

Personally, I prefer to take pen and paper and just draw my designs freestyle. It saves me the time to open up some drawing application and then messing around to make a proper design. If I’m on the computer I could just as easily use my development environment to create the forms and GUI parts and thus finish the job faster. But if I do that, I would already be in the Realization phase while the previous phases aren’t finished yet. It tends to lead to more mistakes in the final product.

Besides, I have a scanner. Once my drawing is done, I use my scanner to scan it and if need be, I would add more texts and other stuff using Paint. But by visualizing the design and drawing it on paper you should get some very good designs to discuss with your customer. And customers don’t mind if you’re just showing them some hand-made drawing, as long as you can bring the concept across!

Thing is, while UML is praised all over in the Academic World, it is generally not used in the industry. And if it is used in any way, it is generally used in informal ways with ad hoc sketches. Rarely does anyone in the industry make complete designs in UML for their projects, simply because it is complex, gets out of sync with projects real fast and generally doesn’t provide that many advantages compared to the effort it takes to create these diagrams.

As a developer, you will need to learn to read documentation and to follow these instructions very strictly, unless there’s a good reason why something would not be possible. If something is impossible, you should provide feedback to the management and/or the customer explaining why something cannot be done. This, as soon as possible so there is plenty of time to change the designs.

As a developer you should also be able to write your own documentation and it should not surprise you if you’re spending more time on documenting your work than writing actual code! Thing is, most code is not as self-explanatory as most people expect it would be. Even if you include comments in your code to explain how everything works, things could still be unclear to other developers or even to yourself if you review your own code one year from now! Your code and your documentation need to be synchronized so people can check if your project does exactly what you’ve described. This allows better testing and for the customer it will be clearer to know what he agreed upon.

So make sure that you know how to document your projects properly!

As a good example of why this is so important there’s one situation that I resolved that succeeded because of me properly writing the documentation! A new feature needed to be added to a desktop application and a colleague of mine was given this task to implement this and 5 weeks to finish it. And this period included Christmas and New Year. He started working on it and started writing code, experimenting and asking advice but did not manage to get any idea of how to solve it and out of frustration, he’d just quit his job! Poof! Gone…

So, with three weeks to go and me having two weeks off for the holidays, it became my task to take this over. It had already been promised to a customer to be finished in the first week of the new year so failure wasn’t really an option. So I started working in my weekend to solve the problem. This meant reading what was required and coming up with a solution that could be implemented in just two weeks. Fortunately, there were two other developers who could help me to implement it all, but we first needed a good plan.

So, on saturday I started reading and making notes. I stayed away from my computer and only used print-outs of the functional design and existing code. It took most of the day but I did end up with some good ideas.

On sunday, I started up my computer and started writing a Word document with lots of details about my plan that would solve it. I added various diagrams and defined specific tasks that needed to be done for the project to finish. I also kept in mind the experience of the other two developers and checked which tasks they could do and which tasks I would have to do. And on sunday evening I printed several copies of this document and also emailed it to my colleagues.

On monday, I discussed my plan with my colleagues and made clear that everyone knew what their part in the design was. Even management had a copy of the design so everyone could take part in this discussion to make sure that my plan would provide what we needed. And once everyone agreed, we started coding.

On friday, we already had a good working version. A week afterwards, we had a well-tested version with some minor issues. And when we were supposed to be finished I was already enjoying my holidays and my colleagues had done the last finishing touches and the whole thing was delivered just in time. And it worked great, even though it had been a lot of work with all kinds of pitfalls.

But because I wrote a detailed document explaining what needed to be done, we could make a success out of something that appeared to become a disaster. And that’s why having proper documentation is so important.

I also have tales telling how not documenting your work can lead to failure. I’ve seen plenty of developers with whom I worked who just skipped making any documentation of the work they did. In one case, a colleague had written a very complex framework but without any proper documentation there was only one person who could use it. Which is very annoying when you have a team of 10 or more developers who all need to use that framework!

It also didn’t help that this framework had all kinds of design flaws which this developer would need to solve. Which he did, but for the others it was still hard to use this framework because some functionality did not work as expected and some weird dependencies were all over the code.

If there had been any documentation then everyone could compare the framework with the documentation and just read the documentation to find possible flaws, or to point out flaws in the code when the code did not do what the documentation told it would. The lack of documentation causes a lack of quality and made things harder for all developers involved. Unfortunately, this is how many developers like to work because they focus on the tools, not on the final product.

Which is why you need to learn hot to plan, design and document your projects even before you learn how to program!

In the next post, I will finally start talking about the programming language that every developer needs to know: Standard C.

So, you want to be a software developer? Part 3.

By now, you should know about useful resources where you can find information you need. And you have trained yourself to think logically and to visualize your ideas. So, should you start picking a programming language now?

Ramona on Computer 3_0001.png

Nope! It is still a bit too soon, although it doesn’t hurt to look around a bit. No, you need to build up your knowledge first and decide on which platform you would actually want to start developing.

Mainframes

To begin, you could decide to start developing on mainframe systems. Mainframes have been in use by large organizations and often require specialized personnel to operate them. Learning to use these systems is difficult since you can’t easily buy one to train at home. These are the Behemoths of computing, the Goliaths in a world full of dwarves. And some of them are quite antique and have already been in use for decades.

This is also where computing meets archeology since mainframes need to be extremely strict in backward compatibility. The systems of today might still be running software that was created 30 years ago.

Modern mainframes can run multiple operating systems using virtualization so it is possible to install Windows or Linux inside such virtual machine. But there are several operating systems that have been created to run on mainframes and most of them are very similar to Unix. IBM actually supports Linux on their mainframes. So learning to work with mainframes isn’t that difficult anymore. But in this field, you need to have good skills to manage a mainframe system.

Mainframes are still very popular in large financial organisations like banks because they’re made to handle huge amounts of data.

Supercomputers

A second group of computers would be the Supercomputers. Here too we would be dealing with specialized hardware that require well-trained experts to handle it. With modern supercomputers you’re actually dealing with a machine that contains thousands of processors that will run instructions parallel to speed up their calculations. These machines are created to make a huge amount of calculations, where mainframes were made to handle huge amounts of data.

Supercomputers are especially useful in environments where you need to make complex calculations. For example, for weather forecasts and molecular modeling, a supercomputer happens to be an excellent tool.

They also make extremely powerful chess engines!

Supercomputers used to have proprietary operating systems but are also moving towards using Linux. Since Linux is an open-source operating system, it has become a very popular choice too.

Home Computers and Apple

About four decades ago, most computers were still mainframes or supercomputers. Having a computer at home was not considered very useful since computers were quite difficult to use and needed experts to handle them. So, not many people considered the option to start building cheaper models for people to use at home, except for a few hobbyists who liked to tinker with electronics.

When Apple was founded by Jobs, Wozniak and Wayne in 1976, no one really expected that the computer they had designed would become so popular. And while the first Apple I kits were still hand-build by Wozniak, it didn’t take long before this inexpensive machine became popular and encouraged other companies to also design similar, cheap computers. Brands like Commodore, Atari and Sinclair suddenly became well-known names even though these systems were far less powerful than the average smartwatch of today. There was a rise in people who started buying computers to use at home, for games and educational purposes. And the first computer magazines started to be sold in bookstores and various other places. And some of these systems also found a nice place in various offices, automating all kinds of things people needed to automate, like their bookkeeping and letter-writing.

Home computers are rare these days and Apple has turned into a variant of the personal computer and into some smart devices.

Personal computers

The Personal Computer started about 3 decades ago as the answer of IBM to the popularity of the Apple computers and the many Home Computers. In 1981 the first IBM PC was created but it took a few years before it started getting momentum on the computer market.

IBM mostly targeted small offices but also tried to get access to the Home Computer market, at which they succeeded. Of course, it did help that many other manufacturers started creating machines that were compatible with the IBM PC and that IBM allowed these manufacturers to do so. Apple did not allow any copycats and their prices were much higher than these new PC’s, thus Apple lost a huge market share in the end.

The Home computer did compete against the PC too, but with 640 KB of RAM versus the 64 or 128 KB that most Home Computers had, it was clear that the PC had the upper hand. Although Atari did last quite long against the PC but that was also because Atari had become popular as a video editing computer. Apple had more specialized in desktop publishing. The PC just became a more generic device.

The PC is still used today and most developers will develop software for the PC. The modern PC can also be used as a server for web pages, databases or files and is thus a very versatile device. Today’s choices in operating systems for the PC would be Windows or Linux although there are a few other alternatives. (FreeBSD and OpenSolaris, for example.)

The PC is an inexpensive yet very powerful device that has many uses. This is definitely something to learn more about.

Laptops, Netbooks, Apple and servers

A simple PC can be set up to serve as a server. With either Windows Server software or Linux, you could turn a PC into a web server to host a web site. Or change it into a database server so your data gets stored in a very optimized way. Or use it as a file server, although people tend to use more specialized NAS disks for this.

Apple has also seen the advantage that the PC has and is trying to become as versatile as the PC is. But with Apple you would limit yourself to their own operating system while the PC can use various different ones. Still, it is possible to install a different operating system on your Apple computer but this isn’t a popular option since Apple computers are generally more expensive than PC’s.

Laptops and Netbooks are limited versions of the PC. In general, a PC could be used 24/7/365 to run continuously without a keyboard, mouse or monitor. I myself am using such a PC as my web server! But laptops and netbooks have been created to serve a user and use batteries so the user does not need to be connected to some wall through wires.

Laptops are excellent devices if you want to develop software and don’t have a fixed desk to work from. You can take them with you wherever you go and thus you could write code while you’re in the bus or train. Well, the lack of Internet might be a problem but they make good tools for travelling developers.

Netbooks are generally smaller versions of laptops. They’re often not powerful enough for the bigger development systems but they can still be useful for maintaining existing code.

You would generally not use servers to develop software but they are very useful for test environments and to allow distribution of your software projects. They can also be used for your version control system that you use to keep track of all changes that you’re making in your code.

And Apple systems are also very useful if you’re developing for IOS devices or are doing visual designs and graphics. Apple has always been popular amongst graphic designers and for desktop publishing.

Tablets, mobile phones and Smartwatches

Today, technology is becoming smaller and smaller. The result is that you can now wear a device on your wrist that is more powerful than the PC was 20 years ago! These devices aren’t generally useful to use for development but they are a good platform to target for your own development.

The main choices you have here are Android and IOS, where IOS tends to be slightly more secure but where it is also harder to get your apps published. There’s also Windows Mobile, the Blackberry and various proprietary systems but they’re generally not that popular.

When developing for these devices you will have to keep all the limitations and extra’s in mind. You are dealing with a device that has limited resources but also a large amount of functions. Things like WiFi, GPS, Bluetooth and often two build-in camera’s are very common in these devices. And you can actually make phone calls with these things too in many cases! Thus it is a very interesting target for developers, although you need a lot of customers if you want to make profits since the prices for apps are generally quite low and the App Market also takes a percentage.

If you fail to make a good app then developing for these devices isn’t very effective.

Raspberry Pi

The Raspberry Pi is technically just a very small PC that can run Linux. The modern version can even run a special version of Windows 10.

The Raspberry Pi is an interesting device if you want to do some hardware prototyping. It is also popular for various home automation projects although it is mostly used by hobbyists.

With Python or C++ (or other programming languages) you can easily write applications that listen to WiFi for incoming commands to turn off the lights, turn on the coffee maker and bread machine and manage all other electronics in your house. You can also connect a hard disk to the USB port and turn it into a simple file server. Or even a simple web server.

Commercially, there isn’t that much use for these devices except for prototyping. Still, the low cost of the Raspberry Pi does open up various uses, even for commercial purposes. It can be used for 3D printers, for example. It has been used with a simple keyboard and monitor to make cheap computers. And there’s a large commercial market for add-ons to the Raspberry Pi, including add-ons for commercial usage.

It might not take long for the Raspberry Pi to find a place in the Commercial World.

Embedded software, Atmel and the Arduino

Currently, the Atmel ATTiny85 happens to be my favorite platform as target for my projects. Basically, you are actually writing applications at the processor-level and not for a whole machine. This is basically when you start working with Arduino boards and similar things.

There is a whole market of programmable processors with additional programming hardware and generally a simplified C compiler so you can write code that gets stored directly inside a processor. This processor would then become part of your own electronic soldering and most likely on a board of your own design and will do basically anything that you have programmed.

These processors are very popular to create tiny robots, to drive LED lights and sound devices and much, much more. This is also a very challenging thing to do because you’re not just programming a processor but you also have to build the whole machine around it.

You need knowledge of electronics and preferably you have the tools and knowledge to make containers for your electronic experiments. Having your own 3D printer would be very useful but then you’d need more skills in 3D graphic design. And learning to work with electronics isn’t easy too, since it differs a lot from programming. It involves a lot of math since you have to make calculations for voltage and current in your project.

These calculations for electronics can be simple. Say, you have a 5 volt battery, a white LED light and some wire. You can connect this but chances are that the LED will burn out because it gets too much current so you have to add a resistor. But to know which resistor you need, you need to know the formula and some other information to pick the right resistor. Well, you have your resources so I have no doubt that you would be able to find that answer.

But imagine how complex it becomes when you have multiple LEDs, a speaker and some other electronic components!

So, make your choice!

So I’ve mentioned several targets that developers can pick for their development. When I worked at IBM, mainframes were my target. Before, it was the Home Computer and afterwards it became the PC. I’m also familiar with Apple, laptops, netbooks, mobile devices and tablets, the Raspberry Pi and a lot of Arduino stuff, including the programming of a plain ATTiny85 processor. (And soon the raw ATMega328 processor.) I’ve seen a lot of it and in my experience, you have to pick one environment first and become familiar with it.

In operating systems, you best start with both Linux and Windows. Linux has more environments where it can be used, including mainframes and small devices like the Raspberry Pi. And technically, Android is a Linux distribution too. So, becoming familiar with Linux is important.

However, most offices and companies use Windows for their computers and Windows is also popular as web server. You should make sure you are familiar with Windows too.

If you want to develop for Apple devices then you will also need to add OS X and IOS to your knowledge base, although OS X is similar to Unix, and thus Linux.

So, it would be best to learn to use both Windows and Linux with a focus on the command line shells, the installation and uninstaling of software and the installation and upgrading of both operating systems. The most practical solution would be to install Windows on your PC and then use a virtual machine to run Linux within your Windows environment.

The biggest problem with Linux are actually the various distributions that are out there. The use of virtual machines would allow you to experiment with multiple distributions on a single PC.

But you will also have to pick some hardware as your development target! And mainframes and supercomputers will be difficult targets for someone with no experience. You need proper training before they would let you work with those things!

So, targets are the desktop computer, meaning building desktop applications. Or the web server with web applications. Or databases on a database server. Or smart devices like tablets and mobile phones. And if you know about electronics you can consider to do prototyping with the Arduino or Raspberry Pi to create a new device to put on the market!

If you try to target all this hardware at the same time, you will only make things very hard for yourself. However, there is a programming language that should be supported by all these hardware systems and whatever operating system they’re using. And that language is called ANSI C. Or ISO C. Or Standard C.

But more about C in a next post! First, you need to pick a target and become familiar with it! Build up your knowledge of your target.

So, you want to be a software developer? Part 2.

Here’s part two about becoming a software developer. In the first part I told you about useful resources that will help you learn and once you’re working as a developer, those same resources will help you finish your projects.

Ramona on Computer 2_0001.png

By now, you should also have a few basic computer skills. Yet if you expect me to start talking about programming languages then no, not in this part. Because, as I said before, programming languages are not important. They are just tools and a good developer can use any tools he likes, even though it sometimes takes a short moment to get accustomed to the new tools.

To become a developer, you need to understand logic and you need to be able to visualize what your project will look like. And unfortunately, not everyone is born with these two abilities, although you can train to improve both abilities.

Having a mathematical background will help, since Math requires good logic. But there are other ways to train yourself.

To train your logic I would advise to learn to play Chess, Checkers or Go. Just these simple board games that require you to think ahead and outsmart your opponent. Especially Chess is a good game because you have various different pieces, each with different values, that you have to deploy over the ‘battlefield’ of 8×8 squares to capture the King of your opponent. Shredder Chess is a good chess engine that you can play online for free or on your mobile phone or tablet. And the various pieces will force you to think more strategically.

And although a game like Stratego and other games also requires strategic insights, the advantage of Chess is that the game starts with no hidden secrets for both players. Each player knows where the pieces of the opponents are and there’s no luck involved. Then again, if your opponent makes a mistake, you could consider that lucky, but it’s just a mistake by your opponent.

Learning Chess means that you have to think ahead in the game. Each move needs to be calculated. How will your opponent respond? And what will you do next. Good chess players can see a dozen or more moves ahead in the game, calculating which moves will win or lose. And to win, you must think further ahead than your opponent.

This is why training against a computer is a good way to improve your brains. Chess engines will use databases for the opening and the end game which will allow them to play strong, but in-between they have to think just like humans, and they generally calculate just up to a few moves ahead. It should not be difficult to think further ahead of the lower levels. And to improve, you should start playing against higher and higher levels until you win about half the games you play.

Next to learning strategy, logic and thinking ahead, you need to learn to visualize. For this, the best thing to do is to read. And since programming requires a good knowledge of the English language I would advise you to read a lot of English Science/Fiction literature. The stories of Conan the Barbarian, Tarzan or even the Dragonlance series are very good reading material. But if you don’t like this genre, start reading other stories where you can imagine how the main character is walking around in the environment set inside the book. Turn the words into a movie inside your head, visualize what is going on.

If that is a bit difficult, then read a book that has also been made into a movie. Movies like “the Hobbit” and “Lord of the Rings” were made on books written by Tolkien and make very good reading material. The movie follows the books reasonably close too, so visualizing it becomes easier. Other books would be the series “A song of Ice and Fire” which is used for “Game of Thrones” or “the Southern Vampire Mysteries” which resulted in the series “True Blood“. The movies should help you to visualize these worlds that are described inside the books.

Another way to learn how to visualize is through role-playing games and preferably those tabletop games with dice. Dungeons & Dragons is such a game but GURPS is my favorite. GURPS is less about superpowers and more adjustable to different worlds and settings. In GURPS, you can play a time-traveller who goes back to WWII with the mission to take down the Nazi’s in 1940 to prevent some disaster in the future. Or you are a time-travelling wildlife photographer being sent to the Age of Dinosaurs, 65 million years ago, filming how these dinosaurs came to their ends. Or maybe you’re just a Civil War-era cowboy fighting some Alien invaders. Then again, in GURPS you can also just be a dungeon crawler, bashing monsters and gathering loot.

Thing is, you need things that will require you to visualize what things will look like inside your head. This is important, since you will be working on various projects in the future and you will need to think ahead to know what it will look like once it is done!

As a developer, you have to understand the whole process starting with a good idea that you want to work out to finishing it completely and providing it to those who can use it. If you’re a carpenter, you might decide to build some chairs and a table and visualize how it should look like while the wood is still part of a tree. Once you are done, it is likely that it won’t look like what you visualized at the start, but that’s basically because you will keep changing your mind during the whole process.

However, if you made this for a customer then it should closely match what you agreed upon with this customer, since he expects that you deliver what you have suggested to him. You might have made drawings and descriptions so your customer will expect your project to match this. Then again, even clients can change their minds and together you could agree on changing some of the details. But in general, you start each process visualizing what you are going to make and think ahead about all the steps that need to be taken to turn the idea into a final product.

Without visualization, the whole process of building your project will be awfully slow and full with challenges. (Developers don’t talk about problems! We call it challenges, since problems are things that need to be solved and solving problems is a challenge!)

So, learn to use logic, learn to visualize things.