So I’ve mentioned the various resources you can use. I’ve also said that you need to train to think more logical and to visualize your designs. And I’ve told you to pick a target platform for which you will be developing at first. (You can always pick more platforms later.) So if you’d expect that I would start focusing on programming languages then you’re wrong!

Even though I did mention C as a very practical language since it is supported by a lot of platforms, I will still have to focus on a very important process in software design, which is basically the whole development cycle.

As I said, I learned programming from my father. My father, who was a software developer for the ING, a large bank in the Netherlands, has had quite a few trainings in software development. This included the System Development Methodology that was developed by a company in the Netherlands called PANDATA. (Now known as CapGemini.) This was a very important development for managing large projects and focuses on planning and organisation of new projects.

With SDM, several phases are identified that can still be applied to modern development. While it originated as a Waterfall model because you would move from phase to face, yet the same model can also be applied to Agile software development but on a smaller scale. SDM isn’t just the process of turning an idea into a final product but also to adding changes to existing projects. And you can do this in a few big steps and thus have a waterfall, or you do this in a lot of small steps and thus be Agile.

So let’s look at the phases defined by SDM and how you would use them for your own projects.

IP: Information planning

In general, each new project of addition to an existing project starts with an idea that needs to be evaluated. For example, you want to create an online address book for your employees with information about valuable clients. There are plenty of projects that already do this but I just need an example.

You would have to start to look at what you want to solve by this project. How much will this project improve your business. Who will be using it and what does everyone expect to get out of this project?

These questions also occur in an Agile environment because you will often see new feature requests that you will have to consider if they should be implemented and what people would expect from this implementation.

In general, this is when you would also sign a contract with the client for whom you’re going to make this once both sides agree on what the idea will be. And no, this does not mean that you will set a deadline for when it will be finished but you are agreeing on the steps that need to be taken to turn the idea into a final product. As a result, you will need some good documentations about this idea before you continue with it.

Thing is, once documented you will have to stick to whatever you’ve agreed with the customer unless things are impossible to implement. Once the project is done, it should match this documentation. But you are allowed to change the documentation if your client agrees to those changes!

DS: Definition study

Once you have an idea of what everyone expects, you could start looking at the needs to implement this. Do you need extra resources for data storage? Do you need extra security? Are there any possible legal issues with storing data of clients in your database? (Some countries do have strict privacy laws!) Which techniques will have to be used and how important will this be?

In an Agile environment, this would basically mean that you create a new work item into the system which will be split in smaller steps in the next phases. You will have to check if you have the required resources to create and publish the project. For example, your customer might have a Linux web server while you use Windows to develop your PHP code. This means that you need at least one Linux environment similar to what the customer uses for testing purposes and additional experiments. That would be a resource that you need to get.

BD: Basic design

During this phase you will consider how the project will look like when it is functional. Basically, this means creating a functional design with images of what the user interface will look like. You will also need to consider how the various parts of your project will work together without being too detailed.

This doesn’t need to take much time, but you need your customer to agree upon how things will look. For one project I’ve worked on, the customer wanted an export button as part of a desktop application that would send a selection of data as XML to a web service. So I used Microsoft Paint to just draw an extra button on a screenshot and mailed this to the customer together with a short description of the data fields that we would sent. It can be that simple and often people don’t even notice this phase during the design.

But in general, you will have to discuss the look & feel of the project with your customer. Even when you’re using Agile methods, your client still needs to be informed about how it would look. Thing is, if the customer disagrees with the new GUI layout, you’d better know this before you’ve written all the code than afterwards.

These functional designs and screenshots would become part of the child work items for your project. In an agile environment, this could mean that you have a dozen or more small designs that are put on the backlog to be picked up.

DD: Detailed design

The next step is going into more details. Here you would think about the database structure, generic architecture of the project, additional libraries that you might need and basically all technical requirements that you will need for each work item from the previous phase.

In my case with the export button, I needed to know which web service I needed to call and what data structure it needed for me to send data to it. I also needed the data model of the application itself so I could make a plan for collecting the data and sending it to the web service.

It is likely that you will have multiple child work items for each work item from the BS phase. For this button I would have a work item for adding the button to the form, a work item to get access to the web service and a work item from mapping the data from the database to whatever the web service needs.

I could also have added just one child work item but it is good to be detailed in the things you do, so you can check if you haven’t forgotten anything. And even though these are all small tasks to do, it is useful to know which part will get you stuck for whatever reason. In this case, it could be that the web service that I try to connect to has problems. When others are examining the work items, it is much clearer when the item “Connect to web service” is still open than if “Add button” is still open. With the latter, it would be more unclear why things got stuck.

R: Realization

This is when you do the actual coding! You create the database, create all the forms, add all the logic to connect it all and you’re basically going through all work items in the system until the root work item is telling you that everything is done.

In a waterfall environment, this often means that you might have a few hundreds of pages with text, containing both functional and technical designs. In an Agile environment it is more likely that you’re just dealing with a few hundred work items that all need to be solved, although even Agile can create large documents for any new project.

But if you’ve done the Agile stuff properly, those documents should have been already translated into many small work items so the progress of the project can be measured more exactly.

I: Implementation

Once the coding is done, the project should be finished but in general it will have a lot of flaws and issues that are not conform the agreements with the customer. These could be critical errors like the system losing data when you click a specific button to trivial ones like two buttons which aren’t aligned properly.

I once had a customer who disliked the fact that the [OK] and [Cancel] button were the same size on a form, even though both words have various lengths. He didn’t like the white space on the [OK] button and it took some time to convince the customer to still agree with two buttons of the same size.

Another customer needed to be convinced that Comic Sans is not a good font for a serious desktop application, no matter how much he liked it.

But testing of your project is extremely important and this is where Agile methods have an advantage! When you’re doing Agile development then you can build weekly or even daily builds of the latest source code so the testers can start testing all things that have been implemented and the customer can take a peek at how far the current progress already is and provide feedback, if need be.

This feedback would generally mean going a few phases back for the current work item and would generally not have a huge impact on the other items. So in an Agile environment all phases tend to be mixed together, since the customer might already be making change requests while the testers are testing the project and you yourself would still be writing code.

In a Waterfall, the testers would need to wait until all your work is done. And feedback from the customer generally has to wait until even the testers are done. The waterfall has a clearer division of the various phases but tends to add more work when the design needs to change during development, simply because you have to go back one or more phases.

O&S: Operation and Support

Once the project is done and well-tested it is time to deliver it to the customer. Web applications are stored on a web server and desktop applications are packaged in a special installer. App Store applications are sent to the proper App Store and for mainframe applications you would generally send some developer over to the mainframe with a floppy disk or USB drive containing the source code to compile the project on the mainframe itself!

And with the Arduino, you’d upload the application onto the processor, add the additional electronics and case and ship it to the customer.

So, what would you need?

What you need is creativity to come up with new ideas for interesting projects. You need to be able to describe them and you need to be able to draw images of the design. You might even need to draw database models and programming structures and more. Knowledge of UML would provide a bonus but you have to be careful with such modeling principles.

Making models and documentation doesn’t need to be perfect! You can misuse UML any way you like for as long as you can get the idea across to your team members, your customer and the users.

UML is like a programming language: just a tool to change an idea into a product. It just isn’t that important although it is very useful.

Personally, I prefer to take pen and paper and just draw my designs freestyle. It saves me the time to open up some drawing application and then messing around to make a proper design. If I’m on the computer I could just as easily use my development environment to create the forms and GUI parts and thus finish the job faster. But if I do that, I would already be in the Realization phase while the previous phases aren’t finished yet. It tends to lead to more mistakes in the final product.

Besides, I have a scanner. Once my drawing is done, I use my scanner to scan it and if need be, I would add more texts and other stuff using Paint. But by visualizing the design and drawing it on paper you should get some very good designs to discuss with your customer. And customers don’t mind if you’re just showing them some hand-made drawing, as long as you can bring the concept across!

Thing is, while UML is praised all over in the Academic World, it is generally not used in the industry. And if it is used in any way, it is generally used in informal ways with ad hoc sketches. Rarely does anyone in the industry make complete designs in UML for their projects, simply because it is complex, gets out of sync with projects real fast and generally doesn’t provide that many advantages compared to the effort it takes to create these diagrams.

As a developer, you will need to learn to read documentation and to follow these instructions very strictly, unless there’s a good reason why something would not be possible. If something is impossible, you should provide feedback to the management and/or the customer explaining why something cannot be done. This, as soon as possible so there is plenty of time to change the designs.

As a developer you should also be able to write your own documentation and it should not surprise you if you’re spending more time on documenting your work than writing actual code! Thing is, most code is not as self-explanatory as most people expect it would be. Even if you include comments in your code to explain how everything works, things could still be unclear to other developers or even to yourself if you review your own code one year from now! Your code and your documentation need to be synchronized so people can check if your project does exactly what you’ve described. This allows better testing and for the customer it will be clearer to know what he agreed upon.

So make sure that you know how to document your projects properly!

As a good example of why this is so important there’s one situation that I resolved that succeeded because of me properly writing the documentation! A new feature needed to be added to a desktop application and a colleague of mine was given this task to implement this and 5 weeks to finish it. And this period included Christmas and New Year. He started working on it and started writing code, experimenting and asking advice but did not manage to get any idea of how to solve it and out of frustration, he’d just quit his job! Poof! Gone…

So, with three weeks to go and me having two weeks off for the holidays, it became my task to take this over. It had already been promised to a customer to be finished in the first week of the new year so failure wasn’t really an option. So I started working in my weekend to solve the problem. This meant reading what was required and coming up with a solution that could be implemented in just two weeks. Fortunately, there were two other developers who could help me to implement it all, but we first needed a good plan.

So, on saturday I started reading and making notes. I stayed away from my computer and only used print-outs of the functional design and existing code. It took most of the day but I did end up with some good ideas.

On sunday, I started up my computer and started writing a Word document with lots of details about my plan that would solve it. I added various diagrams and defined specific tasks that needed to be done for the project to finish. I also kept in mind the experience of the other two developers and checked which tasks they could do and which tasks I would have to do. And on sunday evening I printed several copies of this document and also emailed it to my colleagues.

On monday, I discussed my plan with my colleagues and made clear that everyone knew what their part in the design was. Even management had a copy of the design so everyone could take part in this discussion to make sure that my plan would provide what we needed. And once everyone agreed, we started coding.

On friday, we already had a good working version. A week afterwards, we had a well-tested version with some minor issues. And when we were supposed to be finished I was already enjoying my holidays and my colleagues had done the last finishing touches and the whole thing was delivered just in time. And it worked great, even though it had been a lot of work with all kinds of pitfalls.

But because I wrote a detailed document explaining what needed to be done, we could make a success out of something that appeared to become a disaster. And that’s why having proper documentation is so important.

I also have tales telling how not documenting your work can lead to failure. I’ve seen plenty of developers with whom I worked who just skipped making any documentation of the work they did. In one case, a colleague had written a very complex framework but without any proper documentation there was only one person who could use it. Which is very annoying when you have a team of 10 or more developers who all need to use that framework!

It also didn’t help that this framework had all kinds of design flaws which this developer would need to solve. Which he did, but for the others it was still hard to use this framework because some functionality did not work as expected and some weird dependencies were all over the code.

If there had been any documentation then everyone could compare the framework with the documentation and just read the documentation to find possible flaws, or to point out flaws in the code when the code did not do what the documentation told it would. The lack of documentation causes a lack of quality and made things harder for all developers involved. Unfortunately, this is how many developers like to work because they focus on the tools, not on the final product.

Which is why you need to learn hot to plan, design and document your projects even before you learn how to program!

In the next post, I will finally start talking about the programming language that every developer needs to know: Standard C.