Category Archives: Projects

So, you want to be a software developer? Part 2.

Posted on by

Here’s part two about becoming a software developer. In the first part I told you about useful resources that will help you learn and once you’re working as a developer, those same resources will help you finish your projects.

Ramona on Computer 2_0001.png

By now, you should also have a few basic computer skills. Yet if you expect me to start talking about programming languages then no, not in this part. Because, as I said before, programming languages are not important. They are just tools and a good developer can use any tools he likes, even though it sometimes takes a short moment to get accustomed to the new tools.

To become a developer, you need to understand logic and you need to be able to visualize what your project will look like. And unfortunately, not everyone is born with these two abilities, although you can train to improve both abilities.

Having a mathematical background will help, since Math requires good logic. But there are other ways to train yourself.

To train your logic I would advise to learn to play Chess, Checkers or Go. Just these simple board games that require you to think ahead and outsmart your opponent. Especially Chess is a good game because you have various different pieces, each with different values, that you have to deploy over the ‘battlefield’ of 8×8 squares to capture the King of your opponent. Shredder Chess is a good chess engine that you can play online for free or on your mobile phone or tablet. And the various pieces will force you to think more strategically.

And although a game like Stratego and other games also requires strategic insights, the advantage of Chess is that the game starts with no hidden secrets for both players. Each player knows where the pieces of the opponents are and there’s no luck involved. Then again, if your opponent makes a mistake, you could consider that lucky, but it’s just a mistake by your opponent.

Learning Chess means that you have to think ahead in the game. Each move needs to be calculated. How will your opponent respond? And what will you do next. Good chess players can see a dozen or more moves ahead in the game, calculating which moves will win or lose. And to win, you must think further ahead than your opponent.

This is why training against a computer is a good way to improve your brains. Chess engines will use databases for the opening and the end game which will allow them to play strong, but in-between they have to think just like humans, and they generally calculate just up to a few moves ahead. It should not be difficult to think further ahead of the lower levels. And to improve, you should start playing against higher and higher levels until you win about half the games you play.

Next to learning strategy, logic and thinking ahead, you need to learn to visualize. For this, the best thing to do is to read. And since programming requires a good knowledge of the English language I would advise you to read a lot of English Science/Fiction literature. The stories of Conan the Barbarian, Tarzan or even the Dragonlance series are very good reading material. But if you don’t like this genre, start reading other stories where you can imagine how the main character is walking around in the environment set inside the book. Turn the words into a movie inside your head, visualize what is going on.

If that is a bit difficult, then read a book that has also been made into a movie. Movies like “the Hobbit” and “Lord of the Rings” were made on books written by Tolkien and make very good reading material. The movie follows the books reasonably close too, so visualizing it becomes easier. Other books would be the series “A song of Ice and Fire” which is used for “Game of Thrones” or “the Southern Vampire Mysteries” which resulted in the series “True Blood“. The movies should help you to visualize these worlds that are described inside the books.

Another way to learn how to visualize is through role-playing games and preferably those tabletop games with dice. Dungeons & Dragons is such a game but GURPS is my favorite. GURPS is less about superpowers and more adjustable to different worlds and settings. In GURPS, you can play a time-traveller who goes back to WWII with the mission to take down the Nazi’s in 1940 to prevent some disaster in the future. Or you are a time-travelling wildlife photographer being sent to the Age of Dinosaurs, 65 million years ago, filming how these dinosaurs came to their ends. Or maybe you’re just a Civil War-era cowboy fighting some Alien invaders. Then again, in GURPS you can also just be a dungeon crawler, bashing monsters and gathering loot.

Thing is, you need things that will require you to visualize what things will look like inside your head. This is important, since you will be working on various projects in the future and you will need to think ahead to know what it will look like once it is done!

As a developer, you have to understand the whole process starting with a good idea that you want to work out to finishing it completely and providing it to those who can use it. If you’re a carpenter, you might decide to build some chairs and a table and visualize how it should look like while the wood is still part of a tree. Once you are done, it is likely that it won’t look like what you visualized at the start, but that’s basically because you will keep changing your mind during the whole process.

However, if you made this for a customer then it should closely match what you agreed upon with this customer, since he expects that you deliver what you have suggested to him. You might have made drawings and descriptions so your customer will expect your project to match this. Then again, even clients can change their minds and together you could agree on changing some of the details. But in general, you start each process visualizing what you are going to make and think ahead about all the steps that need to be taken to turn the idea into a final product.

Without visualization, the whole process of building your project will be awfully slow and full with challenges. (Developers don’t talk about problems! We call it challenges, since problems are things that need to be solved and solving problems is a challenge!)

So, learn to use logic, learn to visualize things.

So, you want to be a software developer? Part 1.

Posted on by
Ramona on Computer 1_0001.png

So, you want to be a software developer too? Great! So, where do you start?

I started with a mentor, my father. Someone who could teach me some of the more important principles, even though it still early in the World of Computers. I had to learn from books and magazines but more importantly, I had to learn by practicing! And that’s how you should start too. You need to practice your computer skills first.

But nowadays, you have one major advantage over me. You have the Internet and there are thousands of mentors willing to help you with anything you need. So, before you will learn to write programs, you need to know a few important resources.

The most important resource for all programmers is this simple search engine called Google. You probably know it already and it is used by quite a few people, although some people dislike Google and how powerful this company actually is. They might use Yahoo, Bing or even DuckDuckGo. That’s just fine but in my own experience, Google tends to provide the most relevant answers to your questions.

Google does this by profiling. They keep a history of things you’ve searched for and if you use Google Chrome as web browser, they will most likely have a good idea of your interests. If you also have a GMail or Google Apps for Work account then they will have an even better profile of you and your preferences. And yes, in a way that invades in your privacy! So you have to be smart in how you use Chrome and Google!

One simple way to get around this is by using multiple accounts. See this link on how to add more than one profile to Chrome. Make one account for any development-related stuff and make sure a Google account and mailbox is assigned to this profile. This is what you will use to let Google profile your development requests. Create a second account for anything that you want to keep more private and use whatever you like. This second account can avoid all Google products and can be used if you like to surf for porn or whatever else using Yahoo. Or to write on Facebook and read tweets on Twitter. But use at least two accounts in Chrome if you want to develop software!

While Google is profiling your development account, you will notice how all search results will start to relate more and more towards software development. This way, you take advantage of what many people consider an invasion of privacy. Just make sure that anything you do with the development account is development-related.

So, now you have the most powerful online resource available to you and as you continue to use it, it will learn more and more about the information you need. So, let’s focus on a few other resources.

An old resource is Experts Exchange, which has a bit unfortunate name. People tend to confuse it with Expert Sexchange, which happens when people forget the dash in the URL or put it in the wrong place. You don’t want to confuse those two! Anyways, EE has been a very useful resource for me for about 15 years now, although the knowledge-base of this site has become saturated. They also turned it into a paid website once, started serving advertisement and they have been losing members since new members had no free access to any answers, which annoyed many people. To get free access, you needed to get a minimum number of points per month by answering the questions of other members for which points could be awarded to you. It’s a good system and they’re continuously searching for new ways to get extra revenue. And they have a huge knowledge-base that they are continuously expanding. Still, there is a slightly better resource available.

This alternative is Stack Overflow, where membership is free, although not ad-free. When you answer questions on this site, you can receive points which increase your rank. When your rank increases, you gain more access rights on the whole system and can even become a moderator. And like EE, this site has already a huge knowledge-base. And this is just one of the many Stack Exchange sites! The makers of SO decided to use the same web software to set up various other sites for other topics and this received a lot of support, allowing them to keep maintaining the software and sites. With over 150 different communities and topics, this is one of the best resources online. But for developers, Stack Overflow should be enough.

Another good resource is Quora. But Quora is more generic and can have all kinds of topics. It is a good place to ask non-technical questions about software development, like what kind of chair you need, what the best setup for your monitor and mouse is or even what to do when someone copied your source code without permission.

And of course, there are many more resources online that you can use, but to start, download Google Chrome and set up that developer account. Then use the resources I’ve just mentioned and start searching for more resources that you think are useful. Add all these resources to your favorites and make sure your Google settings are set to synchronize your settings with Google. Why? Because it allows you to share your favorites with your developer account on multiple computers! Or to synchronize between your computer, your laptop, your tablet with Chrome and your Android phone…

You may have noticed that I talk about what programmers have to do and I have said nothing about programming yet! This is because you will need to prepare first. As a developer, the first thing you need to know is how to find the information you need.

One thing to keep in mind is that programming languages aren’t as important as they seem. A programming language is just a tool, like a hammer or screwdriver. You use it to create a product. For developers, that product is a piece of software. And for people who will use it, it just doesn’t matter how you created it as long as it just works as it is supposed to. So relax! Programming isn’t really about learning programming languages but about making products with your typing hands and brains! Just like a carpenter who makes furniture…

A very generic datamodel.

Posted on by

I’ve come up with several projects in the past and a few have been mentioned here before. For example, the Garagesale project which was based on a system I called “CART”. Or the WordChain project that was a bit similar in structure. And because those similarities, I’ve been thinking about a very generic datamodel that should be handled to almost any project.

The advantage of a generic database is that you can focus on the business layer while you don’t need to change much in the database itself. The datamodel would still need development but by using the existing model, mapping to existing entities, you could keep it all very simple. And it resulted in this Datamodel:ClassDiagram(Click the image to see a bigger version.)

The top class is ‘Identifier’ which is just an ID of type GUID to find the records. Which will work fine in derived classes too. Since I’m using the Entity Framework 6 I can just use POCO to keep it all very simple. All I have to do is define a DBContext that tells me which tables (classes) I want. If I don’t create an entry for ‘Identifier’, the table won’t be created either.

The next class is the ‘DataContent’ class, which can hold any XML. That way, this class can contain all information that I define in code without the need to create new tables. I also linked it to a ‘DataTemplate’ class which can be used to validate the content of the XML with an XML schema or special style sheet. (I still need to work out how, exactly.) The template can be used to validate the data inside the content.

The ‘BaseItem’ and ‘BaseLink’ classes are the more important here. ‘BaseItem’ contains all fixed data within my system. In the CART system, this would be the catalog. And ‘BaseLink’ defines transactions of a specific item from one item to another. And that’s basically three-fourth of the CART system. (The template is already defined in the ‘DataTemplate’ class.)

I also created two separate link types. One to deal with fixed numbers which is called ‘CountLink’ which you generally use for items. (One cup, two girls, etc.) The other is for fractional numbers like weights or money and is called ‘AmountLink’. These two transaction types will be the most used transaction types, although ‘BaseLink’ can be used to transfer unique items. Derived links could be created to support more special situations but I can’t think of any.

The ‘BaseItems’ class will be used to derive more special items. These special items will define the relations with other items in the system. The simplest of them being the ‘ChildItem’ class that will define more information related to a specific item. They are strongly linked to the parent item, like wheels on a car or keys on a keyboard.

The ‘Relation’ class is used to group multiple items together. For example, we can have ‘Books’ defined as relation with multiple book items linked to it. A second group called ‘Possessions’ could also be created to contain all things I own. Items that would be in both groups would be what is in my personal library.

A special relation type is ‘Property’ which indicates that all items in the relation are owned by a specific owner. No matter what happens with those items, their owner stays the same. Such a property could e.g. be a bank account with a bank as owner. Even though customers use such accounts, the account itself could not be transferred to some other bank.

But the ‘Asset’ class is more interesting since assets are the only items that we can transfer. Any transaction will be about an asset moving from one item to another. Assets can still be anything and this class doesn’t differ much from the ‘BaseItem’ class.

A special asset is a contract. Contracts have a special purpose in transactions. Transactions are always between an item and a contract. Either you put an asset into a contract or extract it from a contract. And contracts themselves can be part of bigger contracts. By checking how much has been sent or received to a contract you can check if all transactions combined are valid. Transactions will have to specify if they’re sending items to the contract or receiving them from the contract.

The ‘BaseContract’ class is the more generic contract type and manages a list of transactions. When it has several transactions, it is important that there are no more ‘phantom items’. (A phantom item would be something that’s sent to the contract but not received by another item, or vice versa.) These contracts will need to be balanced as a check to see if they can be closed or not. They should be temporary and last from the first transaction to the last.

The ‘Contract’ type derived from ‘BaseContract’ contains an extra owner. This owner will be the one who owns any phantom items in the contract. This reduces the amount of transactions and makes the contract everlasting. (Although it can still be closed.) Balancing these contracts is not required, making them ideal as e.g. bank accounts.

Yes, it’s a bit more advanced than my earlier CART system but I’ve considered how I could use this for various projects that I have in mind. Not just the GarageSale project, but also a simple banking application, a chess notation application, a project to keep track of sugar measurements for people with diabetics and my WordChain application.

The banking application would be interesting. It would start with two ‘Relation’ records: “Banks” and “Clients”. The Banks relation would contain Bank records with information of multiple banks. The Clients relation would contain the client records for those banks. And because of the datamodel, clients can have multiple banks.

Banks would be owners of bank accounts, and those accounts would be contracts. All the bank needs to do is keep track of all money going in our out the account. (Making money just another item and all transactions will be of type ‘AmountLink’.) But to link those accounts to the persons who are authorized to receive money from the account, each account would need to be owner of a Property record. The property record then has a list of clients authorized to manage the account.

And we will need six different methods to create transactions. Authorized clients can add or withdraw money from the account. Other clients can send or receive payments from the account, where any money received from the contract needs to be authorized. Finally, the bank would like to have interest, or pays interest. (Or not.) These interest transactions don’t need authorization from the client.

The Chess Notation project would also be interesting. It would start with a Board item and 64 squares items plus a bunch of pieces assets. The game itself would be a basic contract without owner. The Game contract would contain a collection of transactions transferring all pieces to their first locations. A collection of ‘Move’ contracts would also be needed where the Game Contract owns them. The Move would show which move it is (including branches of the game) and the transactions that take place on the board. (White Rook gone from A1, White Rook added to A4 and Black pawn removed from A4, which translates into rook takes pawn at A4.)

It would be a very complex way to store a chess game, but it can be done in the same datamodel as my banking application.

With the diabetes project, each transaction would be a measurement. The contract would be owned by the person who is measuring his or her blood and we don’t need to send or receive these measurements, just link them to the contract.

The WordChain project would be a bit more complex. It would be a bunch of items with relations, properties and children. Contracts and assets would be used to support updates to the texts with every edit of a WordChain item kicking the old item out of the contract and adding a new item into the contract. That would result in a contract per word in the database.

A lot of work is still required to make sure it works as well as I expect. It would not be the most ideal datamodel for all these projects but it helps me to focus more on the business layer and the GUI without worrying about any database changes. Once the business model becomes more advanced, I could create a second data layer with a better datamodel to improve the performance of the data management.

 

 

 

To Agile/Scrum or not?

Posted on by

The Internet is full buzzwords that are used to make things sound more colorful than they are. Today’s buzzword seems to be “Cloud solutions” and it sounded so new a few years ago that many people applied this term to whatever they’re doing, simply to be part of the new revolutions. Not realizing that the Cloud is nothing more than a subset of websites and web services. And web services are a subset of the thin client/server technologies of over a decade ago. (Cross-breeding Client/Server with the Web will do that.) It’s just how things evolve and once in a while, a new buzzword needs to be created and marketeers are now working on the next buzzword that should make clear the Cloud is obsolete. Simply because new products need to be sold.

Still, the Software Development World hasn’t been quiet either. In the past, a project would be completed through a bunch of steps. It would start with an idea that they would turn into a concept. And this concept would include all requirements for the project.  Designers would then be called to come up with some basic principles and additional planning. When they’re done, they start to implement things, which would include methods to integrate the project into existing products and basically writing all code. It would then be tested and once the tests are satisfying, the whole project could be deployed and the maintenance would start.

If the project had problems in one of these steps, they would often have to go back one step. (Or more, in rare occasions.) This principle is called the “Waterfall model” and it’s drawback is that every step could take weeks to finish. It generally means that you can only update twice per year. Not very popular, these days.

So, new ideas were needed to make it possible to create updates more often. It started with the Agile Manifesto in 2001 and it has become a very popular method these days. Most groups of developers will have heard about it and have started implementing its principles. Well, more or less…

Agile has just four basic rules to keep in mind:

Individuals and interactions over processes and tools.
Working software over comprehensive documentation.
Customer collaboration over contract negotiation.
Responding to change over following a plan.

That’s basically the whole idea. And it sounds so simple since it makes clear what is important in the whole process. Agile focuses a lot on teamwork and tries to keep every team member involved in the whole process. Make sure every member is comfortable with the whole process and basically, talk a lot with one another over the whole process. People tend to forget it, but communication is a key element between people.

Of course, whatever you publish should work, and work well enough so users don’t complain about crashing applications or lost data. You might be missing features that customers would like, but that should not be the main focus of the whole process. Keep it working and keep the customer happy.

Of course, since you’re dealing with customers, you will need to know what they actually want. It’s fine if the CEO decided that the project needs methods X and Y to be implemented but if all customers tell you they want methods A or B implemented, then either the CEO has to change his mind or the company should start looking for a new CEO.

And keep in minds that things change, and sometimes change real fast. It’s hard to predict what next year will bring us, even online. Development systems get new updates, new plug-ins and new possibilities and you need to keep up to be able to get the most out of the tools available.

So, where do things go wrong?

Well, companies tend to violate these principles quite easily. And I’ve seen enough projects fail because of this, causing major damage or even bankrupt companies simply because the company failed at Agile. Failure can be devastating with Agile, since you’re developing at high speeds. And we all know, the faster you go, the harder you can fall…

Most problems with Agile starts with management. Especially the older managers tend to live in the past or don’t understand the whole process. Many Scrum Sprints are disrupted because management needs one or more developers from that sprint for some other task. I’ve seen sprints being disrupted because a main programmer was also responsible for maintaining a couple of web servers and during the sprint, one of those servers broke down. Since fixing it had priority, his tasks for that sprint could not be finished in time and unfortunately, other tasks depended on this task being ready.

Of course, the solution would be that another team member took over this task, but it did not fit the process that the company had set up. This task was for a major component that was under control by just one developer. Thus, he could not be replaced because it disturbed the process. (Because another developer might have slightly different ideas about doing some implementations.)

Fortunately, this only meant a delay of a few weeks and we had plenty of time before we needed to publish the new product. We’d just have to hurry a bit more…

Agile also tends to fail when teams don’t work well together. Another company had several teams all working on the same project. And unfortunately, the project wasn’t nicely divided in pieces so each team had its own part. No, all teams worked on all the code, all the pieces. And this, of course, spells trouble.

When you have multiple teams working on the same code, you will often need an extra step of merging code. This is not a problem is one team worked on part A and the other on part B. It does become a problem when both teams worked on part C and they wrote code that overlaps one another. Things will go fine when you test just the code of one team but after the merge, you need to test it all over again, thus the whole process gets delayed by one more sprint just to test the merged code. And it still leaves a lot of chances for including bugs that will be ignored during testing. Especially manual testing, when the tester has tested process X a dozen of times already for both teams and now has to test it again for the merged code. They might decide to just skip it, since they’ve seen it work dozens of times before so what could go wrong?

As it turns out, each team would do its own merging of the code with the main branch. Then they would build the main branch and tell the testers. Thus, while testers would be busy to test the main branch that team 1 provided, team 2 is also merging and will tell them again, a few days later. The result is basically that all tests have to be done over again so days of testing wasted. Team 3 would follow after this, thus again wasting days of testing. Team one then decides to include a small bugfix and again, testing will have to start from the beginning, all over again.

With automated testing, this is not a problem. You would have thousands of tests that should pass and after the update to the main branch, those tests would start running from begin to end. Computers don’t complain. However, some tests are done manually and the people who execute those tests will be really annoyed if they have to do the same test over and over with every new build. It would be better if they’d just try to automate their manual tests but that doesn’t always happen. So, occasionally they decide that they’ve tested part X often enough and it never failed so why should it fail the next time?

Well, because team 1 and team 2 wrote code that conflicts with one another and that code is in part X. The testers skip it, thus the customer will notice the bug. Painful!…

There are, of course, more problems. I’ve seen a small company that had a nice, exclusive contract with a very big company. Lets call them company Small and company Big. Company Small had created a product that company Big really liked so they asked for an exclusive version of it, with features that company Big would choose. And this would be a contract that would be worth tens of millions for company Small and its ten employees.

And things would have gone fine if company Small had not decided to continue working on its own products and just focused on delivering what company Big wanted, and to deliver in time. But no, other things were more important and the customer would just get what company Small made, with some minor adjustments. And the CEO was quite happy with this progress. That is, until the customer noticed that they did not hear his wishes. All company Big was supposed to do was sign the contract and pay the bill. And once things were done, they would just have to accept what was given to them. So company Big found another company willing to do the same project and just dumped company Small. End of contract and thus end of income, since company Small just worked exclusively for the bigger company. And within five months, company Small went tits-up, bankrupt. Why? Because they did not listen to the customer, they did not keep them happy.

And another problem is the fact that companies respond very slowly on changes. I’ve worked for companies that used development tools that were 5 years old, simply because they did not want to upgrade. I still see the occasional job offering where companies ask for developers skilled with Visual Studio 2008 while there are three newer versions available already. (Versions 2010, 2012 and 2013.) In 2003 I was still working on a 16-bit project that was meant to be used by Windows 3.1 and up, simply because one single user still used an old Windows 3.11 system. At least, we thought they did because no one ever asked them if they’ve upgraded. And that customer never told us that they had indeed upgraded and didn’t think of asking for a 32-bit version…

I’ve seen management hang on to a certain solution even though there’s plenty of evidence that newer options are available. I’ve developed software on 32-bit systems with 2 GB of memory when 64-bit systems were available and had up to 8 GB of memory, plus more speed. I had to use a single-monitor system on a PC that had options for multiple monitors plus we had extra monitors available, but management considered it a waste. The world is changing and many systems now easily support two or more monitors but some companies don’t want to follow.

So, what is Agile anyways? It’s a method to quickly respond to changes and desires of customers with a well-informed team that feels committed to the task and to deliver something the customer wants. (And customers want something they can use and which works…)

Would there be a reason not to use Agile? Actually, yes. It’s not a silver bullet or golden axe that you can use to solve anything. It’s a mindset that everyone in the team should follow. One single member in the team can disrupt the whole process. One manager who is still used to “the old ways” can devastate whole sprints. When Agile fails, it can fail quite hard. And if you lack the reserves, failure at Agile can break your company.

Agile also works better for larger projects, with reasonable big teams. A small project with one team of three members is actually too small to fully implement the Agile way of working, although it can use some parts of it. Such a small team tends to make planning a bit more difficult, especially if team members aren’t always available for the daily scrum meetings. When you’re that small, it’s just better to meet when everyone is available and discuss the next steps. No clear deadlines, since the planning is too complex. What matters is that goals are set and an estimation is made when it is finished. Whenever the team meets, they can then decide if the estimation is still correct or if it needs to be adjusted.

Another problem can be the specialists that are part of the team. Say, for example, that you have a PHP project that needs to communicate with a mainframe and some code written in COBOL. The team might have hundreds of PGP developers but chances are that none of them know anything about COBOL. So you need to have a COBOL specialist. And basically he alone would carry the tasks of maintaining the mainframe side of the project. You can make him part of the Scrum meetings but since he has to do his part all by himself, he doesn’t have much use for the other team members. So again, just decide on a specific goal and estimate when it should be finished. Get regular updates to allow adjustments and let the COBOL developer do his work.

The specialist can become even more troublesome if you have to interact with a project that another company is creating. If you do things correctly, you and the other company would discuss a generic interface for the interaction between both projects. You would then both build a stub for the other company to use for testing. This stub just has to offer some dummy information, but it should be usable.

When both companies have the stubs they need, they can each work on their part. They will have to keep each other informed if some parts of the interface need to be changed or if some rules are changed about the data that can be provided. Preferably, this is done by providing a new stub. Both teams will have just one goal, which is providing all the required methods that are part of the stubs. And when parts are fully implemented, they can offer the other company with new stubs that contain some working parts already.

Still, when two companies have to work together this way, they have to think small. Don’t create a stub with thousands of methods for all the things you want to add during the next 5 years. Start small. Just add things to the stub that you want to finish for the next sprint. Repeat adding things per sprint and communicate with the other company about what they’re going to add next. You don’t have to work on the same method of the stubs anyways. One company might start working on the GUI part that allows users to enter name, address and phone number while the other works on storing employment data and import/export management. The stubs should just give dummy methods for those parts that aren’t implemented yet. Each company should develop the parts that they consider the most important, although both should be aware that everything is finished only if all stub methods are implemented.

Agile is just a mindset. If used properly, it can be very powerful. However, do keep in mind that not all of Agile might be practical for your own situation. Agile requires a lot of time for meetings with developers, with customers and with management. Everyone needs to be involved and everyone needs to be available for those meetings. Scrum becomes more difficult if not all team workers are available on all five workdays of the week. And worse of all,, team members will have to prepare for the meetings. Even for the daily meetings since they have to keep track of their own progress.

Do not fear to just implement part of the whole Agile/Scrum principle. It is made to hybridise with other methods. Use the methods, don’t let the method force itself upon you.

The challenge for the CART system.

Posted on by

In The CART datamodel I displayed the datamodel that would be required for the CART system. Basically, the data model would store the items, transactions and contracts while the templates will be stored in code, as XML structures that are serialized to objects and back again. As a result, I would split the relations from the data, thus allowing me to avoid regular updates to the database structure. All that I might want to update are the templates but even that might not be required for as long as the software will support “older versions” of those objects.

But serializing those objects to and from XML isn’t as easy as it seems. Since I’ve separated data from relations, the data itself doesn’t know its own ID. Why? Because the ID is part of the relation, thus I would not need to store it within the XML. (It would be redundant.) But if I want to use these objects through a web service, having the ability to send this ID to the client is very practical, so I can send back changes through the same web service. I would need the ID to tell the server what to do with which object.

Thus I’ll end up with two methods of serializations. One is just to serialize the data itself. The other is the data plus its ID. And now I will have to decide a way to support both. Preferably in a simple way that would not require me to generate lots and lots of code.

In the data layer, I would split up every object into a relation and a data part. The data would be stored as XML within the relation object. To make sure the relation object will be able to expose the object, I would need to give it a DataObject property that would be the data object. It’s get/set methods should connect to the XML inside, preferably even by using an object as buffer so I don’t have to serialize it over and over again.

In the business layer, I should not have an XML property, nor should I have a DataObject property. The data fields should be there, with the ID. And basically, I would need a mapping to convert between the data layer and the business layer. The trouble with this approach is that I would define all data twice. Once in the data template and once in the business layer. That’s not very smart. I need to re-use things…

I’m considering to add my serialization method for the data templates. This means that I will include the ID within the template, so it becomes part of the object. All properties would be defined as data members, including the ID. That way, the ID is sent from the business layer to the client. But to store the template in the relation object, I would need to create my solution.

One solution would be by implementing methods to convert the data to XML plus a constructor that would accept XML to create it. It would also mean that I need a way to recognize each object type so I can call the proper construction and probably inherit a bunch of code or write interfaces to make objects more practical to be called upon. It would be complex…

Another solution would be by defining my own attributes. One would be for the class name, thus allowing me to find classes based on this custom attribute. The other would be for the property and my code would just use all of those to generate the XML or to read it back again. This too would allow custom field names. It would be a cleaner solution since I would define the template as just a bunch of properties. Which, basically, they are.

But this second solution is a bit complex, since I still need a way to call the constructor of these specific classes. So I’ve opened a question on StackOverflow, hoping I will get an interesting answer that would solve this easily. Why? Because part of being a good developer is to ask other experts for possible solutions when yourself don’t have a good answer! 🙂

The CART datamodel

Posted on by

Well, my back problems made me think a lot about the CART system that I’ve mentioned before. And it made me consider how I need to handle the difference between plain data and the relationship between the objects. Because the most troubling thing I had to deal with was that I did not want to change my datamodel for every new item that I want to store. So it made me think…

The CART system is mostly created to handle relationships between items, transactions and contracts. It’s not about handling of the data itself. Actually, the system doesn’t even care about the data. All that matters are the relationships. Data is just something to display to the user, something to store but normally not something that you’ll need to process very often at the data layer. So, considering the fact that you can serialize objects in .NET to XML data, I’ve decided to support a basic structure for my Garage Sale project for all the items, transactions and contracts. And each of them will contain a Data property that has the serialized data, that I could convert to data objects and back again.

This idea makes it also more clear where the templates are within my system. My templates are these object definitions! I will have a very generic database with a very simple layout, and I can generate a very complex business layer around this all that I can change as often as I like without the need to change my database model. As a result, I might never have to change the data model again!

Of course it will have a drawback, since the database will contain serialized objects. If I change those objects, I will also need to keep track of the changes in those stored structures and either update them or keep up multiple versions of those objects. Updating those structures would mean that I have to create update apps that know both the old structures and the new structures. It should then convert each old structure to a new structure. Maintaining multiple versions might be more practical since that would leave all old data intact in your system. Anyways, it’s some added complexity that I’ll have to consider.

But now, my datamodel as created by Visual Studio 2012 by using the Entity Framework:EF-CART

 

So, what do you see?

  • The DataObject is the base class for any CART object. It has a unique identifier, a name that can be used to show the related object and a Data property that will contain an object as XML.
  • DataItem is a generic item class, containing an additional description just for more practical reasons. When a user wants to select an existing item, having a description makes it possible to avoid reading the object within the data.
  • The Collection table is just an item with an extra bonus. It can contain multiple child items without the need for any transactions. Actually, this is just a shortcut solution to allow more complex structures within your items. For example, you might create a complete Household collection containing husband, wife, four children and a dog. And although you could link them together by using transactions, having them in a collection just saves the need to create those transactions.
  • DataTransactions is the base class for the transactions, having a sender, receiver and subject item connected together. It also has a link to a rule and a timestamp, indicating when the transaction tool place. (Or will take place for future transactions.)
  • IntegerTransaction is just a basic transaction with a multiplier. This way, you don’t have to add a lot of transactions when your customer buys ten bags of flour.
  • DecimalTransaction is also a basic transaction that will handle items that can be traded in all kinds of different numbers, including fractional amounts. For example, the price of a product, or it’s weight, length or light intensity.
  • DataRule is the basic contract type. It’s a collection of transactions that are all related to one another. For example, the sale of a product would result in a sale rule.
  • The Contract class is more than just a rule. It’s a rule that can contain child rules, thus allowing structured contracts that are made up of several subcontracts. These are used when you have to deal with complex
    situations, like mortgages. A mortgage would include a rule for purchasing a house, a rule for lending money and paying it back, plus other rules for all kinds of insurances.

Now, as I’ve said before, this datamodel should offer me more than enough structural parts to use for my Garage Sale project. All I need to do is compile it and then just leave it alone. There should not be a need to include anything else.

Well, okay… That’s not completely true, since I might want to handle user accounts and store large images of products. But these things would actually require new database structures and should preferably be part of separate databases.

Looking back at this design, it surprises even me how less data it actually has. But the trick here is that I’ve separated the relationships between objects from the actual data itself. Each object can still contain a huge amount of data. But it’s just not important for the model itself.

An interesting sub-project.

Posted on by

While working on my WordChain project (See post “The Wordchain data model.“) I reminded myself that I might have a simple problem when I have to deal with images. Since the WordChain project uses simple routing for each word, any link is most likely being translated to a word. This would require me to use a special route for images. But another problem that arose is the storage requirements, which would increase enormously if I would add a lot of images. And storage of such images in e.g. a Microsoft Azure environment would make hosting a bit more expensive.

The solution here would be using a separate host for just the images, and store the URLs to those images within WordChain. This will allow me to use a cheap data store for the images and a bit more expensive host, like Azure, for the WordChain website.

But it also means that I will have to think of a solution to host the images somewhere, and write a simple project that will serve these images to wherever I want to use them. These images should be divided into categories and preferably some more information needs to be added to keep track of the author, original source, license and whatever else could be interesting. It could grow to some interesting gallery application that could be used to serve images to several other websites.

So here’s a new project that I want to design. And again, I will need to start designing things. However, this time I can’t start with the user interface, since the user interface would just give the image. I need to design the tool that will allow me to upload images to the host, including the other data that I want to be part of it.

Project born from pain…

Posted on by

It was February 14, 2013. I had the flu, but needed medicines for my Diabetes. So I had no choice, took my car to drive the 800 meters (!) from home to the apothecary to pick them up. It was snowing a bit, the streets were slippery because and well.. My car just made sure that I did not have to walk over 800 meters of slippery, icy streets. Just about 20 meters.

It did not work. I slipped and badly injured my back while I tried to get back into the car after picking up my medicines. I managed to stand up reasonably quickly, which assured me that I had not broken anything severe, but the pain… The pain…

I’ve started this project on Sunday, four days later. The pain is still there and my physician subscribed Paracetamol 1.000 mg and Ibuprofen 600 mg, three times per day. That’s probably enough to sedate a horse, but I still feel the pain. To make it worse, it just hurts most when I try to stand or sit, although standing or sitting quietly will bring the pain down again. So here I am, sitting at my computer, trying to deal with the pain and making my mind wonder about what to do. Still, as times passes by, I do manage to handle the pain better.

Well, that’s not too difficult. I’m just going to create a new project which I would consider practical and educational. I don’t know how much time it will take to finish but that’s no big deal since I’m not going anywhere. But first, what should the project be?

Well, after considering how Wikipedia and Twitter work, I think it would be nice to have something with similar, practical usage. A website that links words and phrases together by hyperlinks and short texts. This could even be expanded by adding images, video and other content, as long as things are kept short. I am thinking about a header with an icon and title, a few special tabs on the right side for specific topics and beneath the title the working area. Below the working area there would be a footer with copyright information and other stuff about the site itself.

The working area would be divided into three parts. A left column containing a word cloud showing the mostly related words to the current topic, with a list of links below for outside references to other websites. The right column would contain short phrases which would also lead to other topics, plus a list of related meta-data like images, videos, documents and other stuff. The center part would be the main content and would contain a short description, possibly with a small image or video included. This content area might also contain an input-form so visitors can offer their own data here.

This would be a perfect web site to create using Visual Studio 2012 with MVC 4. Plus the entity framework for storage. However, there is another challenge: how to fill it with data? And for this, I decided on a reasonable unique concept: a second website! Why? Because I want the main website to be read-only. A root domain name followed by the word that needs to be looked up, nothing more. I have no use for a “/setup” or “/admin” path because that would mean I could not use the words “setup” or “admin” as descriptions. A second site that would share the same database would be more practical.

But two sites means using two (sub)domains. But that’s not a big problem. All I need to do first is add those two sub domains to my current domain. So I go to my domain registrar and add two A labels to my DNS settings. I want to create word.wimtenbrink.nl for the site itself and wordsetup.wimtenbrink.nl for the configuration site. And I need to tell IIS on my web server to start hosting those sites, even if they’re not doing anything. It will take some time for the DNS system to actually find my sites but by reserving the domain names already, I have one delay less once I really need to start testing in a production environment.

Then again, this production environment is still experimental. If the project is finally finished, the real version will be hosted somewhere else. Most likely under Microsoft Azure or a RackSpace cloud. Why? Because my simple web server at home won’t be able to handle more than a few requests per minute.

Now I have the whole hosting solution ready so it’s time to design a few things. I start with the user interface of the site itself. This should tell me the data that I will need to store. And this can simply be drawn with pen and paper. Once the design is done, I need to consider the data that I see within my design and how to store it all. Storage would end up in an SQL Server database plus a small bit within the configuration file of the project. Once storage is done, I need to think of a web service that will allow me access to this data, and I would need to do that one twice. Once for read-only access for the site itself and once with read/write options for the setup. (And the reading part would be shared between both.) Once the web service is set up, I can start designing the user interfaces. The site itself will use MVC 4 and ASP.NET with C# as the programming language. The setup is something that I want to be a challenge so I’m considering to write it as a client/server desktop application and possibly port it to Android using Delphi or Oxygene.

All in all, this project will thus have the following steps, which I will elaborate in some next posts:

  1. Design the user interface on paper. Scan it, add some descriptions to it and make a functional design.
  2. Consider all data that needs to be stored, create a database structure by generating an Entity Framework model around the data.
  3. Define the web service methods that I need and find a simple way to share all read methods between two different websites.
  4. Create the user interface for the web site. Use a simple SQL script to fill it with some dummy data.
  5. Build the setup application using whatever technique is most appropriate.

And yes, this is a waterfall technique. If I was more than one person, Scrum could be very useful since multiple me’s could work on each separate thing. Since I want to create it all by myself, doing it step by step will be good enough.

And now back to bed. It took three days to write this all with all the pain in my back. At least in bed I can easily hold the pen and paper and do the first step…