Everything Old is New Again

One of the aspects of having been involved with computers for 20+ years that I enjoy is being able to see how things keep coming back around. One story that keeps coming back is how "non-programmers" are going to be able to program one day. I just read a story in the New York Times about Intentional Software. For those too who choose not to read the article, Intentional Software is a company that is in the process of making a set of tool that allow "non-programmers" to define the intention they have for how the software should work. Then those intentions are rendered as code. Finally, the code is compiled into an executable.
Let me start by saying I think this is a great idea. Despite the fact that it may mean fewer software development jobs in the long run, I still think it is a great idea. Unfortunately it will not work. You see, it attacks the wrong problem. The article describes three advantages of Intentional Software's solution:

1. The people who design a program are the ones who understand the task that needs to be automated.
2. The design can be manipulated simply and directly, rather than by rewriting arcane computer code.
3. Human programmers do not generate the final software code, thus reducing bugs and other errors.

The first advantage is simply not true. The idea that there is a domain expert that completely and thoroughly understands any business process is a myth. There are certainly people who understand some aspects of a process, but there is no one who understands the process completely. Furthermore, even if you could find one person who understands the process well, that person has an ideal implementation in mind. This is almost always not what the business is doing now. What the business is doing now is a result of the interplay of many people who have different ideas of what the business should be doing now. Some of these ideas are well thought out and the result of debate and/or experience. Others are a result of people wanting to get to lunch, or leave at a certain time. This mess is what defines the non-automated process. This is why collecting requirements for a software project is so difficult. Since there is no one who understand the entire process, many people are interviewed to collect the requirements. The requirements are collected based on how clearly each person can explain their views on how things should be done. They are also effected by how well each person can sell their ideas. Even if the system is implemented close to this description of the intent of the software, if the right people were not involved, the final system will be less than ideal. Of course there is also the problem that while the software is produced and tested the needs of the company are in flux as a result of many internal and external factors.

The next advantage is closer to the mark. "Arcane computer code" is what I do 5 days (sometimes more) a week. I find it neither arcane nor do I think of it as "computer code". It is difficult for some people to understand this, but in many ways I am more comfortable with say C# than I am with English. The advantage of an arcane computer language is that they tend to be very precise and limited. I know that the computer is going to do in most cases. It is true that sometimes I overlook or confuse things, but I do this much less often in C# than I do in English. The higher the abstraction, the more effort goes into understanding what is actually going to happen. Very high level languages have the issue that one must be familiar with the subtly of the language. There is a greater risk that my interpretation will not match the interpretation of the computer. Furthermore, high level abstractions tend to be very domain specific. Take an inventory abstraction for instance. Is it tracking the inventory in a vending machine, a retail store, a warehouse, a factory? Even within these distinction there are distinctions. Is the vending machine selling discrete items (i.e. candy bars, sodas, etc) or is it vending coffee (by the cup). Is the warehouse holding drugs, steel, food? Each of these variations must be represented somehow. Either there needs to be a huge third party ecosystem that creates each of these specialized abstractions, or the abstraction must be generic enough that it is adequate to cover all the possibilities. Adequate solutions are not good enough to differentiate your business, by the way. This is why most companies use software, after all. They want a competitive advantage. So they also need to be able to create their own high level abstractions.

This brings us to the final advantage. Since we have removed the "human" element from the process there will be fewer errors right? Not really. Firstly we have not removed humans we are using a different set of humans. If we have a high level abstraction that is widely applicable we will reduce errors, because we have a large number of people that are sharing the effort to find errors in code. This is why GUI tool kits actually do make programmers more productive. As you move the abstraction layer up you naturally are working with smaller audiences. The inventory abstractions I spoke of in the last paragraph show how high level abstractions must target a specific domain to be of real value. Even then, the parts of the software that really make a difference tend to be unique to a single product. That is why they make a difference. So now you are back to using software produced in house for one application. All of the advantages of having many people reviewing and correcting the software go away.

So where does that leave us? Software is hard because:

1. No one really knows what it should do
2. Software that give me a business advantage is different than software that other people are using
3. There are technical challenges in creating software

Intentional Software's solution will help with item three, but does little or nothing to help with the first two. I don't mean to pick on them. There has been any number of technologies that where supposed to make software development easier that failed prior to them. If one looks at Delphi, Visual Basic, or an ancient Dos programming tool called Layout, they all had the same promise, and none of them changed the world. They all try to make the act of developing software easier, but if you look at item number one, it has nothing to do with the act of writing software, except that it is a prerequisite. Item two means that any sort of software factory won't really solve the problem either. Solving the third problem is helpful, but not enough to change the experience significantly. Software is hard for real world reasons.

To Paraphrase Albert Einstein, "Software Development should be as simple as possible, but not simpler".