On the Art of Computer Programming
I have been a software engineer for about 30 years, and it is remarkable how many new observations I still have about the art of programming. I could write volumes about this, but this morning had an important observation about the need for a good programmer to be able to shift their point of view from an abstract to concrete view with great facility.
There is a memorable quotation from F. Scott Fitzgerald that says:
The test of a first-fate intelligence is the ability to
hold two opposed ideas in mind at the same time and still retain the
ability to function.
I am not sure whether it’s the test of a first-rate intelligence, or even whether Fitzgerald really did. After all, he wrote it when he was amidst one of the most dark and self-deprecating phases of his life. But I can say with certainty it is a skill that an extraordinary programmer must possess.
Extraordinary programming involves two fundamentally opposed concepts: abstract conceptualization and mastery of detail. These two skills are rarely present in the same person. I often have found great "idea people" who have been able to conceptualize broad and sweeping ideas and yet are frustrated by the need to conquer all of the minutiae required to realize their dreams. To be successful, they must empower others to do the leg work. By contrast, there are many detail-oriented people who can master unbeleivably complex systems from top to bottom, but cannot see the larger purpose.
Software systems consist of myriads of components, each of which serves an abstract purpose. So, extraordinary programmers view software systems as simple things. It is this simplicity that drives their design. A computer serves a purpose which can be described simply. A program serves a simple purpose. A library serves a simple purpose. A function serves a simple purpose. To create the best software, all systems must be reduced to the simplest most abstract notion of what is to be accomplished, and such discrete systems are combined easily to form larger discrete systems. That is the goal of good design.
To implement such systems, a programmer must start with an abstract notion. Then, during the debugging and implementation process, only the truly extraordinary programmer is willing to explore virtually every single detail of how the abstract notion behaves, and every detail of how every abstract building block upon which it rests behaves. They see how functions they call operate, inspect the assembly language generated, and would not hesitate to understand fully perhaps 20 or 30 dependent functions nested deep within software they did not write.
Only the extraordinary programmer resists having the details become the master to the abstract, or the abstract the master of the details. Any system which is brilliantly conceived and poorly implemented, or brilliantly implemented but built upon an overly baroque or complex abstract notion, is flawed. To avoid this, it requires being able to simultaneously see a simple, elegant system while also seeing thousands of details which make it operate. The extraordinary programmer knows that every detail counts. Every detail supports the abstract concept or it is superfluous and should be removed like a cancer.
There are many essays and philosophies which support this way of thinking. But, it is harder to do, and more rarely seen than one supposes. In software, it is the exception rather than the rule, and until it becomes the standard of software design, we will continue to be frustrated by the software we use.
