A critique from Alistair Cockburn on how the agile movement is under attack from Taylorism led me to an essay by Dave West on the philosophical incompatibilities between lean and agile techniques, and this in turn led me to finally give a read to Peter Naur’s 1985 text, “Programming as Theory Building.” (Also available in Appendix B of Cockburn’s “Agile Software Development” book, and here.) I don’t know why I had not read it earlier. Not only did I find it a brilliant example of the kind of clear argumentation that I think is missing from much software research today, I also found that it should have been a key building block of my Ph.D. thesis: for the first time since I finished it, I felt the urge to go back and tinker with it some more. Perhaps I did read it at some point, absorbed it, and forgot about it.
Naur explains what he’s after in the abstract to his paper:
(…) it is concluded that the proper, primary aim of programming is, not to produce programs, but to have the programmers build theories of the manner in which the problems at hand are solved by program execution.
The actual code that the programmers deliver is not the point of programming. That code will probably soon need to be changed again: it lives in a state of constant flux. Instead, the real goal of the members of a development team is to understand in depth the problem that they are trying to solve and the solution that they are developing to solve it. If the team builds an appropriate theory, its software will be a better fit to the context in which it will perform, and the team members will find it easier to carry out the inevitable modifications and enhancements to its software. In fact, Naur stresses the extent to which the theory is important and the code is unimportant in a pretty clear way: he claims that the code in isolation from its developers is dead, even though it may remain useful in some ways:
During the program life a programmer team possessing its theory remains in active control of the program, and in particular retains control over all modifications. The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modifications of the program cannot be intelligently answered. Revival of a program is the rebuilding of its theory by a new programmer team.
Reading Naur’s paper I felt a very deep connection to the ideas I put forward in my thesis: Naur’s programmer’s theories are essentially mental models in the sense I (and many others before me) present them, and both he and I claim that the overarching goal of a software development organization is to build those models (or theories) during the life of the project. I could actually restate my thesis contributions as extensions to Naur’s sketch in two ways: first, I explore what I think is the main challenge that software team members find today: to build consistent mental models (or in the terms of the thesis, to develop a shared understanding) of the world, among potentially large groups of people, in the face of abundant, shifting, and tacit information, and unclear or exploratory goals. Second, I outline some attributes of team interaction that make such a challenge easier to overcome.
I was glad to see that several of my conclusions mirror Naur’s. He argues that programming methods (taken as sets of work rules for programmers that tell them what to do next) are unhelpful from a Theory Building view because we can’t really systematize theory production: like other knowledge construction endeavours, it is an organic process. Developers can, and perhaps should, have a set of techniques and tools at their disposal, but they are ultimately in charge of choosing the actions that will best help them build their theories at any given time. Naur also argues that documentation is not an appropriate mechanism to transmit knowledge in software projects, an observation that I explore when I discuss the differences between the Shared Understanding paradigm and the more prevalent paradigms in software research (which I named Process Engineering and Information Flow). He claims that since the main end result of a development effort is the inarticulated theory that the programmers have built, “the notion of the programmer as an easily replaceable component in the program production activity has to be abandoned,” an observation that I think is better received now than it was at the time (it was taken as one of the organizing principles of the agile movement), and that in my own analysis I labeled proportionality of action and responsibility.
I really enjoyed reading someone far smarter than I am presenting these arguments clearly and concisely. I only wonder, how is it that more than 25 years later we still need to be making roughly the same points—how is it that they still feel fresh, mainly uncharted, and in need of advocacy?