There is a call for action making the rounds among software researchers. You may have heard of it: it goes by the name of SEMAT (Software Engineering Method and Theory); Ivar Jacobson presents it as a “revolution” whose “goal is to re-found software engineering as a rigorous discipline.” You can read the call for action here. The list of adherents is rather impressive and includes several people I admire deeply.
But this is an entirely misguided movement. Let’s analyze the call for action, bit by bit. It begins:
Software engineering is gravely hampered today by immature practices. [...]
We start with what seems to be a minor point, a mere choice of words, but the problems begin by setting this supposedly re-foundational effort firmly within the old paradigm that got us where we are now. Thus we do “software engineering” research, and we should do what other engineers and engineering researchers do. Never mind that people in the community (most prominently Tom DeMarco) have spoken against this paradigm, or that software development has little to do with most established engineering disciplines: the people behind SEMAT have chosen to ignore these criticisms and move on.
Note that the SEMAT call for action presents the current state of the software industry as “hampered” by “immature practices,” which may be true (we’ve only been developing software for a few decades; our practices are necessarily immature), but as an observation it is symptomatic of the software-crisis hypochondria that afflicts many software researchers. The call continues:
[...] Specific problems include:
- The prevalence of fads more typical of fashion industry than of an engineering discipline.
This is how the consequences of the engineering paradigm I mentioned above manifest themselves. It’s true that the software industry is notoriously susceptible to fads. But notice how the alternative to these fads, according to the signatories of SEMAT, is to behave like an “engineering discipline.” That is, they offer a dichotomy between the archetypical fad-based industry (whimsical, pretentious, superficial, frivolous) and the archetypical measurements-based industry (conservative, positivist, serious, professional).
In fact, some “fads” end up being pretty good ideas. Object-orientation is mainstream today; it was once a fad. Extreme programming has been shown to be more effective than its alternatives under many conditions; some people still label it as a fad. There is a danger of labelling any grassroots innovation, any experimentation by practitioners, as a “fad”, as not conforming to the seriousness of our engineering discipline —there is a danger of stifling progress by an appeal to an unsuitable paradigm.
(This criticism doesn’t even get into the problem that several of the fads that have been inflicted on the software industry were started or promoted by some of the signatories!)
Let’s continue with SEMAT’s list of problems:
- The lack of a sound, widely accepted theoretical basis.
- The huge number of methods and method variants, with differences little understood and artificially magnified.
I agree with the first of these points, and I have argued in the past for the need to develop better foundational theories in our field. The second point, however, makes me doubt whether the signatories realize the implications of the first. We have plenty of evidence that software development is not homogeneous. Developing an operating system is not the same as developing videogames; developing mission-critical control systems is not the same as developing mission-critical data warehousing scripts. Different settings demand different methods. This diversity must play an essential role in our theories —unless we want our theories to be disconnected from the evidence we have collected.
In fairness, what the SEMAT adherents probably mean is that we have far too many academic software processes. Whole PhDs have been built on the basis of designing supposedly new methods, practices, or tools (really just a variation of old ones, under a different name) with almost no empirical validation. If this is the case, if what SEMAT refers to is our pathological growth of academic software methods without empirical validation, then I would agree with this point.
- The lack of credible experimental evaluation and validation.
- The split between industry practice and academic research.
In this case I agree with the second point, but I partially object to the first. Industrial practice and academic research are certainly disconnected, and we need many more researchers bridging this gap if we are to solve the problem.
In fact I, too, believe that we need credible experimental evaluation and validation. I only object to this point because of its emphasis on experimentation, not on empiricism in general. This is another example of locking ourselves in the engineering paradigm: we want controlled experiments and measurements, the rest of our empirical work is not worth mentioning. As a researcher that has applied both experimental and non-experimental empirical methods in software development, I have seen that controlled experiments tend to abstract away the important elements of the phenomena we study, whereas non-experimental methods, such as case studies, provide a necessary richness to our evidence. Software development is hard, in part, because of the numerous elements and interactions in the socio-technical systems in which it takes place. These elements are almost impossible to replicate in the lab. Controlled experiments only give us a narrow piece of the story.
The SEMAT call for action then switches gears and offers a proposal:
We support a process to refound software engineering based on a solid theory, proven principles and best practices that:
- Include a kernel of widely-agreed elements, extensible for specific uses
- Addresses both technology and people issues
- Are supported by industry, academia, researchers and users
- Support extension in the face of changing requirements and technology
SEMAT, then, proposes to re-found software theory by decree. We decide to pick up “a solid theory” (which one?) that includes “a kernel of widely-agreed elements.” (which ones?) I simply cannot think of any scientific field that has prospered under such an epistemological basis. If we have a kernel of widely-agreed elements, that in itself is our theory and there is no need to refound anything. But we do not have a theory precisely because we do not yet have a kernel of widely-agreed elements; we cannot create such a kernel out of thin air.
Furthermore, I cannot see how such a kernel can arise out of the combined work of the signatories. The list includes, for instance, Alistair Cockburn and Watts Humphrey. It includes Larry Constantine and David Harel. It includes Ken Schwaber and Barry Boehm. The list itself suggests that, more than a kernel of widely-agreed elements, what we already have is several competing conceptions of software development, several vaguely formulated theories, that need to be refined and tested against each other. This is, I believe, where we should spend our efforts.
The rest of the items in SEMAT’s proposal are mush. Of course our theories need to address technological and social issues. Of course they need wide support by several communities to be successful. Of course they must be flexible. But what should they consist of? What stake is SEMAT putting on the ground? Unfortunately, beyond a wish to be more like an engineering discipline, this proposal is completely vague, and therefore I cannot support it.