I recently came across Tom DeMarco’s “Controlling Software Projects” for a second time, and I remembered my problem with it immediately: The very first line in the book states that “You can’t control what you can’t measure”, and the rest of the text builds upon that phrase to argue that we need metrics to rein the chaos of software development.
But the book falls apart because the statement is wrong, and you and me are living proof of that. We control things we don’t measure, all the time. You are able to control your body (your breathing, your hunger, your thoughts) without measurements, and you are an expert at it. In some cases measurements can help –for example, for controlling your weight, or your cholesterol. In many others, however, measurements would not even make sense (do you need to measure the length and number of your hairs to know when to get a haircut?)
What bothers me is that the phrase has picked up among software project managers, and is often used to justify absurd metrics and policies: Relying on measurements such as lines of code for programmer efficiency, or number of errors found for software quality, are excellent ways to lose control over a project. They are deficient and misleading proxies for the real constructs that one wants to study.
There is a kernel of truth to the phrase, however. The unknown is, by definition, uncontrollable, and sometimes numbers help us increase our knowledge. But I would much rather read a software development book that relaxes the statement to something like “You can’t control the unfamiliar”, and builds a thesis from there. It doesn’t have the same punch, but it has the benefit of being true.
“The problem with the saying “you can’t manage what you can’t measure”—what makes it a fallacy—is that we manage things we can’t measure all the time. We manage cancer research. We manage software design. We manage all manner of things that are deeply intellectual, even creative, without any idea of what numbers we ought to have to guide us. Good knowledge worker managers tend to measure qualitatively, not quantitatively.”