I recently came across Tom DeMarco’s “Controlling Software Projects” for a second time, and I remembered my problem with it immediately: The very first line in the book states that “You can’t control what you can’t measure”, and the rest of the text builds upon that phrase to argue that we need metrics to rein the chaos of software development.
But the book falls apart because the statement is wrong, and you and me are living proof of that. We control things we don’t measure, all the time. You are able to control your body (your breathing, your hunger, your thoughts) without measurements, and you are an expert at it. In some cases measurements can help –for example, for controlling your weight, or your cholesterol. In many others, however, measurements would not even make sense (do you need to measure the length and number of your hairs to know when to get a haircut?)
What bothers me is that the phrase has picked up among software project managers, and is often used to justify absurd metrics and policies: Relying on measurements such as lines of code for programmer efficiency, or number of errors found for software quality, are excellent ways to lose control over a project. They are deficient and misleading proxies for the real constructs that one wants to study.
There is a kernel of truth to the phrase, however. The unknown is, by definition, uncontrollable, and sometimes numbers help us increase our knowledge. But I would much rather read a software development book that relaxes the statement to something like “You can’t control the unfamiliar”, and builds a thesis from there. It doesn’t have the same punch, but it has the benefit of being true.
Robert Glass, by the way, puts it better than me in his excellent book “Facts and Fallacies of Software Engineering” (the relevant passage is available here for free):
“The problem with the saying “you can’t manage what you can’t measure”—what makes it a fallacy—is that we manage things we can’t measure all the time. We manage cancer research. We manage software design. We manage all manner of things that are deeply intellectual, even creative, without any idea of what numbers we ought to have to guide us. Good knowledge worker managers tend to measure qualitatively, not quantitatively.”
11 Comments
January 12, 2007 at 12:42 pm
I prefer what Tom Gilb said, “Anything you need to quantify can be measured in some way that is superior to not measuring it at all. “
January 14, 2007 at 12:32 pm
Thanks for chipping in, Mike!
Gilb’s phrase does not really contradict my argument. I’m not against measurements, I’m against believing that one *requires* measurements to achieve control.
Having said that, I’d take Gilb’s phrase with a grain of salt. Sometimes getting proper and accurate measurements may cost more than the benefits we might obtain from having them.
January 16, 2007 at 3:25 pm
The idea that you need to measure everything to control it make sense when you include non-quantitative measurements. You don’t measure your hair but you look at a mirror and compare its lenght to you usual standard. That is a way of measure that relies on qualitative standards. You can measure the amount of food you have by the sensation of fullness and, with hope, happiness of your belly. Not all measures have to be in ones and zeroes, but DeMarco’s work miss this fact and falls in excess when measure the efficiency of a worker by the lines of code he writes….
January 16, 2007 at 3:53 pm
Right –if we expand the definition of measurement to include qualitative assessments, then I have no problem with the phrase. But the standard meaning of the word calls for numbers, and that’s the way the phrase is interpreted most of the time!
July 31, 2007 at 2:59 pm
I do not agree on your statement that “We control things we don’t measure”, since in all your examples there is some type of measurement. Not all the measurements have to be quantified some can be qualified (as good, normal, etc); these are also types of measurements.
For example, not because you think you can not measure the number of times your hurt beats per minute doesn’t mean that your body (or your mind) isn’t doing some kind of measurement unconsciously. Have you ever had a heart attack? Do you know what you feel before your body knows that your heart is going to stop? This is because your unconscious self measured something in your body that’s not right, and this is a type of metrics an it has a specific way of evaluation and of showing a result as simple as healthy or sick or as complicated as any name of strange disease you can come up with..
If you think thoroughly on every example you gave you can find a kind of metric that is helping control the process, and of course it is measurable.
The main issue here is to find the appropriate metric and also to find the appropriate way to measure it, because if you do not do that you might reach wrong conclusions like thinking that you can control things that you can not measure.
July 31, 2007 at 3:58 pm
Thanks for your comments, Leopoldo. Again, if in the meaning of ‘measurement’ we include all sorts of qualitative assessments, then of course, “you can’t control what you can’t measure” —meaning, “you can’t control it if you don’t know what’s going on”.
But really, the phrase is consistently used as an argument for quantitative measurements, and against qualitative and unconscious ones. This is what bugs me.
September 27, 2007 at 12:56 pm
The concept of measurement is a left-brain activity and precludes all emotion and intuition. Human beings are vibrant souls who are designed to rely on emotions and intution for decision-making but when we have cut ourselves off from that internal wisdom, we fall back on measurements to tell us what we should know. For sure, we need both sides of the brain firing away to find solutions.
November 28, 2007 at 10:17 pm
Apparently the “You can’t manage what you can’t measure” disciples also suffer from an inability to understand that “feedback” is different from quantitative measurement.
Of course management depends upon a feedback loop. That is so stunningly self-evident it doesn’t merit comment. Otherwise, all managers could sit at home in a sound-proof room and issue outbound emails all day long and never check their inbox.
However, quantifiable metrics and “dashboards” aren’t sufficent to drive good management. In fact in many instances such “hard data” fails the manager because: (a) we’re measuring the wrong thing; (b) the measurement is flawed or doesn’t include all the necessary factors; or (c) the interpretation of the measurement is wrong. Most importantly having quantifiable numbers can lull a manager into a false sense of security, deluding himself / herself that he has things under control, when in fact, all he has is a piece of paper which doesn’t describe emerging competitive threats, employee morale, or thousands of other factors which impact a business but are difficult to quantify with great granularity.
“You can’t manage what you can’t measure” is a crutch for the insecure and the inept.
November 9, 2008 at 4:32 pm
That’s why Maslow created B-language, so that we can assemble a discriminator circuit. We can begin to measure Beauty, Justice, Fairness, etc., at the group level as we develop cyberspaces reflecting our value systems. Maslow has a list that jives well with a Gardner view. But more important is to sniff out the nay-sayers from the yea-sayers. At the bottom of the nay-sayers ( life negative) are the mentors of the suicide bombers, who, if they had the courage to blow THEMSELVES up, would solve the problem.
November 9, 2008 at 4:35 pm
You said, “Good knowledge worker managers tend to measure qualitatively, not quantitatively.”
I beg to differ. They want ALL of the good guys.
February 26, 2009 at 11:47 am
[...] on measurement Jump to Comments Related to yesterday’s post and to an entry I wrote two years ago on measurement, I just found an old (2003) post from Martin Fowler on why we can’t measure [...]