Andy Ko blogged yesterday about having a paper rejected at FSE, the Foundations of Software Engineering Conference, because it used qualitative research methods. His post includes depressing snippets from his reviews, and his replies are fitting and well worth reading:
Transforming empirical observations into numbers does NOT make them objective, nor does it prevent bias and misinterpretation.
I’ve written about this before. As a community, our excessive focus on quantitative methods is preventing us from developing the right constructs to study. It’s a shame that when researchers address this fundamental problem they’re shot down by their peers.