Just a reflection spawned from a recent post by Greg Wilson and a later discussion at the lab (I had to leave early, so I don’t know how it ended):
In a research discipline with few or no mature constructs, such as ours, experimentation is little more than disguised, expensive, and poorly grounded argumentation. It is misleading: it proves nothing, it can be made to support almost anything, and it has an aura of “truth” disproportionate to the weakness of its real validity.
It is best to develop constructs and their underlying theories through argumentation enriched by qualitative observations, and to proceed to experimentation only when the constructs seem sound.
I realize that a great part of the appeal of experimentation is psychological. It is comforting to hear that someone ran an experiment and “demonstrated” this or that. But these demonstrations are flimsy, and will be superseded and forgotten once we develop a better understanding of the relevant constructs and theories.