Experimentation and argumentation

Just a reflection spawned from a recent post by Greg Wilson and a later discussion at the lab (I had to leave early, so I don’t know how it ended):

In a research discipline with few or no mature constructs, such as ours, experimentation is little more than disguised, expensive, and poorly grounded argumentation. It is misleading: it proves nothing, it can be made to support almost anything, and it has an aura of “truth” disproportionate to the weakness of its real validity.

It is best to develop constructs and their underlying theories through argumentation enriched by qualitative observations, and to proceed to experimentation only when the constructs seem sound.

I realize that a great part of the appeal of experimentation is psychological. It is comforting to hear that someone ran an experiment and “demonstrated” this or that. But these demonstrations are flimsy, and will be superseded and forgotten once we develop a better understanding of the relevant constructs and theories.


About Jorge Aranda

I'm currently a Postdoctoral Fellow at the SEGAL and CHISEL labs in the Department of Computer Science of the University of Victoria.
This entry was posted in Academia. Bookmark the permalink.

5 Responses to Experimentation and argumentation

  1. Lorin says:

    People who are influenced by the outcome of software engineering studies? I thought they were a myth. 😉

  2. Pingback: More on measurement « Catenary

  3. Pingback: Andy Ko and the semblance of objectivity in numbers « Catenary

  4. Pingback: The thorny and the obvious | Catenary

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s