My ICSE 2007 picks

Before letting the ICSE 2007 topic to rest, I wanted to write some notes about some papers that I found satisfying and relevant to my research interests. ICSE is a huge conference, and although it has something for everyone, it’s easy to get lost in its proceedings. Greg Wilson has already posted an initial list of his picks; if you’re into the social side of software development, these are my recommendations, in no particular order:

‘Good’ Organisational Reasons for ‘Bad’ Software Testing, by David Martin, John Rooksby, Mark Rouncefield, and Ian Sommerville. In this ethnographic study, Martin & Co. show that testing in agile software development depends heavily on the organizational context of the team. The motivation of the research team is that there’s very little real world data on how companies do testing and verification:

“Many descriptions in Software Engineering research of the application of testing and verification methods to real world problems, whilst welcome, should be considered more-so as demonstrations than as reports of how testing is done in practice.”

The research team spent a month’s work of fieldwork observing testing and verification in a small software company. They found that the company largely dismissed test plans and textbook “best practices”, using instead a “thoroughly pragmatic” approach for apparently good reasons:

“W1REsys are not ignorant of best practice in software testing [...] but, for good organisational reasons, chose not to adopt this but to orient their testing to meet organizational needs. Their driving priorities are:

1. The dynamics of real customer relationships [...]
2. Using limited effort in the most effective way [...]
3. The timing of software releases [...]
4. The need to ‘grow the market’ for the system [...]

These priorities, they argue, influence their selection of software development practices far more than technical testing concerns.

All in all, although the participating company seems to dismiss some low-effort, high-payoff testing tools and practices, and although the research team does not delve into these possibilities, this is an engaging, insightful paper. It’s also necessary: as I’ve argued before, software development academia and industry are now so disconnected that we need this sort of work to tell it like it is and bring them back together.

The Social Dynamics of Pair Programming, by Jan Chong and Tom Hurlbutt. Another ethnographic study of agile software development. The researchers observed professional pair programmers from two software development teams, and documented their dynamics and interactions. Being an ethnographic study, the paper does not offer any quantitative results –such as those presented by Arisholm & Co four months ago–. However, the paper is chock-full of insights about the way pair programming really works. Some of the findings:

1. Drivers and navigators: Although almost every description of pair programming emphasizes a division of labor between “driver” and “navigator”, Chong and Hurlbutt found that “pair programmers behave in ways inconsistent with the driver/navigator division of labor that is described in the pair programming literature”, and suggest that trainers and agile development texts should just drop this artificial division that people are bound to ignore anyway.
2. Keyboards: It seems innocuous, but having the capability of switching keyboard control within the pair appears to be a much more productive set-up than having just one keyboard and one typist.
3. Expertise: Pair programming is often presented as a way to spread expertise between team members — you pick up stuff from experts you work with, and pass on your expertise to your own pairs. But Chong and Hurlbutt warn us to be careful about pairing people with considerable expertise differentials: the expert tends to drive the bulk of the programming discussions, makes decisions without consulting, and does not seem to explain the rationale behind his actions.

Some of these findings are perfect hypotheses for confirmation/rejection through quantitative, controlled studies. I hope to see more on this in the short term…

Software Development Environments for Scientific and Engineering Software: A Series of Case Studies, by Jeffrey Carver, Richard Kendall, Susan Squires, and Douglass Post. This paper reports some technical and organizational details of the developers of five high performance computing projects. For outsiders, it’s easy to forget some of the concerns of these groups –portability and maintainability are goals as important as performance, as the code written today will most likely still be in use thirty years from now.

Carver & Co. discuss lessons learned from talking to these groups. The lessons range from the very insightful (e.g., validation and verification is extremely difficult when you don’t know what’s the right answer) to the exasperating (some experienced developers reject IDEs because they feel they impose “rigidity” on their development activities –don’t ask me why).

I would have liked more information on the organizational configurations of these teams, and I disliked the authors’ confusion between agile development and the anything-goes approach to development in these projects. However, this is still a very valuable window into the dynamics of these high-performance computing and engineering teams.

Information Needs in Collocated Software Development Teams, by Andrew Ko, Robert DeLine, and Gina Venolia. One more of the very few ICSE studies of real developers doing real work. The authors observed 17 Microsoft developers for 90 minutes each, and recorded the tasks performed by these developers, their interruptions, and the reasons why they often could not complete the task they were working on.

From these observations, the research team came up with a list of types of “information needs” of developers (“what is the program supposed to do? which changes are part of this submission? what is the purpose of this code?”). They report on the frequency and the outcome of the searches for each of these information needs. They suggest that their list is representative of the problems developers have in their daily work, and that it should drive the creation of new tools, processes, and notations from the software engineering community.

Perhaps unfortunately, too many developers seemed to be doing debugging work at the time of the observations, and so the list is biased towards debugging tasks. However, it is still valuable as a roadmap for software development research.

Global Software Engineering: The Future of Socio-technical Coordination, by James Herbsleb (requires ACM subscription). After explaining what is different about global software engineering (less communication, less effective communication, lack of awareness, and team incompatibilities), Herbsleb discusses a series of challenges that are relevant to the area and are likely to see action from the research community. It’s an informative and candid discussion, with bits such as this one about offshoring:

“Most development organizations seem to be placing large bets that the correct configuration [of software teams] is to have an analyst and marketing group physically near the customer, but are also willing to locate one or more development groups remotely. As yet we have little evidence of the effectiveness of this arrangement.”

Re-reading this post, I notice it would seem as if ICSE as a conference was full of relevant, exciting, reality-based papers about how to understand and improve software development. Unfortunately, it’s not. I found these papers to be the exception rather than the rule. Here’s to seeing the trend improve at ICSE 2008!

Ed Yourdon on the Peopleware panel

Ed Yourdon, one of the participants of the ICSE Peopleware panel I blogged about, has an extremely informative description of the panel’s discussion in his blog.

Here’s an insightful bit: During the panel, Tom DeMarco half-jokingly blamed Barry Boehm (another panelist) for the relatively slow adoption of agile development. Boehm, after all, had reported that the cost of fixing software defects rises exponentially on the time between their injection and their discovery. Yourdon describes:

[DeMarco] said that as a result, the commandment “get the requirements right!” was drummed into the heads of a generation of software engineers. Tom turned towards Barry, smiled, wagged his finger, and said, “And I have never forgiven you!”

Barry Boehm relieved the tension in the air by agreeing with Tom. He explained that, back in the 1970s, he had linked up with Win Royce at TRW, where the two of them found that the waterfall methodology worked pretty well. But he acknowledged that they were working in an application domain (aerospace systems, military systems), and in a time, when the end-user’s requirements were fairly well-defined; consequently, it made a great deal of sense to capture those requirements early, rather than discovering later on that a great deal of software had been built to implement the wrong requirements. But Boehm acknowledged that by the 1980s, things had begun to change drastically … and obviously this continues to be true today.

Well worth a read.

Also, Yourdon entered a small comment in my ICSE post. I think it’s the first time that the author of books I read more than a decade ago as an undergrad comments my blog. It’s quite an honour.

Unfortunately, he called me Chris.

Artful Making

Some years ago, while I was an undergrad and for a couple of years after graduation, I participated heavily in two theatre groups, mainly as an actor, but also as a translator, backstage staff, and assistant director.

I loved it all along, absolutely, in great part because of the peculiar kind of teamwork that we had back then: a deep sense of responsibility and trust (I’ll catch you if you fall, and I know you’ll do likewise), of respect for your teammates’ skills, of shared love for what we were doing (we certainly weren’t there for the money), and of unstructured, organic creativity.

As a software developer I tried to find the same type of experience, unsuccessfully. I thought that perhaps software development had some characteristics that made it too difficult to nurture those qualities, or that, perhaps, what we went through at the theatre was a fluke, an unrepeatable and already gone experience.

But it’s not a fluke –you probably have been a part of a team as satisfactory as my theatre troupe, and know what I’m talking about–, and there’s nothing that should keep software from being developed with this sort of team dynamics. I just didn’t know how to make it happen, how to reproduce the environment that would foster such teamwork.

Enter Artful Making. In the Peopleware panel that to me was the highlight of ICSE, Tim Lister recommended this book by Rob Austin and Lee Devin. I can see why: I’ve just finished reading it, and I loved it.

Artful Making builds the case that a number of knowledge work companies are erroneously using an “industrial making” approach to their projects: they try to standardize processes, to shield themselves from uncertainty, and to plan heavily before acting. For many industries, Austin and Devin say, this is the right approach. But for those occupations that share a particular set of characteristics (low cost iterations, need for innovation, and reliable repetition), the industrial making approach is not helpful. For this kind of work –and especially for software development–, Austin and Devin suggest instead an “artful making” approach, one that, strangely enough, is based on their observations of how theatre troupes work. They convincingly claim that the business world has a lot to learn from these artists, they point to cases where this learning is currently happening (in particular, to agile software development), and they distill and explain the four progressive qualities that they deem essential for artful making:

• Release: What you do when you focus and let yourself flow in an activity. It’s controlling by aiming and allowing variation, rather than by restraining. It “contrasts with restraint, the usual method of industrial control”.
• Collaboration: Where “each party, released from vanity, inhibition, and preconceptions, treats the contributions of other parties as material to make with, not as positions to argue with, so that new and unpredictable ideas emerge.”
• Ensemble: What happens when “individual members relinquish sovereignty over their work and thus create something none could have made alone.”
• Play: Play represents the ongoing, productive interaction between maker and user. “… [The] product of a business should be redefined as the experience of its interaction with customers, an interaction in which both product and customers vary over time.”

Contrary to what I initially expected, the book is not touchy-feely, it’s not new age-y, and it has nothing to sell. Instead, it’s a fantastic look at the similarities between actors and knowledge workers, and a deceptively simple roadmap to make artfully, and to have fun while doing so.

King’s College Circle

This new perspective of King’s College Circle at the University of Toronto is freaking cool:

(Click the picture to see it in full size.)

By Sam Javanrouh, at Daily Dose of Imagery.

Suggestions for a more woman-friendly community

Found via Greg Wilson: DevChixs’ GloriaJW has a great article on gender barriers in tech communities. And as Greg also points out, the article’s comments add interesting angles to the post.

Playing it as (you think) it is

I got an idea for a videogame-based study that can help bridge the gap between software developers and academics.

The video game itself would be a simulator of a software development company, similar to what SimCity does for cities, SimAnt for ant colonies, and SimSewageSystem for the less-than-pleasant aspects of urban planning. I am aware of one software development simulator video game, SimSE, developed at UC Irvine by Emily Navarro and others, and used for educational purposes. There are many other software simulators, but they are not really supposed to be video games, and they are not fun.

In my game your company can of course take software projects, hire personnel, and have them deal with every aspect of software development, from setting the office layout to implementing XP or the CMM or the methodology of your choice. You deal with requirements changes, data corruption, and programming egos. You need to decide whether to ship your product with lots of known bugs, and who knows how many undiscovered ones, or to keep pouring money in a seemingly never-ending fix cycle in order to minimize the chances of wreaking havoc in your users’ lives.

What’s that you say? It’s not fun? Well if you think so you haven’t tried SimAnt. Or Patrician. In Patrician you’re a Hanseatic League merchant, and you get to do crazy stuff like expanding your warehouses, hiring drunkards in the tavern, and deciding how much money you donate to the local charities. And it’s fun.

Anyway, that’s not the point. What I’m trying to get at is that this game would give its players the capability to alter its simulation parameters. Let me explain. All simulators require some parameters, and the choice of parameter values generally reflects the assumptions of the game designers. In SimCity you can’t really expect your city to grow unless you cut taxes, and no matter how much money you pour into the health system your citizens won’t stop complaining. In The Sims you need to make friends with your whole neighbourhood to advance your career, and happiness largely depends on your furniture. Etcetera. Simulations of software development make similar assumptions: if you don’t do requirements work your costs will go up later in the project by 90%; the best developers are 3 times more productive than the worst; the best current tools can send your productivity up by 50%. I just made these figures up –for most simulators, if you disagreed you’d have no choice but to go with these assumptions (or stop playing the game).

Simulators, then, reflect the way that their designer sees the world, and since most software simulators are created by academics, they reflect an artificial account of how software gets developed in the real world. But in this game that I’m proposing, you allow players to set the simulator parameters the way they see fit. What happens then is that you have a much better chance of understanding how they see the world, and what is important to them.

So this is the study: We get people both from academia and industry to play this game (don’t ask me how because I haven’t figured that out yet). Each player sets the game parameters the way they think best reflects reality. I’m thinking of having about 10 parameters, in the form of questions with a slider that they can adjust. They can try out the game, and adjust the parameters again if necessary until satisfied.

And when they’ve all played the game, we compare their parameters. We don’t care how well they did in the game (though that would be interesting too). What we care about is how are the choices made by industry people and academics different. Do they assign similar values to people, processes, and tools? Do they intend to reward some strategies over others? Are each of the two groups internally consistent? The result would be a map of the perceptions of both groups with regards to priorities and success factors in software projects.

The reason why answering these questions is important is that there is a big divide between software engineering academia and industry, and we need to reduce it. At the last ICSE, for instance, which is largely an academic conference, I heard many variations of the wish of “making software engineering more like the other engineerings”, but I’ve hardly ever noticed such a need from the people that actually earn their bread from writing code. If we do nothing to address this divergence of goals and expectations, we risk pushing our field into irrelevance. If, on the other hand, we make the differences in perception explicit, we can begin to discuss them and fix the problem in two ways: by adjusting our research goals, and by educating developers on research that has been empirically demonstrated to work but that seems counterintuitive for them.

Playing this game, then, really is a bit of a convoluted approach to get people’s perceptions on software development. There are other methods to do so, such as a simple questionnaire. But I’d argue this method is more reliable than simply asking them to fill out a survey, since one can answer it as one pleases and forget about it, but investing effort and emotion in a simulation makes one more likely to consider these questions seriously, and to reconsider one’s answers in the light of their consequences.

So that’s the gist of it. The problem is, there’s no game yet. I’d like to develop it, or to help develop it, but mostly at this point I’d like to get your feedback. Is this feasible? Stupid? If you have any ideas on how to make this project better, or if you’d like to participate in any way, send me a note. It doesn’t appear to be a short project, but it might be a fun one!

CheatNeutral

A funny parody of Carbon Offsetting projects:

What is Cheat Offsetting?
When you cheat on your partner you add to the heartbreak, pain and jealousy in the atmosphere.
Cheatneutral offsets your cheating by funding someone else to be faithful and NOT cheat. This neutralises the pain and unhappy emotion and leaves you with a clear conscience.

Can I offset all my cheating?
First you should look at ways of reducing your cheating. Once you’ve done this you can use Cheatneutral to offset the remaining, unavoidable cheating

Their whole website is nice, but their short film is particularly good.

What the World Eats

Catspaw pointed me to this photo-essay from Time magazine. It has photos from sixteen families around the world, and the food they eat in a week. It’s very interesting. I couldn’t help but notice the Casales family of Cuernavaca, Mexico:

24 liters of Coca Cola. Yup, that’s Mexico alright.