Before letting the ICSE 2007 topic to rest, I wanted to write some notes about some papers that I found satisfying and relevant to my research interests. ICSE is a huge conference, and although it has something for everyone, it’s easy to get lost in its proceedings. Greg Wilson has already posted an initial list of his picks; if you’re into the social side of software development, these are my recommendations, in no particular order:
‘Good’ Organisational Reasons for ‘Bad’ Software Testing, by David Martin, John Rooksby, Mark Rouncefield, and Ian Sommerville. In this ethnographic study, Martin & Co. show that testing in agile software development depends heavily on the organizational context of the team. The motivation of the research team is that there’s very little real world data on how companies do testing and verification:
“Many descriptions in Software Engineering research of the application of testing and verification methods to real world problems, whilst welcome, should be considered more-so as demonstrations than as reports of how testing is done in practice.”
The research team spent a month’s work of fieldwork observing testing and verification in a small software company. They found that the company largely dismissed test plans and textbook “best practices”, using instead a “thoroughly pragmatic” approach for apparently good reasons:
“W1REsys are not ignorant of best practice in software testing […] but, for good organisational reasons, chose not to adopt this but to orient their testing to meet organizational needs. Their driving priorities are:
- The dynamics of real customer relationships […]
- Using limited effort in the most effective way […]
- The timing of software releases […]
- The need to ‘grow the market’ for the system […]
These priorities, they argue, influence their selection of software development practices far more than technical testing concerns.
All in all, although the participating company seems to dismiss some low-effort, high-payoff testing tools and practices, and although the research team does not delve into these possibilities, this is an engaging, insightful paper. It’s also necessary: as I’ve argued before, software development academia and industry are now so disconnected that we need this sort of work to tell it like it is and bring them back together.
The Social Dynamics of Pair Programming, by Jan Chong and Tom Hurlbutt. Another ethnographic study of agile software development. The researchers observed professional pair programmers from two software development teams, and documented their dynamics and interactions. Being an ethnographic study, the paper does not offer any quantitative results –such as those presented by Arisholm & Co four months ago–. However, the paper is chock-full of insights about the way pair programming really works. Some of the findings:
- Drivers and navigators: Although almost every description of pair programming emphasizes a division of labor between “driver” and “navigator”, Chong and Hurlbutt found that “pair programmers behave in ways inconsistent with the driver/navigator division of labor that is described in the pair programming literature”, and suggest that trainers and agile development texts should just drop this artificial division that people are bound to ignore anyway.
- Keyboards: It seems innocuous, but having the capability of switching keyboard control within the pair appears to be a much more productive set-up than having just one keyboard and one typist.
- Expertise: Pair programming is often presented as a way to spread expertise between team members — you pick up stuff from experts you work with, and pass on your expertise to your own pairs. But Chong and Hurlbutt warn us to be careful about pairing people with considerable expertise differentials: the expert tends to drive the bulk of the programming discussions, makes decisions without consulting, and does not seem to explain the rationale behind his actions.
Some of these findings are perfect hypotheses for confirmation/rejection through quantitative, controlled studies. I hope to see more on this in the short term…
Software Development Environments for Scientific and Engineering Software: A Series of Case Studies, by Jeffrey Carver, Richard Kendall, Susan Squires, and Douglass Post. This paper reports some technical and organizational details of the developers of five high performance computing projects. For outsiders, it’s easy to forget some of the concerns of these groups –portability and maintainability are goals as important as performance, as the code written today will most likely still be in use thirty years from now.
Carver & Co. discuss lessons learned from talking to these groups. The lessons range from the very insightful (e.g., validation and verification is extremely difficult when you don’t know what’s the right answer) to the exasperating (some experienced developers reject IDEs because they feel they impose “rigidity” on their development activities –don’t ask me why).
I would have liked more information on the organizational configurations of these teams, and I disliked the authors’ confusion between agile development and the anything-goes approach to development in these projects. However, this is still a very valuable window into the dynamics of these high-performance computing and engineering teams.
Information Needs in Collocated Software Development Teams, by Andrew Ko, Robert DeLine, and Gina Venolia. One more of the very few ICSE studies of real developers doing real work. The authors observed 17 Microsoft developers for 90 minutes each, and recorded the tasks performed by these developers, their interruptions, and the reasons why they often could not complete the task they were working on.
From these observations, the research team came up with a list of types of “information needs” of developers (“what is the program supposed to do? which changes are part of this submission? what is the purpose of this code?”). They report on the frequency and the outcome of the searches for each of these information needs. They suggest that their list is representative of the problems developers have in their daily work, and that it should drive the creation of new tools, processes, and notations from the software engineering community.
Perhaps unfortunately, too many developers seemed to be doing debugging work at the time of the observations, and so the list is biased towards debugging tasks. However, it is still valuable as a roadmap for software development research.
Global Software Engineering: The Future of Socio-technical Coordination, by James Herbsleb (requires ACM subscription). After explaining what is different about global software engineering (less communication, less effective communication, lack of awareness, and team incompatibilities), Herbsleb discusses a series of challenges that are relevant to the area and are likely to see action from the research community. It’s an informative and candid discussion, with bits such as this one about offshoring:
“Most development organizations seem to be placing large bets that the correct configuration [of software teams] is to have an analyst and marketing group physically near the customer, but are also willing to locate one or more development groups remotely. As yet we have little evidence of the effectiveness of this arrangement.”
Re-reading this post, I notice it would seem as if ICSE as a conference was full of relevant, exciting, reality-based papers about how to understand and improve software development. Unfortunately, it’s not. I found these papers to be the exception rather than the rule. Here’s to seeing the trend improve at ICSE 2008!