I spent Sunday at a workshop organized by the Consortium for Software Engineering Research (CSER). The theme was "empirical software engineering", a subfield that has emerged since the late 1980s whose practitioners focus on studying and evaluating software development methods and tools in systematic, rigorous ways. I went there to try to persuade educators at other Canadian universities to start using DrProject to manage undergraduate programming teams. A few people seemed interested, but what really got the room talking was the ethical issues surrounding the collection and publication of data on how students actually use tools like DrProject.
At one end, Ontario's new privacy law says that information can only be used for the purposes for which it was originally collected. If interpreted strictly, this would disallow studies like the one we did two years ago, in which we tried to find patterns in students' use of CVS that correlated with the grades on assignments, since we did not obtain explicit permission from those students to publish our analysis of their data. At the other end, there are dozens of papers at the SIGCSE conference every year in which educators present data on students grades. I can't speak for all of them, but I'm pretty sure that most haven't asked for their students' permission.
Janice Singer (National Research Council), Peggy Storey (University of Victoria), and Steve Easterbrook (University of Toronto) are writing a book chapter on the ethics of doing software engineering studies. I'm looking forward to it, and will blog when it appears.
The other big topic yesterday was the mechanics of actually doing empirical studies. Coincidentally, two articles landed on my screen this morning: one from DanC singing the praises of agile development in the gaming industry, and an update on Steve Yegge's piece on good agile vs. bad agile (which I covered a couple of weeks ago). There are lots of strong opinions in both, but no actual data; what I took away from yesterday's workshop is that it is possible to study these issues, instead of just arguing about them, and that our profession would be a lot better off if we did that more often.