The Third Bit

Testify! It’s nice to see a shout-out to the Empirical Software Engineering Journal.

I’ve always wondered why Lutz Prechelt was never able to get his tech report about interpersonal variation published.

For the record, two of my favorite empirical studies are:

Shneiderman experiments on flowcharts (doi:10.1145/359605.359610)

Knight/Leveson experiment on N-version programming (doi:10.1109/32.44387).

Both were negative results (!), and both seemed to have a genuine impact on practice.

But regarding Schneiderman’s experiments on flowcharts, see David A. Scanlan’s paper “Structured Flowcharts Outperform Pseudocode: An Experimental Comparison” from IEEE Software 6(5), Sept 1989. Scanlan points out that Schneiderman et al compared structured pseudocode with spaghetti flowcharts. If you use equally structured representations, flowcharts actually _outperform_ pseudocode.

Thank you! That was a fantastic talk, well worth the effort and expense of driving out from Michigan for DevDays.

On the “Greatest Hits II” slide, was an effort made by Fagan to classify how serious and/or subtle the errors were? Thinking about my own work, it seems to me that the majority of bugs I create are pretty easy to track down and fix, and it seems like these would be very amenable to the review process. On the other hand, many of the more hellish bugs I’ve faced were very subtle interactions between different layers of code, of the sort I wouldn’t expect a code review to catch that often. (Not an argument against code review, just wondering if the numbers Fagan reported painted too rosy a picture.)

Huh, now I’m wondering how TDD would affect that study….

Slides look interesting but is there an audio/video podcast of this talk?

Thanks for the slides Greg. I will be doing a recap of StackOverflow DevDays for our company, and the slides will help to cover some of the details that I couldn’t write down fast enough.

@Sol, I also drove up from Michigan (Lansing). Where did everybody else drive from?

I don’t know of any audio or video — if someone recorded, I’d be grateful for a copy or link.

Intrigued by the title of these slides on YCombinator today, I clicked through and was pleasantly surprised to see that you were their maker! (I’m Katy from the Hacker Within at UW-Madison, how nice to run into you on the internet again!)

Anyway, I just wanted to say that, as a woman in the sciences constantly struggling with software development, I thought your comparison of this problem to Dweck’s work on gender in the sciences was right on target.

Though… I can’t tell what the last map is supposed to indicate. Any hints?

@Katy The map shows the birthplaces of the project students and external project mentors I’ve worked with since 2002. You’re right, a caption would help…

The slides are getting some attention at reddit.com/r/programming

It would be nice if you made a vide available

Sol: One of the (claimed) benefits of code review is that “subtle interactions between different layers” will be noticed as a bad smell and cleaned up before they lead to hellish bugs. “You’re not supposed to use that object like that; use this instead.” is the big win, because it has the biggest cost to fix later. Not that “you missed an array bounds check” is bad, but it should not be the primary purpose of the review.

Hard core test-driven development and Dependency Injection style programming also claim to help keep your layers clean. This idea appeals to me, but (as far as I know) the evidence is not as sound as for code review.

@Greg (and everyone:. Someone in the audience recorded the presentation, you can download the MP3 at:

http://craigagreen.com/downloads/gregwilson.mp3

(This isn’t my recording, got the link off of meta.stackoverflow.com)

Could you please provide more detail on the references you list e.g. Nagappan et al (2007) & Bird et al (2009)? What are the actual publication titles? Great presentation anyway

+1 on more reference details. I’m interested in the stuff on Greatest Hits V1.

By the way @Greg, if you expand this presentation into book form I’ll be the first to pre-order it on amazon

The topic is very fascinating to me

@James see http://pyre.third-bit.com/blog/archives/2910.html