Are Some Languages Better Than Others?

Stefik et al 2013: An Empirical Investigation into Programming Language Syntax

• First studied compared learnability of
  • Perl
  • Quorum (the language their team is building)
  • Randomo (a placebo whose syntax was "designed" by rolling D&D dice)
• Conclusion: Perl is as hard for novices to learn as a language with a randomly-designed syntax

We first present two surveys conducted with students on the intuitiveness of syntax, which we used to garner formative clues on what words and symbols might be easy for novices to understand. We followed up with two studies on the accuracy rates of novices using a total of six programming languages: Ruby, Java, Perl, Python, Randomo, and Quorum. To our surprise, we found that languages using a more traditional C-style syntax (both Perl and Java) did not afford accuracy rates significantly higher than a language with randomly generated keywords, but that languages which deviate (Quorum, Python, and Ruby) did.

Are Some Languages Better Than Others?

• Second study
  • More subjects and multiple assessment strategies
  • Languages in the C family are as hard to learn as a randomly-designed language
  • Ruby and Python are significantly easier
  • Quorum is easier still
• Reaction has shown just how little most developers know or care about the scientific method
• Discussed in this podcast

You Can't Ask Them

Altadmri & Brown 2016: 37 Million Compilations: Investigating Novice Programming Mistakes in Large-Scale Student Data

• Ask educators for learners' most common mistakes
• Compare their answers to data from the BlueJ Blackbox project
• Weak consensus among educators
• Weak correlation with observations
• Educator experience had only weak effect on results

We used the Blackbox data set to check whether the educators' opinions matched data from over 100,000 students and checked whether this agreement was mediated by educators' experience. We found that educators formed only a weak consensus about which mistakes are most frequent, that their rankings bore only a moderate correspondence to the students' data.

Let's Talk About Test-Driven Development...

Fucci et al 2016: An External Replication on the Effects of Test-driven Development Using a Multi-site Blind Analysis Approach

• 39 professionals working on real projects
• Replication of study done by other researchers
• No significant difference between test-first and test-last development

Method: We analyzed 82 data points collected from 39 professionals, each capturing the process used while performing a specific development task. We built regression models to assess the impact of process characteristics on quality and productivity. Quality was measured by functional correctness. Result: Quality and productivity improvements were primarily positively associated with the granularity and uniformity. Sequencing, the order in which test and production code are written, had no important influence. Refactoring effort was negatively associated with both outcomes. We explain the unexpected negative correlation with quality by possible prevalence of mixed refactoring. Conclusion: The claimed benefits of TDD may not be due to its distinctive test-first dynamic, but rather due to the fact that TDD-like processes encourage fine-grained, steady steps that improve focus and flow.

More About Diagrams

Cherubini & Venolia 2007: Let's Go to the Whiteboard

• Look at what developers draw when they're talking to teach other...
• ...and how well they can understand their own drawings hours or days later
• Diagrams are a cache for short-term memory, not archival...
• ...which may explain why UML hasn't caught on

Most of the diagrams had a transient nature because of the high cost of changing whiteboard sketches to electronic renderings. Diagrams that documented design decisions were often externalized in these temporary drawings and then subsequently lost. Current visualization tools and the software development practices that we observed do not solve these issues,

What Happens When Teams Go Agile?

Khomh et al 2012: Do Faster Releases Improve Software Quality?

• Looked at Firefox before and after the transition to rapid release and found:
  1. Users do not experience more post-release bugs
  2. Bugs are fixed faster
  3. When crashes do happen, they happen sooner after startup
• Still don't have an explanation for that last one...
  • ...which is how science progresses

We found that (1) with shorter release cycles, users do not experience significantly more post-release bugs and (2) bugs are fixed faster, yet (3) users experience these bugs earlier during software execution (the program crashes earlier).

Paradise Unplugged

Ford et al 2016: Paradise Unplugged: Identifying Barriers for Female Participation on Stack Overflow

• Only 5-6% of Stack Overflow contributors are women
• What do they find significantly more problematic than men?
  1. Lack of awareness of site features
  2. Feeling unqualified to answer questions
  3. Intimidating community size
  4. Discomfort interacting with or relying on strangers
  5. Perception that they shouldn't be "slacking"

...in online communities, such as Stack Overflow...only 5.8% of contributors are female.... Through 22 semi-structured interviews with a spectrum of female users ranging from non-contributors to a top 100 ranked user of all time, we identified 14 barriers preventing them from contributing to Stack Overflow. We then conducted a survey with 1470 female and male developers to confirm which barriers are gender related or general problems for everyone.

Open Source in General

Steinmacher et al: Social Barriers Faced by Newcomers Placing Their First Contribution in Open Source Software Projects

• Identify 58 potential barriers (including 13 social barriers)
• What matters most?
  1. How easy is it to get set up to make a contribution?
  2. How easy is it to find a task to start with?
• Other work has also identified "how warmly was my first contribution received?"

...our study qualitatively analyzed social barriers that hindered newcomers' first contributions. We defined a conceptual model composed of 58 barriers including 13 social barriers. The barriers were identified from a qualitative data analysis considering different sources: a systematic literature review; open question responses gathered from OSS projects' contributors; students contributing to OSS projects; and semi-structured interviews with 36 developers from 14 different projects.

There Is No "Geek Gene"

Patitsas et al 2016: Computer Science Grades Are Not Bimodal

• The "geek gene" is computing's most enduring and damaging myth
• But only 5.8% of course grade distributions at a large university were actually multi-modal
• And CS faculty are more likely to see distributions as bimodal if they think they're from a CS class
  • Even more likely if they believe some students are innately predisposed to do well in CS

We statistically analyzed 778 distributions of final course grades from a large... university, and found only 5.8%...passed tests of multimodality. We then... showed 53 CS professors a series of histograms displaying ambiguous distributions and asked them to categorize the distributions. A random half of participants were primed to think about the fact that CS grades are commonly thought to be bimodal; these participants were more likely to label ambiguous distributions as "bimodal". Participants were also more likely to label distributions as bimodal if they believed that some students are innately predisposed to do better at CS.