The Third Bit

After cursing PowerPoint for eight solid months while building screencasts for Software Carpentry, I think I have figured out what I want instead. Here’s a top-down view of a PowerPoint slide (drawn, ironically, with OmniGraffle):

Here’s the same slide seen edge-on:

The vertical axis in this view is time: all of the objects in the slide are created when the slide is created, and are cleared away when we advance to the next slide.

When I’m presenting in person, I budget roughly three minutes per slide. I don’t worry about losing the audience’s interest by keeping a static image up for that long because I’m a fairly animated presenter: I point at things, walk back and forth, make eye contact, and so on, all to provide the motion and flow that my slides don’t. Many good presenters, though, prefer to keep things moving on screen as well, and PowerPoint supports that by allowing items to be revealed progressively. If we do that, the side view looks like this:

When I first started creating screencasts back in May, I quickly realized that my “static slide for three minutes” style didn’t work: unless something was changing on screen every 15 seconds or so, viewers’ attention would wander. I experimented with using progressive reveal in PowerPoint, but discovered three shortcomings. First, there is no easy way to sync my scripts (written in the notes portion of the slide) with the reveal of particular elements. Second, when slides are exported to PNG or PDF, the progression is lost—everything comes out together. Third, there’s no way for an element to span several slides: PowerPoint insists on a strict slide/object hierarchy.

The third point is the key to what I want instead. I want to be able to create “slides” like this:

Here, there’s no longer a notion of “a slide”. Instead, there are graphical objects with lifespans, and slides horizontally across the timeline that display whatever is “alive” at that moment. This model seems better suited to creating screencasts (or other continuous video), and I suspect it would be better for traditional live presentations as well (particularly if it was possible to modify the properties of objects as a function of time, e.g., to change an object’s location between one snapshot and another).

I think I could create something like this in an image editor that supported layers (one layer per object) and scripting (to turn layers on and off), though that would be a fairly clumsy user interface. A more elegant one doesn’t feel like it would be hard to design, but I’m hoping someone has already built this for me. If so, I’d welcome pointers.

Teaching

Yet another good post from Mark Guzdial pointed me at an article in The Economist about the value (or otherwise) of a PhD. Key stat (bold emphasis mine):

A study in the Journal of Higher Education Policy and Management…shows that British men with a bachelor’s degree earn 14% more than those who could have gone to university but chose not to. The earnings premium for a PhD is 26%. But the premium for a master’s degree…is almost as high, at 23%. In some subjects the premium for a PhD vanishes entirely. PhDs in maths and computing, social sciences and languages earn no more than those with master’s degrees. The premium for a PhD is actually smaller than for a master’s degree in engineering and technology, architecture and education. Only in medicine, other sciences, and business and financial studies is it high enough to be worthwhile. Over all subjects, a PhD commands only a 3% premium over a master’s degree.

Research, Teaching

The most interesting thing I read over the holiday was Falk and Dierking’s article “The 95 Percent Solution” in the Nov-Dec 2010 issue of American Scientist. (The article is behind a paywall at its source, but Google turned up this cached PDF copy.) The key stat is at the end of the first page:

Elementary-school-aged U.S. children perform as well as or better than most children in the world, but the performance of older U.S. children has been mediocre at best. Interestingly, however, for more than 20 years, U.S. adults have consistently outperformed their international counterparts on science literary measures…

Their explanation (which they back up with copious primary and secondary research) is that American adults have access to, and use, a “vibrant free-choice science learning landscape…filled with a vast array of digital resources, educational television and radio, science museums, zoos, aquariums, national parks, community activities such as 4-H and scouting and many other scientifically enriching enterprises.” Putting it another way, people do learn about science: they just learn it from Mythbusters, rather than in class.

There’s a lot of interesting stuff in the article (not least the observation that trying to reproduce what works in free-choice settings in the classroom doesn’t work). There are also lots of implications for computer science education…

Teaching

It’s a shame people only get meaningful academic credit for creating new knowledge (regardless of its importance), and not for explaining existing knowledge better than anyone has before. For example, have a look at Neil Brown’s “Ghost of Unix Past” series, or almost anything from the SysAdvent blog (such as Adam Fletcher’s look at what really happens when you run ‘ls’). If things like this were counted toward promotion and tenure, I think there’d be a lot more of them, but how to count is a hard problem. As Titus Brown recently posted on his blog:

This is the problem with the online world for scientists: there’s no real systematized incentive to any of this online stuff. And that makes it really tough. I’m going through Reappointment right now…Nowhere on there is there a place for “influential blog posts”—how would you measure that, anyway?

Teaching

I wasn’t particularly hopeful when I first heard about Grace, a new programming language being developed for teaching purposes. Having followed their blog for a while now, I’m positively underwhelmed:

1. Early in a 21st Century full of CAD tools and other rich ways to interact with complex data, Grace’s creators still insist on representing programs as sequences of ASCII tokens.
2. Their post on Nov 4, 2010, titled “Let’s Start With Syntax“, says exactly nothing about usability testing.

Unsubscribing and moving on…

Teaching

This video from Dr. Tae (whose first name is surprisingly difficult to find) is a half-hour grumble about the state of higher education. The key moment, though, comes at 3:57, when he asks the rhetorical question, “Would you pay tuition to sit in this room?” He never faces the answer head-on: yes, because it’s all I’d be able to afford. If you want to cut classes from 200 students to 20 students, you need 10 times as many instructors. There aren’t that many good teachers out there, but even if there were, someone (Dr. Tae’s fairy godmother, maybe?) will have to find ten times as much money to pay for it. And yes, there’s a lot of fat and waste at universities (don’t get me started on the number of time-wasters whose only accomplishment is to subtract value from everyone else), but even if there’s a 50% saving there (which there isn’t), someone would still have to stump up 5X the current budget to bring the kind of education he idealizes to the masses. Anyone who isn’t willing to address that issue directly and honestly is just blowing gas.

Teaching

Chapters are coming in for The Architecture of Open Source Applications. By coincidence, Tim Bray recently posted a nice, compact description of the architecture of Android, and there’s a quartet of articles in ACM Transactions on Computing Education on three programming environments for novices (Greenfoot, Scratch, and Alice) that include some architectural discussion. The latter are unfortunately stuck behind an influence-reducing paywall…

Architecture of Open Source Applications, Teaching

An interesting post from Carleton’s Prof. Nick Rowe about the economics of prof/student ratios (and other things). His exclusive use of male pronouns grates a bit, but the points he makes are good ones.

Teaching

Professors Leiserson and Amarasinghe teach a course called “Performance Engineering of Software Systems” at MIT.  According to this article, they bring in experienced programmers from industry to do code reviews on students’ projects. Why didn’t I think of that?

Teaching

Jen Dodd recently posted an article titled “3 rules for running events“, plus one metarule that I particularly appreciated: “Stop deluding yourself.” In the same spirit, I’d like to offer up three rules for running student programming projects. To set the stage, here’s the number of student programming projects I’ve organized, supervised, or otherwise been guilty of since David Wallace first asked me to look after a couple of summer interns in Edinburgh half a lifetime ago:

Yes, the numbers for 2008 and 2009 are crazy, but those are the years I ran consulting projects at the University of Toronto and started UCOSP. If you only count students I directly supervised, the numbers for 2008-09 drop back down to the high thirties—say, a dozen or so per term, three terms a year.

So what have I learned in those 23 years?

Rule 1: It’s Not Thirteen Weeks, It’s Three

This was the hardest one for me to learn, and it’s almost always the hardest to get across to both students and their clients. University terms may be thirteen weeks long, but students are usually juggling five courses, and many have part-time jobs as well. That means they can only put eight hours a week into their project without sacrificing grades somewhere else. If you figure a full-time work week is 35 hours, that means students actually spend 8×13/35 = a bit less than three weeks working for you. In that time, they have to:

• figure out what problem they’re actually going to solve,
• learn some new technologies,
• digest the existing code base,
• get to know their teammates,
• build something, and
• jump through whatever hoops are required for getting a grade, like writing a final report or some documentation that no-one will ever read.

That’s an awful lot to squeeze into three weeks: very few open source projects expect their GSoC students to start checking things in after three weeks of full-time work, but students in school are expected to be done in that time. Prof. Karen Reid says that she usually divides the term into three pieces:

I find that I spend the first 3 weeks working hard to get the students up to speed and essentially demanding that they get something real done in the first 3 weeks. In other words, my students are more successful if they push hard at the beginning. After that, they usually have a good idea of what they need to do for the remainder of the term and I can kind of let them set the pace. Then I spend the last 3 weeks defining what it means to be done.

There’s another catch lurking in here too. The iron triangle of project management is scope, schedule, and resources. In a student project, both the schedule and resources are fixed (13 weeks and N students respectively), so the only thing that can give is scope. There are two ways to reduce it: lower quality, or fewer features. Lowering quality is self-defeating—the students you want in a project course are the ones who take pride in their work and care about their grades (which aren’t necessarily the same thing), and they’re not going to like being told that the only way to pass a course is to produce crap.

That leaves the number and scope of features as the only free variable. Problem is, neither students nor clients are going to be excited about fixing a couple of minor bugs or adding one small new feature. If you want to get people on board, you have to aim higher, and be willing and able to reduce scope as the term goes on without making anyone feel like the project has failed—which brings us neatly to our second rule.

Rule 2: It’s Not About Technology

It really isn’t. When I ask students I’ve supervised in the past what they learned in their project, they never mention technology—never. They might have learned Ruby on Rails, or CUDA, or touch-screen interface design, or database performance optimization, but that’s not what they remember afterward. What sticks is how to run a project: how to run a progress meeting, review someone else’s code, manage their time, present their work in five minutes or less, and negotiate scope with a client.

I’ve tried teaching these things in regular software engineering classes, but it has never worked. (This is one of the reasons I have so little use for standard undergrad software engineering textbooks: you can talk about riding bicycles all you want, but the only way to learn how to do it is to do it.) On the upside, once I students understand that I’m trying to teach them process, rather than technology, the problems I mentioned in the previous section are greatly reduced: cutting the set of features we’re going to deliver, for example, becomes an exercise in scope negotiation rather than a failure on the students’ part.

So what goes into a rational student-oriented development process?

1. A weekly status meeting (face-to-face if possible, online if not). Whoever is running it (me for the first few, one student in turn thereafter) is responsible for drawing up an agenda and posting a summary afterward. They’re also responsible for checking that the previous week’s to-do items were completed, and for keeping the meeting on track (politely, but firmly). The first meeting each term usually runs 90 minutes or so; by the end of term, we can do them in 45 minutes or less.
2. Version control, ticketing, a blog, an archived mailing list, an IRC channel, and (most recently) code review—in short, the same infrastructure you’d use for a small open source project. You’ll note that “wiki” isn’t on the list: we’ve set them up in the past, but no one has ever made much use of them. You’ll also note that five of these six items are about communication—all six, actually, if you think of version control as a way to share files.
3. Demos and presentations. I emphasize this less when project teams are distributed across several universities, but if they’re collocated, I expect every team to present or demo weekly or every couple of weeks. I usually don’t give grades for each presentation or demo except to cure procrastination.

And that’s about it. On some projects, I’ll ask students to draw up a plan for the term at the end of their second or third week (i.e., once they’ve learned something about the problem—if they have to do it at the start of term, waterfall-style, all they can do is write some science fiction and hope I won’t hold them to it). On others, there’s some formality around handing off their code to their client, such as submitting it as a patch, doing a presentation at the client site, or showing off their work to all comers at a local pub.

Other people handle process differently, of course. Andrew Ross, of Ingres, says:

I tend not to have regular weekly meetings with my teams. Instead, we have meetings as needed to discuss things that can’t be covered acceptably in emails/IM’s/IRC/calls. We do the latter constantly. The more important underlying concept is keeping students from drifting away and losing contact.

Rule 3: Steady Beats Smart Every Time

I once had three students working on separate projects during the same summer term. Two had straight A’s; the third was struggling to maintain a low ‘B’ average, but he’s the only one I would have hired back, because he was the only one I could actually rely on. One of the ‘A’ students had spent his whole life acing exams, and didn’t know how to do anything else. He panicked when asked, “What do you think we should do next?” Literally—you could see his pulse race and his mouth dry out. The second had the same fatal flaw I had when I was twenty: he’d do the first three quarters of every job in record time, but getting the next 20% out of him was like pulling teeth, and the last 5% never got done all.

The third student, though, was as reliable as a grilled cheese sandwich. If he told me on Monday that something was going to be done on Friday, it was done on Friday; when I asked him, “Where are you?” he always gave me a straight answer: no “almost done”, no “just another couple of bugs” if he hadn’t actually started. It took me a couple of months to appreciate him, but once I did, I started looking for that same quality in every student I interviewed.

Of course, this isn’t to say that every student with low grades is a gem waiting to be uncovered, or that everyone with an ‘A’ average is unreliable. Far from it: grades are a fairly reliable indicator of ability and persistence, especially grades in courses that no one loves. But the correlation is a lot weaker than I, a former ‘A’ student, once believed.

Keep in mind that even the steadiest students will doubt themselves sometimes. Quoting Karen Reid again:

I find I spend a lot of time reassuring students who are climbing the learning curve. Having different levels of expectations for students depending on their background is something I have to explain to students used to the same evaluation standards.

And “steady beats smart” applies to supervisors as well as students. If you’re unreliable—if you miss meetings, promise to do things but don’t get around to them, or pretend to know more about technical matters than you actually do—your students will respond in kind. If you can’t or don’t commit at least 3-4 high-quality hours a week for each project you’re running, it would be better for everyone if you did something else. (This is, by the way, one of the many reasons I prefer team projects to individual ones: the number of hours required per project grows only slowly with the team size, at least up to half a dozen students, so you can reach more students without sacrificing everything else.)

And finally, a metarule:

Have Fun

Students won’t ever enjoy a project more than you do. After all, they have to do all the hard work, like tracking down bugs, while you get to do the fun stuff like argue over what it’s all supposed to do. And if you’re not having fun, they will quickly start to treat the project like just another course. It’s very hard to pull out of that downward spiral, so don’t get into it: no matter what happens, grit your teeth and have some fun. Go out for ice cream; borrow a projector and introduce them to Tron, WarGames, or Startup.com. They’ll remember that long after the course is over, too, and so will you.

Later: a recent study confirmed what most of us probably knew already: what makes people happiest (or saddest) are group events and achievements, not individual accomplishments.  Maybe that’s why students enjoy team projects, and come away appreciating most what they learned about teamwork rather than technology…

Student Projects, Teaching