I gave a talk at PyGTA last night on what we’re doing to integrate Python into the undergraduate curriculum at the University of Toronto, and what I’ll be doing with my PSF grant to promote Python in science and engineering. I was pleased at how many people turned out, given the weather, and thought everything went well…

…someone asked what I thought about Extreme Programming (XP). It was late, I was tired, so I ran off at the mouth a little. (I do that.) Here’s what I should have said:

• I’ve never used XP on a real project, so take my opinions with a large grain of salt.
• That said, I’m sceptical of the claims made about it, partly because they fly in the face of my personal experience, but also because the Scot in me instinctively mistrusts anything that’s hyped as hard as XP has been.
• Most importantly, I don’t think that the particular practices that make up XP (pair programming, on-site customer, stand-up meetings, etc.) are the real reason it’s successful. The reason is that teams that adopt diametrically opposed methodologies, like CleanRoom, also see their productivity go up. One possible explanation is that common practice is the worst of all possible worlds, and any change at all would be an improvement. (There are days when I believe this.) A more likely explanation is that what really matters is deciding that you want to be a better programmer. If you make a sincere commitment to that, then exactly how you get there is a detail. It’s kind of like dieting: Atkins, South Beach, macrobiotic, seasonal, or fruitarian is secondary to being sincere about eating better and exercising more.

This hearkens back to a point I made a couple of times during my talk. It’s easy to make students jump through hoops in a course. What’s hard is convincing them that jumping through those hoops after the course is over really will make their lives better. The best way I’ve found so far is to bring in experienced programmers who are doing exciting things, and have them say, “Comments, version control, test-driven development…” I’d be interested in hearing what other people do to make the case.

A web-friend of mine just interviewed for a tech lead position at Google. Here’s a (slightly tidied up and anonymized) version of their experiences:

Most of my work, at least at the start, should be in “production software”–googlese for the software that helps keep Google’s amazingly huge distributed system running smoothly and seamlessly, and is mostly Python though with ample helpings of C++ here and there and a little bit of Java where integration is needed with some Java-centric application server (e.g. to serve google-ads on sites using such servers).

Plenty of “sideshows” doing such things as statistical analysis and data mining on the huge wealth of data Google collects, maybe giving [name deleted]‘s team a hand in data-quality assurance, etc, etc. Plus, every Google techie is supposed to use 20% of his time working on his or her own pet projects which might become Google’s Next Big Thing—that’s how gmail was born.

The selection process is grueling—multiple rounds of phone interviews where they ask you (depending on the fields of expertise you claim) everything from what’s 2¹⁰, to how you would tweak bits in C to find out if a machine’s stack grows up or down in memory, all the way to having you “program on the phone”… then all of a sudden they rush you to Silicon Valley and you get a long full day of nonstop interviewing. I didn’t quite ace mine because I hadn’t thought of cramming on TCP/IP fundamentals, so I didn’t remember which bits are on in the three packets of the handshake (it’s SYN, SYN+ACK, ACK—I could have worked it out, but not jetlagged and after about 6 hours’ interviews .

I made up for that when they had me program at the whiteboard a C++ implementation of unbounded precision multiplication; I did a test-driven implementation of the trivial routine with std::vector<digit> containers, then did some handwaving about the Karatsuba algorithm (far too hard to implement standing up at a whiteboard, of course and could sense I had struck lucky… The guy interviewing me at that time had never really done unbounded precision computation work (at least not implementation of high-quality libraries for it), so by just opening the door a crack to the huge and mathematics that underlies that field (in which I had the good fortune to dabble a bit — a byproduct of my interests in combinatorial arithmetic) I had apparently exceeded expectations.

Lots of back-of-envelope computation and the like, too. A friend of mine thought he was doing well in his second Google phone interview when asked to sketch a way to compute bigram statistics for a corpus of a hundred million documents—he had started discussing std::map<std::string> and the like, and didn’t get why the interviewer seemed distinctly unimpressed, until I pointed out even if documents are only a couple thousand words each, where are you going to STORE those two hundred billion words—in memory?! That’s a job for an enterprise-scale database engine!

So, at least as far as the interviewing process goes, it seems designed for people with a vast array of interests related to programming, computation, modeling, data processing, networking, and good problem-rough-sizing abilities—I guess Google routinely faces problems that may not be hugely complex but are made so by the sheer scale involved. I can just hope the actual day-to-day work is as interesting, fascinating and challenging as the interviews were—but from all I hear, it probably is. And they have bar-quality espresso machines in rest areas…

A couple of years ago (summer of 2003, actually), I noticed something that’s been nagging me ever since. When I log into a computer, the first thing I fire up is email. When my students log in, the first thing they run is an instant messaging client. They keep IM running all the time, just as I keep my email client up, and whenever they run into something they don’t understand, they’re more likely to ask one of their buddies than to ask Google.

Now, I don’t know how this is going to change the way people develop software, but I know it will, just as email did. Just today, for example, I:

• asked a guy in Germany whether the documentation for his open source library was accurate;
• searched the archive of the Python Developers’ List to remind myself of something I worked on in 2000;
• sent a stack trace to another developer in Boston;
• arranged meetings with three different teams;
• apologized for missing a meeting with a fourth team;
• triggered a complete system build by sending email to a daemon process running on a machine in B.C.; and
• received a reply telling me that some code I’d checked in didn’t compile on BSD (missing header files…).

None of this would have happened if my generation didn’t take email for granted: it might have been technically feasible, but we just wouldn’t have thought to do it. The teenagers and twenty-somethings in my class take IM for granted; the only thing standing in the way of them being really creative with it is that dinosaurs like me still make the rules. What will software development projects feel like ten years from now, when they’re in charge? If anybody knows of creative uses of IM in software development, I’d be grateful for pointers.