Category Archives: DrProject

Getting Ready to Wag the Dog

A conversation with an ex-student yesterday reminded me of a post I’ve been meaning to write for at least a few months. He’s now a user experience designer, and believes that with a new generation of natural interfaces hitting the market (voice, tabletop, etc.), software developers will have to involve designers early and often, or suffer massive failure.

My counter is that people who care about UXD said the same things when GUIs first hit the mass market in the 1980s and when the web exploded in the 1990s Both times, though, the bulk of the software industry carried on treating design as an afterthought or luxury, in part because of intertia, and in part because companies that didn’t spend money on design seemed to do just as well in the market as those that did. (Remember the first few years of eBay and Hotmail? Brr…) If you’re going to claim that “this time will be different!”, you’d better have more than a wish-it-were-so argument to back it up.

My ex-student accepts that software developers have been able to slack on design because of the different costs of physical and virtual change. If you want to change the shape of the arms on a pair of sunglasses you have to cast new dies and tweak the assembly line, both of which cost tens of thousands of dollars (at least). If you want to change the layout of a few menus, on the other hand, you can just rewrite a few lines of code, recompile, deploy the new JAR files, and hey presto, you’re done. Yes, the real long-term cost of changes to software is much greater than that, but the difference in malleability is still so great as to be a difference of kind rather than degree.

I think we’re about to see a fundamental shift in the malleability of physical things, and that as a result, we’re about to see the software design tail start to wag the physical manufacturing dog. 3D printing and mass customization are at the same point now that desktop printing was in 1985: hardware was temperamental, and still out of reach of casual home users, but had suddenly reached the point where you could get a laser printer and a Mac for about what your cousin was spending to buy a delivery van for her flower shop. I was there—in the summer of 1985, I was programming a first-generation Mac with a three-digit serial number, and showing my brother (an industrial design student at Carleton) how MacWrite and MacPaint could rock his world. I believe this is about to happen to a hundred different kinds of manufacturing: plastic children’s toys, cutlery, sunglasses, door handles, and just about everything else made of plastic or aluminum alloy that’s the size of your fist or smaller.

What I think will happen then is a reversal of what designers working in the software industry have been calling for since the 1970s. Instead of software design becoming more like “real” engineering, we’re about to see the latter become more like the former. If sunglasses are printed on demand in the store, and changing the design is as simple as uploading a new spec from the web into the printer, then people designing sunglasses will be under exactly the same pressure to “fix it today” that software designers have been dealing with since the advent of timesharing terminals. Five years from now, I predict that software designers who’ve been griping about never being given enough time or enough respect will be on the lecture circuit teaching their ID and IE counterparts how to be agile, live with chaos, and cope with design cycles measured in hours rather than months.

Long story short, I think that anyone who’s trying to get software firms to act as if they were designing for a 1950s-era assembly line is betting against history. I think that if you want to reshape design today, you should put your money on the barbarians on horseback with their funny-looking stirrups and their recurved bows, not at the orderly legions drilling in their tidy little camps. Whichever way it goes, there’ll be a lot of noise and confusion—but that’s just another way to say, a lot of opportunities for people who are willing to turn things upside down.

Counting Down to 0.2

As I’ve mentioned previously, we’re moving DrProject over to Django.  Work started in September, and the second release is coming up soon.  The schedule highlights some of the differences between doing development with full-time developers, and doing it with students who are working 1/5 or 1/4 time:

  • Friday March 20: feature freeze.  Anything that adds functionality must be up on review board by the close of business.  Anything that’s posted in rough shape just to get it in, or without tests, will be rejected.
  • Friday April 3: code freeze.  Integration, testing, and bug fixing must be wrapped up by the close of business.
  • Thursday April 9: release.  We’ll spend the week between code freeze and release tidying up, asking friends to test our installers and setup instructions, etc.

I’m pretty pleased with the state of the reworked code; looking forward to reactions from the rest of the world.

Code Sprint Wrap-Up

I really did mean to blog several times a day about this term’s code sprint, but too much was happening during the day, and I was too tired at night. There were lots of high points:

  • Andrew Trusty, Andrew Louis, Andrey Petrov, Severin Gehwolf, and Blake Winton came out for part or all of it to help, just because. Thanks, guys.
  • The PyVCal team, who are building an abstraction library on top of Perforce, Subversion, and Git, got some scripts working with all three, and Derek Kwok started building a mock implementation of the API to use in testing.
  • The OLM team made huge strides forward: marking, some major simplification of their data model, and lots more.
  • Basie (the rebuild of DrProject on Django) also made huge strides: we have charts (using Flot), a partly-AJAXed ticketing system, user preference pages, lots of new UI design, more wikification, and lots of other things that I’m forgetting right now.
  • Karen Reid and I started using Twitter. But we could quit any time, honest.

Many thanks once again to Alan Rosenthal, Vlad Sekulic, and especially David Wolever for tech support, and to Jason Whyne, Liz Blankenship, Heather Grant, and James Leung for coming all the way to Toronto to hack for three days.

Further commentary from Mike Conley, Phyliss Lee, Aran Donohue, and Andrew Louis.

Things I’d Like To Finish In the Next 489 Days

One of the things I teach my students is that the real purpose of a schedule is to tell you when to start cutting corners and dropping features. The ticker on my web site tells me I have 489 days left in my contract with the university; I signed up hoping to study ways of teaching second-stage novices [1] how to be better programmers, but after four failed attempts to get NSERC funding [2], it’s time to lower my sights. Here are the things I’d like to finish off before my stint at U of T is over:

  1. Help Samira Abdi, Jeremy Handcock, and Carolyn MacLeod finish their Master’s theses, and get Aran Donohue, Alecia Fowler, Alicia Grubb, Zachary Kincaid, Jason Montojo, and Rory Tulk through theirs.
  2. Publish Practical Programming (the “CS-1 in Python” book that Jennifer Campbell, Paul Gries, Jason Montojo, and I have been writing). It’s currently in beta, and due for release in a month or so; we’d like to do a Python 3 update in a year or so, but that’s likely to slip.
  3. Finish the study of how scientists actually use computers. Data from the initial survey is now being processed; we’ll put together a follow-up survey in the next couple of months, write a “popular science” paper for American Scientist in the spring, present results at the SECSE workshop in Vancouver in May, and submit a paper by year’s end.
  4. Co-edit a special issue of Computing in Science & Engineering on “Software Engineering and Computational Science”. Andy Lumsdaine and I have four articles lined up, and are looking for two more—if you’d like to volunteer, please give me a shout.
  5. Submit a proposal for a professional master’s degree in Computer Science to U of T’s School of Graduate Studies. This is mostly a matter of filling in forms, but that’s kind of like looking at Everest and saying, “It’s mostly a matter of going uphill.”
  6. “Finish” a much-improved DrProject. I originally planned to use it as a platform for research, as well as teaching; there isn’t enough time left for that, but I still hope to make it easier for software engineering instructors to introduce students to modern tools.
  7. Rewrite Software Carpentry. Tina Yee has translated some of the lectures into MATLAB; the next step is to make the whole thing look like it was written in the 21st Century [3].

Everything else has to go by the boards. In particular:

  • I have resigned from my contributing editor post at Doctor Dobb’s Journal. It was a lot of fun, and I really enjoyed working with Jon Erickson, but as I said back in October, I’d rather not do it than do it badly.
  • The software developers’ reading group I’d planned to start this January isn’t going to happen. I’d really like something to pick up the slack now that DemoCamp seems to have stalled (if only to provide an excuse to get together with former students on a regular basis), but someone else is going to have to organize it.
  • After this term, I’m going to stop supervising student projects (except those directly relevant to DrProject and/or Software Carpentry). Next to 10:00 am coffee breaks with the lecturers, this is the part of university life I enjoy the most, but there just isn’t time… :-(
  • The Software Project Coloring Book (my attempt to write down everything I try to teach undergraduates about real-world software development) is being put back on the shelf. I have written 35,000 words, but those were the easy bits: conservatively, I’d need 4-6 months of full-time work to finish it off.

On the upside, Sadie got me some biking gear for Christmas, so now I’ll have to shed the twenty pounds I’ve picked up in the last couple of years, and I get to start taking our daughter to music classes every week. To quote a friend, it isn’t what I planned—it’s better.

[1] People who already know how to write programs, but not how to develop applications. I’m specifically interested in undergraduate Computer Science students, and graduate students in other disciplines.

[2] Companies like Nitido, the Jonah Group, Idee, and Rogers have kindly donated a few thousand dollars each to keep things like DrProject going, as have several of my fellow professors, but a $24K grant from The MathWorks is the only “research” funding I’ve been able to raise.

[3] As I said yesterday, I’m looking for a mentor in the Toronto area who can show me how to do this.

Post-Mortem on This Term’s Work

As I’ve mentioned several times, we’ve started rebuilding DrProject on top of Django. We had a post-mortem last Tuesday on our first term’s work, which Blake Winton was kind enough to summarize. The highlights (good and bad) are listed below.

Good

  • Everybody gave more than expected/requested/paid for.
  • The team produced some remarkably clean code in a very short time, especially given its prior lack of exposure to Django.
  • Distributed collaboration worked much better than expected. (The team was spread across four universities in three countries.)
  • The dev and commit mailing lists worked well, as did weekly status meetings on IRC.
  • Regular, frequent, small commits went well/were a good idea.
  • pyflakes and pep8 were great tools for checking code and style (but it would have been nice to have a script to run both on all files in one command).
  • Code reviews rocked. (Author’s note: this was the best part of the term for me, and something I’m going to want to encourage in my undergrad software engineering class next term.)

Bad

  • Tempers were strained occasionally during reviews and design discussions (more considerate language should have been used).
  • Too much juggling people from one sub-project to another. (Author’s note: this was my fault.)
  • svnmerge was a pain. (Author’s note: one developer was a real fan of distributed version control systems, and kept trying to move us in that direction. I should have resisted more strongly.)
  • The project blog wasn’t used effectively—it was never clear what should go in the blog vs. on the mailing lists.
  • People sometimes didn’t know what they should be working on. (Author’s note: also my fault—next term, the newcomers will be given a couple of short, specific tasks at the start to get them up to speed.)
  • The release process kind of fell apart, the code freeze never really happened.

Thanks, Blake

And while we’re wrapping up the term: when we started rebuilding DrProject on top of Django in September, Blake Winton volunteered to help out.  Since then, he has done detailed line-by-line code reviews on over half of our 950 commits, been a voice of reason during online design discussions, and shown the whole team (myself included) how a mature, even-tempered, and good-humored developer gets things done.  We wouldn’t be anywhere near as far along as we are without him, so thanks, Blake—we really appreciate it.

And Search Returns…?

Here’s a design challenge for anyone who wants it: what should search return?  More specifically, suppose you have something like DrProject, which stores tickets and wiki pages in a relational database, and revisions to files using a standard Subversion repository. Tickets are identified by ticket number (it’s actually the pair [projectId, ticketId], but never mind that for now), while wiki pages are identified by their names (which are strings), and repository entries are identified by a combination of path and version number (e.g., [/trunk/license.txt, 33]). If one ticket, one wiki page, and one file contain the word “Turing”, what should ‘search(“Turing”)’ return?

Should it be a set of URLs?  I need those eventually if I’m building a web page to display the results, but that feels like mixing two levels—my instinct is that I should be returning “things” that are then converted to URLs.  (For example, if I’m running a command-line admin tool on the desktop, I probably don’t want URLs coming out of my search function…)

What about returning a set of pairs, where each pair is [type-of-thing, thing-id], e.g., ["wiki", "Authors"] or ["ticket", 127]?  That feels wrong because it will force all the client code (e.g., the stuff that creates URLs for HTML pages, and the command-line admin tool) to do a type-switch in order to fetch and process the actual items.

OK, so what about defining an abstract base class called ‘SearchResult’, then require each component of DrProject to subclass it, so that we have ‘TicketSearchResult’, ‘WikiSearchResult’, and so on?  These classes could provide ‘getItem()’ methods to fetch the content that matches, ‘toUrl(baseUrl)’ methods to create links, and so on.  That feels like a better fit than either of the previous ideas, but I’m still dissatisfied.  Is there a cleaner way to do this that I just don’t know about?  If so, I’d welcome a pointer…