Step 2: Build a schedule. Most students have never had to do this, so for some, it’s the most valuable part of the course. In order to explain how to go about it, I need to describe two important roles in real software projects: the product manager, and the project manager.

The product manager is the person who owns the spec. Typically, while developers are building Version N, she is talking to customers in order to find out what should go into Version N+1. She never, never asks them what features they want, because if she does, what she’ll get is a mish-mash of conversations overheard in frequent flyer lounges, and buzzwords plucked from the lead paragraphs of “Wired” articles.

Instead, she asks, “What can’t you do right now that you want to?”, “What do you find irritating in the current product?”, and, “Why are you buying our competitor’s software instead of ours?” She then translates the answers into a list of features to be considered for Version N+1.

(Note 1: the product manager usually also talks to developers to find out what they don’t like about the current software, and adds their wishes to the pile. Typically, these are things like “refactor the persistence layer”, “clean up the build”, and “upgrade to the latest version of Java”.)

So, it’s Monday morning. Version N shipped last Thursday (note 2: never, never ship on Friday); the team has had a weekend to catch its collective breath, and is ready to start work once again. (Note 3: if people are so burned out from the previous round that they need a whole week to recover, go and read Evan Robinson’s excellent article Why Crunch Mode Doesn’t Work: 6 Lessons.) At this point, the product manager divides up the list of desired features, and assigns them to the developers. Each developer then has some time—typically a few days to a couple of weeks—to do a little research, write some throwaway prototype code, and most importantly think. How could this feature be implemented? Is there a quick-and-dirty alternative that would take a tenth the time, but only deliver half of what was asked for? What impact will each alternative have on the build? On deployment? Will the feature be testable? And so on.

The outcome of this process is an analysis & estimation document, typically referred to as an “A&E”. There’s no set form for them, but usually, they include some background information (i.e., what the developer didn’t know before doing her research that she needed to learn in order to answer the question), a discussion of the alternatives, lessons learned from any prototyping that was done, and most importantly, an estimate of how much time would be needed to build each alternative. This time includes estimates from QA (for testing), the technical writer (for documenting), the people responsible for the build and creating the installer, and so on.

It’s Monday morning again. Three weeks have gone by, and all the A&E’s are done. When the time estimates are totalled, they come to 700 developer-days. Unfortunately, there are only 240 available: the size of the team is fixed, and the next release has to be available in May. This is normal. There is never enough time to add everything that everyone wants to a piece of software, and even if there was, it probably shouldn’t be done anyway.

So what do you do? The answer is, you find a large whiteboard and draw a 3×3 grid on it. The X axis is labeled “effort”; the Y axis, “importance”, and each is divided into “high”, “medium”, and “low”. (Note 4: finer-grained divisions, such as a 1-10 scale, are pointless, as they just encourage arguments over whether something is a 6 or a 7.)

Now, write each feature’s name on a yellow sticky note, and put it on the grid. (Or, if you don’t mind the smell of felt pens, write each feature’s number in the appropriate grid cell.) You should wind up with something like this:

You can probably guess what happens next. First, we throw away the high-effort, low-importance items in the bottom-right three cells. Next, we start assembling the other items into a schedule, starting with the upper-left corner. These are the things that will give the highest return on invested time; more importantly, starting with these means that if something goes wrong (and something always goes wrong), the team still has a good chance of having delivered something by the project’s end.

The items on the diagonal are the ones that have to be argued over. Should the team teackle Feature 14 (high effort, high importance), or Features 18-22 (lower importance, but less total effort)? It can take several sessions to sort this out; the most important thing is that management doesn’t shave the developers’ estimates in order to make things fit. If they do, then developers will start padding their estimates, which means management will shave even more, and the whole exercise becomes science fiction.

The team now has a schedule. For the rest of the development cycle, it belongs to the project manager. Her job is to make sure everyone is doing what they’re supposed to be doing, to handle interruptions (there are always interruptions), and most importantly, to track the team’s progress. After a few weeks, the project manager should compare how much has actually been done with how much was supposed to be done, and adjust the schedule accordingly. This does not mean, “Ask team members to start working nights and weekends.” (If you don’t know why, go and read Why Crunch Mode Doesn’t Work: 6 Lessons, and yes, it’s worth citing twice.) Instead, it means dropping some low-importance or high-effort features from the schedule, or replacing a full implementation with an easier-but-incomplete alternative. The product and project manager negotiate these issues as they arise, since they’re the two people who own the 3×3 grid’s axes.

Believe it or not, your customers will actually thank you for doing this, provided you do it early. “I’m sorry, we’re not going to have the frobnosticator for May 1″ is OK on October 1, or even January 1, since it gives whoever was counting on the frobnosticator time to make other plans. It is not OK on April 30; neither is saying (or worse, not saying) that it’s “done”, but full of bugs.

(Which brings us to a pet peeve. Engineering project management textbooks often say that there’s a tradeoff between schedule, resources, and features: if you fix the number of people working on something, and what features they’re to produce, that determines the schedule, and so on. Some people in the software industry claim that it’s actually a four-way tradeoff, with “quality” as the fourth attribute. That’s dreck: if a feature only works half the time, it isn’t done (or it’s only 50% done), so treating quality as an independent dimension is bafflegab.)

Back to CSC49X. The schedule is fixed at 13 weeks (minus 3 for warmup, writing the final report, etc.), and team members are only supposed to put in approximately eight hours a week each. It’s still worth writing an A&E, though, since there are still features to be prioritized, work to be divided up, and schedule slippages to be recovered from. Some A&E’s from previous projects are up on the web to act as guidelines. When it’s your turn to write one, remember: the most important thing isn’t the document, it’s what you figure out about how to build your software while writing it.

• Part 1

Another batch of CSC49X projects are due to start next week, so this seems like a good time to fulfil the promise I made two years ago to write up how the course is run, and why. (For readers outside U of T, CSC494/495 is a term-long project course, in which students work directly under the supervision of a professor.)
Step 1: Figure out where the goalposts are, so you know which way to kick the ball. If you’re an academic, this means finding something that is interesting and challenging enough to be publishable, but not intractable. In a startup, it means figuring out what you can build that people will pay for. Working for someone else? Check your job description (including criteria for performance evaluation and bonuses).

Life’s easier if you’re a student: all you have to do is check the marking scheme. Here’s what we’ll be using in 49X this fall, and why.

Warmup exercise: 10%

The purpose of the warmup exercise is to give students a chance to familiarize themselves with the tools they’ll be using in the rest of the course. Whether it’s Hibernate, , Flash, or something more exotic, students have to have some hands-on experience before they can start planning the rest of their term. The code students write for this is thrown away after being graded; they are strongly encouraged not to try to turn their finger exercises into finished products.

Analysis & estimation: 10%

Now that they know what they’re going to be working with, students can analyze their customers’ requirements, figure out what features will satisfy them, prioritize, and draw up a schedule. This is an important enough topic to merit a post of its own; until then, check out the lecture on development at the Software Carpentry site.

Code: 10%

Yes, that’s right: the code students write for this course is only worth 10% of their final grade, even though it’s where most of them will spend the bulk of their time. There are two reasons for this: (1) if they don’t know how to program, they shouldn’t be in 49X, and (2) if they don’t create some code, they can’t test, do a demo, or write their final report.

Testing: 10%

Testing is just as important as coding, so it’s given the same weight. Note, though, that only automated tests count: if I can’t check the project out of Subversion and re-run the tests (possibly after configuring a server or two), then as far as I’m concerned, the code hasn’t been tested. And it’s no good saying, “But I can’t write unit tests for my GUI,” because (a) if you design your program the right way, you can write ‘em for the back end, and (b) tools like Selenium can do a lot more than most students realize is possible.

Demos: 10%

I used to require students to prepare a 20-minute PowerPoint lecture on a topic of their choosing, and deliver it to either their coursemates or a junior class. It was a valuable experience, but it ate up a lot of time. This summer, I took a page from David Crow‘s DemoCamp playbook, and had students do 10-minute demos instead. It worked well: in most cases, the first one (which isn’t graded) was shaky, but the second (which is) was slick, so I’ll stick to this format in future.

Teamwork: 10%

Most of the students I take on are mature, responsible, and polite; the teamwork mark is the stick I use to beat the ones who aren’t. Everyone starts with 10 out of 10, but if they only do their work at the last moment, or check in code that breaks the build, or are rude or condescending in email, marks come off.

Final report: 40%

Students deliver software to their customers; they deliver their final reports to me. Each term, I tell them to write the document they wish they’d had when they started the project, so that whoever inherits their work will have an easier life. What does it do? Why does it do it that way? How is it structured? Built? Tested? What went well? What should the next person or team avoid like stinky cheese? Most importantly (for me, anyway), what have the students learned from the whole experience? Past reports have ranged from 8 pages to almost 50; these days, I look for 8-10 pages per student (i.e., roughly 25 pages for a 3-person team). If time permits (i.e., if students start work early enough), they can submit their report once to get feedback from me, revise it, and then resubmit for marking. This usually makes at least one full letter grade difference in the mark, which gives teams yet another reason to plan their time carefully.

This grading scheme is labor-intensive: I probably spend 6-8 hours reading and grading each project in a term, or roughly 50 hours in total. I’ve thought several times about using peer grading to reduce my load (and give students some experience of what life is like on the other side of the red pen), but I’ve never been able to convince myself that it would actually work.

A screencast for the online marking tool (built using TurboGears and AJAX) is now up. It’s a very cool tool — I’m really looking forward to seeing how students and TAs react. I’ve also relocated the DrProject screencast to the department’s server in order to reduce the load on ours. We’re just days away from the 1.1 release, with performance enhancements, web-based administration, tag clouds, and more — stay tuned.