What I do is get a quick understanding of the products, then start asking questions. I use the chapters of “Practical Development Environments” as a guide, since they were written with this in mind. For example, version control – number of developers, number of sites, time for checkout for developer, release builds, number of branches, branching and merging strategies, etc. Then on to the build system, testing, documentation, automation, communication. You get the idea. It usually takes about an hour for 2 to 4 people. There are other more specific questions depending on the kind of product (desktop, web) and domain (networking, libraries). Then I summarize my understanding and email it to the potential client, usually with some recommendations, and we go from there.

However, it’s only once I’ve looked at source code, build files, build breakages, test coverage, the release process and internal documentation that I start to get a feel for the health of a project. Some things are very subtle – who are the technical leads, do they work well with each other, how does the rest of the team view their direction.

Now, to answer your real question. How would I evaluate teams of people learning to develop software together? I’m going to assume that the projects are a few months in duration and that guidance in the form of the previous year’s best projects and your comments on them is available. I’d want to see evidence of thought in how the project’s code is structured, and appropriate comments are important to everyone involved. Tracking bugs is a good sign, along with their resolution. Unit testing the harder parts and regression tests are excellent. Writing a good user guide or online help shows good taste.

I guess I like the idea of half the points for getting there and half for how you got there, but no-one learns in a vacuum so much of this depends on how the examples of good development are presented.

- Paddy.

If you mean “a team who is delivering salable product,” then I’m more interested in how they’re meeting deadlines and producing shippable (meaning: enough features and few enough bugs to satisfy customers) software than I am about the state of the codebase.

If you mean “a team making maintainable, solid code,” then code reviews are probably the best thing you can do. Is the code readable, is the documentation adequate, are there unit tests, do the various team members generally understand all the code and where things are.

It matters because there’s beautiful code (sorry for the reference!) that doesn’t the get job done or isn’t delivering features quick enough, and there’s successful commercial teams with less than stellar code but who are doing exactly what they need to be doing.

You could argue that sacrificing code “goodness” for the sake of delivering features isn’t wise in the long run, and I would agree if the software is supposed to be in a mature place in its lifecycle, but if it’s new software the equation isn’t the same and it might indeed be the correct trade-off.

I assume the projects are customer driven, rather than simply make-work tasks? Customer satisfaction should play a role in the evaluation. Did they meet the requirements? Were they on time? Did they cut appropriate features (in consultation with the customer) to make a deadline if they *weren’t* on schedule at some point?

Self-evaluation is important, too, even though it is hard for newbies to be objective (as you point out). Identifying their mistakes and identifying actions they could have taken to avoid them, even after the fact, shows that they have learned something. How could the team have worked better together? Which of the practices that they learned about in class might have made the project more successful?