The Third Bit

One of the ways DrProject improves on Trac is role-based access control. A role is a set of capabilities, such as WIKI_VIEW or TICKET_EDIT, and users’ relationships to projects are represented by triples of the form (user, project, role). It makes administration a lot easier, and once Qiyu Zhu’s web-based role editor comes online, it’ll be easier still.

But nothing worth having comes without some kind of price. Suppose a portal administrator decides to delete a role—what should happen to people who actually have that role with respect to one or more projects? For example, if the MEMBERSHIP table looks like this:

then what should happen if the “viewer” role is deleted? Options include:

1. Nothing—leave the MEMBERSHIP table as it is. That seems bad because role values like “viewer” are foreign keys for the CAPABILITY table, and dangling foreign keys would make joins and other operations scary.
2. Delete all records from MEMBERSHIP that have the role being deleted. This effectively removes everyone who had that role from those projects (if you don’t have an explicit role, you have whatever rights the anonymous user has). This solves the dangling foreign key problem, but now the admin might have to go through all of the users in the portal and give them new roles in those projects one by one.
3. Only allow admins to delete roles that aren’t being used. This option forces admins to reassign people before deleting the role, so they’re easier to find, but still means a lot of hand-work.
4. Allow admins to reassign users’ roles during the deletion process.

We’ve decided to go with #4, and Liz Blankenship has mocked up a UI for it. Aninteractive version is online (follow the trail of green arrows), or you can check out the screenshots below. Comments would as always be welcome.

Step 1

Step 2

Step 3

DrProject

Liz Blankenship, our usability maven, has started blogging about her DrProject work. It’s turning into quite a summer…

DrProject

Kosta has posted some more thoughts about integrating IRC into DrProject. He’s still working on mechanisms, but I’m looking ahead to the user interface. In particular, how should IRC logs be displayed? His first pass will be a “magic” wiki page that conversation entries are appended to (or possibly one separate “magic” page for each IRC conversation topic). That will work for short projects, but:

1. What happens with something like DrProject itself, which could have tens of thousands of lines of conversation after a year has gone by? Even in undergrad course projects, there are likely to be thousands of lines in a term. Email is naturally segmented into chunks; what is the chunking (if any) for chat?
2. What happens to the event log? We clearly don’t want one entry in the log for each IRC chat message—what should we show instead?

Thoughts?

DrProject

Nick Jamil and Jeff Balogh have posted a fairly detailed design for DrProject‘s new ticketing system. As far as I know, there isn’t another one like it: when a project is first created, its ticketing is just a to-do list, but users can add extra fields (like “owner”, “due date”, or what have you) to meet their needs. I think the initial low cost will make ticketing more attractive to undergrad students; I’ll be very interested to see what information teams (undergrad, professional, open source, or whoever) choose to add when they’re given complete freedom.

DrProject

I left Toronto for Austin mid-day Wednesday, and got back at midnight last night. Lots happened in the interim, so here’s a linkandthoughtdump (which I bet actually is one word in German):

• Gave a talk about Beautiful Code to the Austin Python Users’ Group Wednesday at Enthought‘s swanky offices. (They’re the kind folks who provide web hosting for the Software Carpentry course.) About 27 people in attendance, and good discussion afterward; was grateful to Travis Vaught and Sergey Fomel for rides from the airport and to the hotel respectively.
• Gave another talk titled “HPC Considered Harmful” at the Texas Advanced Computing Center‘s Second Annual Scientific Software Days. I was a bit nervous about telling people at a supercomputing center that focusing on massive parallelism and peak performance is wrongheaded, but there were a lot of nodding heads.
• I made lots of notes from two other talks that I want to follow up on at some point:
  • Robert van de Geijn’s FLAME system lets you draw matrix operations, then automatically generates the corresponding high-performance code. It’s a great example of a real high-level programming tool for scientists (and yet another special case of what a real extensible programming system would support).
  • Eric Jones (also from Enthought) talked about a tool they’re building that watches changes to variables in Python programs, and automatically generates interactive plots of their values. It sounds simpler and less impressive than it actually is; I’ve asked him to put together a screencast, and I think you’ll be wowed—I was. (Later: Steve Eddings from The Mathworks sent me a link about data linking in MATLAB, complete with a video tutorial.)
• At roughly the same time, half a world away, Diomidis Spinellis presented a study comparing the code quality of Linux, Windows, OpenSolaris, and FreeBSD. Very cool work; wish I’d been at ICSE’08 to ask questions.
• Meanwhile, Dmitri Vassiliev, who is continuing his work on SlashID this summer, has discovered that generated code is next-to-impossible to debug. Not to be a one-note symphony or anything, but I said in that same article about extensible programming systems that the real challenge is not extending notation, but creating extensible debugging tools so that those notations and high-level representations can be fixed when they break. Robert van de Geijn doesn’t think FLAME needs a debugger; respectfully, I disagree.
• Science in the Open has a plea to scientists to make their raw data available, motivated by yet another irreproducible result.
• Kosta Zabashta has posted early thoughts about integrating IRC into DrProject. (Gray on black? Kosta…your design skills rival mine…) I need to tell him that DrProject’s RPC module doesn’t handle tickets because Jeff Balogh is going to replace the entire ticketing system with an extensible one this summer, using his Dojo Form Editor as a front end…
• Elisabeth Hendrickson has thoughts on automating tests for legacy web applications. Students, take note.
• Thanks to Nick Jamil and others, we have instructions for installing DrProject on Windows. Yay!
• Everything old is new again, including Ada and the Bletchley Park Colossus.

And then there’s this:

Thanks again to Sergey Fomel for inviting me down, and for introducing me to the reproducible research community—I’m looking forward to many more discussions.

Beautiful Code, DrProject, Python, Research, Software Carpentry

We’re pleased to welcome Liz Blankenship, a Season of Usability intern, to the DrProject team. Liz, a grad student in HCI, is going to help us redesign DrProject’s admin interface. Along the way, I’m hoping she can give us some advice on a few other things as well, chief among them the notion of the “All” project.

The background is this: Trac (DrProject’s ancestor) only allows one project per installation. We weren’t going to install it forty times to manage a class of eighty students working in pairs, so one of the first things we did back in 2005 was extend it to support multiple projects per portal. We then faced two questions, which we decided were related:

1. What project does a newcomer to the portal see by default (i.e., what’s “home”)?
2. How do reach everybody who has an account with a particular portal (e.g., to nofify them of impending downtime)?

Our solution was to say that every portal has an undeletable project called “All”. Every user of that portal is automatically a member of that project, so mailing “all@wherever” will reach everyone, and that project’s wiki acts as the portal’s home page. As a bonus, this also provides a logical place for people to file tech support tickets: if you need your password reset, for example, you could file a ticket against “All”.

It was nice in theory, but it hasn’t worked out that well in practice:

1. Lots of people find it confusing. This might be the project name (some people think that registering for “All” means asking to be put in all of the projects managed by a portal), but I don’t think that’s the whole story.
2. There turned out to be lots of reasons not to automatically include everyone in the “All” project. For example, external clients for undergraduate projects really don’t care about administrative matters; they only want email forwarded from their specific project, so we had to allow people to opt out and/or turn off email forwarding.
3. Almost nobody figures guesses “ticket All for tech support”. Instead, many users file tickets against “All” that should go against other, more specific, projects. Since we don’t have a way to move tickets from one project to another, this causes recurring irritation.

So what should we do? Options include:

1. Change the name of “All” to something like “home”. This would be (nearly) trivial, but it seems unlikely that a simple renaming will actually solve our problems.
2. Modify the URLs so that the project is a parameter, rather than part of the path; if a project isn’t specified, take the user to a default project. This would make the URLs harder to read aloud (a test I think any good web application should pass), but it would be more reliable than using URLs that might or might not contain a project name.
3. Get rid of “All” and:
   1. require admins to mark one project as the landing pad, or
   2. give people a 404 if they don’t specify a project when trying to connect.
4. Create a special kind of project for the landing pad with:
   1. a wiki that can only be edited by the admin,
   2. that automatically says something informative about all the public projects,
   3. and whose mailing list includes every user, but can only be sent to by the admin.

As you can guess, #4 is my current favorite. It could even be implemented without significant code changes, by defining a new role and giving every user who isn’t an admin that role in the default project. There’d still be the problem of its name, though.

So, what do you think we should do? And why?

DrProject

Our summer interns started this morning—we got Summer of Code, we got NSERC USRA, we got ITCDF, we got you name it, a lab and a half’s worth. I gave the least coherent welcoming speech of my life (bad cold, little sleep), our trusty sys admin Alan helped ‘em get their accounts set up, and whoosh, off they went.

And speaking off “off they went”: we have been very lucky these past few months to have David Wolever working full-time on DrProject. He and his fiancee are headed to Brazil this summer to do some digital inclusivity work, but they’ll be back in the fall. Safe journey; we look forward to hearing about your adventures.

• Cytoscape Graph Layout: Victoria Mui (GSoC)
• Documentation/Testing: Luke Petrolekas (GVW)
• Dojo Form Editor: Jeff Balogh (GSoC)
• DrProject:
  • Admin: Qiyu Zhu (GSoC)
  • Chat: Kosta Zabashta (USRA)
  • Miscellaneous: Nick Jamil (GVW)
• Eclipse Feature Diagram Plugin: Nicole Allard (CSC494)
• Web-CAT:
  • Eclipse Plugin: Geofrey Flores (USRA), Qi Yang (GSoC)
  • Python Back End: Eran Henig (GSoC)
• Flare Dataflow Editor: Ming Chow (CSC494/495), Wenbing Li (CSC494/495)
• Hackystat
  • Data Visualization: Eva Wong (GSoC)
  • Visual Studio Plugin: Matthew Basset (GSoC)
• OS161 Visualization: Xuan Le (ITCDF), Edward Robinson (CSC495)
• OpenAFS Control Panel: Joseph Yeung (OpenAFS)
• PyGraphics: Chris Maddison (ITCDF), Hardeep Singh (ITCDF)
• SlashID: Dmitri Vassiliev (CSC494/495)

DrProject, Teaching

We enabled web-based self-registration in DrProject a month ago; since then, 13 new people have signed up to get notices about new versions (or to file bug reports, which are very welcome).  At this rate, we’ll conquer the known universe in… um… carry the six… just a hair over 39 million years.

DrProject

Just read Elisabeth Hendrickson’s posts on next-generation functional testing tools, and thought the links (her articles, plus tools she finds intriguing) would be interesting to readers of this blog. Long story short, I’m trying to figure out whether DrProject should provide support for testing, and if so, what kind and at what level—no other portal does, so I can’t imitate prior art.

• Agile-Friendly Test Automation Tools/Frameworks
• Functional Test Tools: The Next Generation (1, 2)
• FitNesse
• ZiBreve
• Green Pepper Software
• StoryTestIQ (self-described as a mashup of Selenium and FitNesse)
• Concordion (“…lets you turn a plain English description of a requirement into an automated test…”)

DrProject, Research

One of the ways I can tell that DrProject is getting better is that its database schema is getting cleaner.  I’ve posted earlier versions from December 2006 (which was 1.0 something or other), Version 1.2 from early 2007, and the first Elixir-based version from July 2007. Today’s is much tidier, and I even know what most of the fields are for:

We have to add two or three new tables this summer, but I hope we’ll be able to keep things at least this clean.

DrProject