Research Threat Models

Richard Littauer and others are writing some guidance for researchers who want their code to outlast their present job. Their motivation is at least partly the unprecedented attacks on science by the Trump administration; to help, I’ve written the threat models shown below. I’d be very grateful for feedback.

When making plans, it’s useful to know what you’re planning for. Being explicit about threat models helps you prioritize, build consensus with colleagues, and check whether you’ve forgotten something important.

1. Individual threats are those that affect one or a few members of your team, such as a foreign student having their visa revoked without notice or needing to take extended medical leave. The most common way to prepare for this is to require everyone to document their work thoroughly, but that rarely works in practice:

   1. The hours spent writing those descriptions are hours not spent doing research, so people will always short the former to focus on the latter.
   2. People invariably fail to write down the “obvious” parts of their work that are anything but obvious to the next person. In practice, “automate what you can and create checklists for what you can’t” seems to be a better approach, particularly if you check those checklists by having someone else try to use them while their authors are still available to take notes and update them.

2. Leadership threats are individual threats that affect the project’s leader, such as being targeted by elected officials for pointing out the holes in the pseudoscientific conspiracy theories they are parroting. One way to prepare is to have a designated successor (who can also stand in for you if you ever want to take a holiday); another is to talk with peers about who will inherit what if your project is shut down.

3. Institutional threats are those that affect large groups at once, such as your university or professional association being blacklisted by government funding agencies. These scenarios are equivalents of major storms or forest fires in which so many people are affected at the same time that the rest of the community can’t absorb them. Regional and national governments handle disasters like these by having evacuation plans to get victims to safe(r) places, and corollary plans for putting beds in high school gyms and flying in food and emergency medical personnel to help people when they arrive. The equivalents in research are to lobby governments to change visa rules for scientific refugees and to set aside contingency funding to attract and support them.

One important thing about these threat models is their timescales:

• If you start making a checklist for publishing your team’s datasets today, you will be able to test it out (and see its benefits) in a week or two.

• If you want universities and regional or national governments to be ready to absorb and support an influx of smart, hard-working people despite the nationalist jingoism of populist politicians, you need to think in terms of many months.

• Finally, if you want professional societies to take something vaguely resembling principled positions, look at how long it is taking them to adopt open access (which is much, much less contentious) and settle in for a years-long fight.

Thinking about this stuff is scary, in part because most researchers have no personal experience with community organizing or disaster preparedness. Tessera Strategies are running a book club to help people learn about the former; the application deadline for participation in the first round has passed, but please keep an eye on their blog for future offerings.

As for disaster preparedness, almost everything you’ll find online starts from a place of individualized Hobbesian despair, i.e., assumes that there’s nothing you can do to prevent disaster and that it will be you against the world if and when one strikes. Professionals know that both assumptions are wrong: there are entire graduate programs devoted to creating, analyzing, and executing plans for disasters, and a great deal of research proving that most people work together for the greater good when the worst finally happens. Most of what I’m familiar with deals with natural disasters and their physical or economic aftermath; again, I’d be grateful for pointers to prior art that focuses on recovery for research.