Advice

I’ve collaborated on a handful of papers that give practical tips on building research software and teaching others how to do it, all of which are available online:

I also have some shorter guidelines for:

Being Fired

These ten rules are based on my experience with DataCamp and on the experiences of friends and colleagues there and at other companies.

1) Insist on a record of all conversations,
because the biggest mistake you can make is to assume good faith on the part of those who fired you. In most jurisdictions you have a right to record any phone calls you are part of, and if that feels too confrontational, insist on communicating by email. If they insist on communicating by phone or video call, follow up immediately with an email summary and make sure you send a copy to your personal account.
2) Pause before speaking, posting, or tweeting.
If possible, have someone you trust look everything over before you say it or send it. (Don’t use someone who still works for the company, even if they are your closest friend: it puts them in a legally and morally difficult position.)
3) Keep your public statements brief:
people may care, but most won’t care as much as you do. A simple recitation of facts is usually damning enough.
4) If you want to correct something online, add a timestamped amendment:
do not just take it down or edit it. If you do, you will be accused of rewriting history, and muddied waters only help whoever fired you. (Also, be prepared for them to dig through everything you’ve ever said online and re-post parts selectively to discredit you.)
5) Speak directly to all the issues rather than omitting or ignoring things you’d rather not discuss.
Your honesty is your greatest asset, and it’s hypocritical to criticize your opponents for spin or selective reporting if you’re doing it too.
6) Don’t sign any agreement that might prevent you from speaking about moral or legal concerns,
or make sure the agreement explicitly excludes your concerns before signing it. (And yes, it’s very privileged of me to be able to say this: someone whose immigration status, essential health benefits, or family income is being threatened may not have a choice. That is why I think people who do have a choice also have an obligation to fight.)
7) Don’t cite the law unless your lawyer tells you to.
It probably doesn’t mean what you think it means, and they almost certainly do have lawyers on their side who will seize on any mis-statement or mistake you make.
8) Don’t try to get them to acknowledge that they were wrong.
This probably wouldn’t have happened if they were the sort of people who could.
9) Go for long walks, cook some healthy meals, pick up the guitar you haven’t touched in years
anything that requires you to focus on something else for a while. This isn’t just for your mental health: exhausted people make poor decisions, and you need to be at the top of your game.
10) Remember that it’s OK to cry
and that there’s nothing shameful in asking for help.

Changing Organizations

1) Be sure this is where you want to focus your efforts.
It’s going to take years, it could well fail, and there are many other things you could do, so be sure this particular change is the one you want to fight for.
2) Ask those who will be affected.
Nihil pro nobis sine nobis (nothing for us, without us) is always a good motto, and asking for opinions often reveals potential allies.
3) Be specific.
Few people would argue in public against fair hiring practices, but the organization has to implement something specific in order for that to mean anything, and people very well might argue against those specifics. You should therefore pick something specific and achievable and make a concrete plan for achieving it. And start with whatever is likely to have broadest support, because success breeds success.
4) Figure out who actually has the power to make that change and what they actually care about.
Your neighbors don’t make policy for your local public school: school board trustees do, so that’s who you need to influence. Help someone who wants the same change as you get elected, or help someone who doesn’t oppose your change with something they care about in exchange for support for your cause, but whatever you do, do it for the people whose vote counts. Conversely, figure out who is going to be negatively impacted by the change you want and what you can do to help them: for example, if it’s going to eliminate jobs, what else can those people usefully do?
5) Build alliances,
because “I’ll help you if you’ll help me” makes the world go around. It’s hard when people want the same thing for very different reasons, but this rule is not cynical: people whose beliefs are aligned may still have different priorities. (The flip side of this rule is to accept that some people will never be your allies: if everyone wanted it, you wouldn’t have to push for the change.) People who want the same thing you do, who have a high profile (either inside your group or externally), or who have a wide range of connections are all useful allies.
6) Test the waters.
At every stage, refine your idea and presentation in front of a small group first. But remember: when done honestly, “refining your idea” sometimes means accepting that you wanted the wrong thing. And note that it can be useful to ask someone to be the official skeptic: giving them a way to critique in private may temper their public criticism, and you just might convert them.
7) Keep it visible.
It’s easy to start a blog and create a Mastodon account, but that very ease has reduced these channels’ impact. Is there a newsletter you can be included in (which makes you look more official)? Can you make a presentation as part of some other event (rather than organizing a gathering of your own)? Can you get someone with a higher profile to mention what you’re doing and point people your way? Can you post notices in the lunchroom? The elevator? The washrooms? Local restaurants? Some people will do things for t-shirts and stickers that they won’t do for money… And always share a single point of contact that someone actually checks (frequently).
8) Collect data but tell stories.
Sooner or later someone is going to ask what financial impact this change is going to have, so be ready for that. Data about other changes or from other organizations helps a lot, but data doesn’t have nearly as powerful an impact as stories. Don’t explain what kind of people this will help: explain how it helped a specific person, or how it would make a specific person’s life better. And if you’ve never cried when telling that story, tell a better story.
9) Learn how to run meetings and make decisions.
You can get the change you want without being a great public speaker, but you must know how to run a meeting and make decisions.
10) Celebrate when you can, grieve when you need to.
Burnout is an occupational hazard for everyone trying to make meaningful change, in part because we get so used to doing things on our own at the start that we don’t share the load even when there are people to share it with. Not all of your allies will become your friends, but those who do will be able to share your victories and commiserate with your defeats like no one else.

Being a Good Research Partner

Data scientists often have to bridge the divide between academic research and commercial practice. The ten simple rules listed below may help mitigate the frustration you encounter as you try to o this.

If you are a researcher in academia…

1) Remember that companies work in weeks, not seasons.
Academic semesters are rooted in the seasons of an agricultural era, but practitioners in industry have to work at a more accelerated pace. In the time it takes you to write a grant, a company might develop and release two new versions of their product in order to keep up with their competition. Discuss timescales with your industrial research partners early on, and be realistic about how slowly things will proceed.
2) Be open.
Research is of no use to practitioners who cannot easily find it and read it. While Jimmy Wales (the founder of Wikipedia) may not actually have said, “Open information drives out closed,” the principle holds: with so much information freely available on the Internet, any paywall or login barrier put between you and your hoped-for audience will send a large number of people elsewhere.
More importantly, these barriers send a clear signal that you do not care if practitioners read your work or not: as one colleague observed rather sourly, it’s the equivalent of inviting people to your house for dinner and then expecting them to pay for the drinks.
3) Value action over insight.
The goal for practitioners is not to understand the world, but to change it. “We know X” is much less useful to them than “we can do Y”. When presenting your findings, you should therefore focus on how someone might act on it.
One way to do this is to add slides titled, “What Difference Does It Make?” at strategic points in your presentations. If you can’t think of what to write next, you may want to rethink what you’re focused on.
4) Don’t hesitate to sacrifice detail for clarity.
Understanding doesn’t have to be complete in order to be actionable. For example, atoms aren’t actually little colored balls connected by springs, but that’s still a useful model in organic chemistry. You may need to hedge conclusions with qualifiers in order to get your work past Reviewer #3, but those “maybes” and “howevers” can often be omitted if they don’t change what practitioners should try next.
5) Apologize in advance for the state of academic publishing.
Modern academic publishing isn’t actually a conspiracy by a handful of large companies to line their pockets with government money that could and should be used to lift researchers out of penury, but it is functionally indistinguishable from a system that was. The best way to prepare your industry partners for its Kafkaesque production pipelines and interminable delays is to have them watch Gilliam’s Brazil.

If you are a practitioner in industry…

6) Remember that universities work in seasons, not weeks.
The timescale mis-match described in Rule #1 is due in part to the fact that academic researchers are almost always multi-tasking, and that many of those tasks are things they’ve never been trained to do. As students, they juggle several courses at once (which effectively means that they answer to several bosses who don’t communicate with each other). Later, they are responsible for teaching, writing grant proposals, and administrative duties.
Collectively, this mean that their “work week” is only a few hours long, and that they will often appear to move at a snail’s pace. Be as sympathetic as you can: they are even less happy with the situation than you are.
7) Remember that academic success is measured in publications, not sales.
University presidents routinely make about the economic value of research, but the only things that truly matter for academic advancement are publication, publication, and publication. Researchers are not given grants or tenure for doing things that are “merely useful”, even if doing so requires a deep understanding of subtle complexities and months of hard work. For all the jokes practitioners make about the ivory tower, academic life is hard, uncertain, and poorly paid. People stay in it for the love of new knowledge; respecting their priorities is essential to building a productive relationship. (That said, practical problems often do unlock the door to genuinely new research topics by pushing researchers out of their comfort zone.)
8) Do the background reading.
H.L. Mencken once wrote that, “There is always a well-known solution to every human problem—neat, plausible, and wrong.” Your problem is almost certainly one of those, and is almost certainly more complex than you first realize. While Rule #4 tells researchers to sacrifice detail for clarity, this rule asks practitioners to make an effort to grasp at least some of that detail so that you don’t waste time reinventing wheels and so that your research partner can think, work, and talk at full speed.
9) Don’t overstate what has been learned.
This rule is also a complement to Rule #4. The “maybes” and “howevers” that researchers are so fond of do sometimes matter; if your research partner has found that regular doses of a new drug seems to slow tumor growth in lab rats, do not embarrass them by claiming that they have discovered a cure for cancer.

If you are either…

10) Apologize in advance for the state of your data.
The final rule applies equally to both researchers and practitioners. Files’ names and locations, the meanings of column headers in tables, how those tables relate to one another, how missing values are represented and handles: everything that has made sense to you for years will suddenly seem a little foolish when you have to explain it to someone else. Apologize in advance, and then forgive yourself, because no matter how bad your data is, theirs may well be worse.

An old proverb says, “If you want to go fast, go alone; if you want to go far, go together.” In my experience, this is wrong: going alone is good for a fast start, but after that, both speed and distance come from having partners. Researchers and practitioners can each do great things on their own, but both are better able to solve big problems—problems that really matter—if they find ways to work together.

Talking People Into Things

I don’t always exhibit good judgment, but I am pretty good at talking people into things. Here are ten simple rules for doing it that I hope you will only use for good.

1) Don’t.
If you have to talk someone into something, odds are that they don’t really want to do it. Respect that: it’s almost always better in the long run to leave some particular thing undone than to use guilt or any underhanded psychological tricks that will only engender resentment.
2) Be kind.
I don’t know if there actually is a book called “Secret Tricks of the Ninja Sales Masters”, but if there is, it probably tells readers that doing something for a potential customer creates a sense of obligation, which in turn increases the odds of a sale. That may work, but (a) it only works once and (b) it’s a skeezy thing to do. If, on the other hand, you are genuinely kind, and help other people because it’s what good people do, you just might inspire them to be good people too.
3) Appeal to the greater good.
If you open by talking about what’s in it for them, you are signalling that they should think of their interaction with you as a commercial exchange of value to be bargained over. Instead, start by explaining how whatever you want them to help with is going to make the world a better place, and mean it. (If what you’re proposing isn’t going to make the world a better place, propose something better.)
4) Start small.
Most people are understandably reluctant to dive into things head-first, so give them a chance to test the waters and to get to know you and everyone else involved in whatever it is you want help with. Don’t be surprised or disappointed if that’s where things end: everyone is busy or tired or has projects of their own, or maybe just has a different mental model of how collaboration is supposed to work. Remember the 90-9-1 rule (90% of people will watch, 9% will speak up, and 1% will actually do things) and set your expectations accordingly.
5) Don’t build a project: build a community.
I used to belong to a baseball team that never actually played baseball: our “games” were just an excuse for us to hang out and enjoy each other’s company. If you actually want to accomplish something, you probably don’t want to go quite that far, but sharing a cup of tea with someone or celebrating the birth of their first grandchild can get you things that no reasonable amount of money can.
6) Establish a point of connection.
“I was speaking to X” or “we met at Y” gives them context, which in turn makes them more comfortable. This must be specific: spammers and cold-callers have trained us all to ignore anything that starts, “I recently came across your website”.
7) Be specific about what you are asking for.
People need to know this so that they can figure out whether the time and skills they have are a match for what you need. Being realistic up front is also a sign of respect: if you tell people you need a hand moving a few boxes when you’re actually packing up an entire house, they’re probably not going to come back.
8) Establish your credibility.
Mention your backers, your size, how long your group has been around, or something that you’ve accomplished in the past so that they’ll believe you’re worth taking seriously.
9) Create a slight sense of urgency.
“We’re hoping to launch this in the spring” is more likely to get a positive response than “We’d eventually like to launch this.” However, the word “slight” is also important: if your request is urgent, most people will assume you’re disorganized or that something has gone wrong, and may then err on the side of prudence.
10) Take a hint.
If the first person you ask for help says “no”, ask someone else. If the fifth or the tenth person says “no”, ask yourself if what you’re trying to do makes sense and is worth doing.

Handing Over and Moving On

This advice is for founders who are handing on their projects; see my CarpentryCon 2018 talk for more detail.

1) Be sure you mean it.
Letting go will be hard on you, but not letting go will be even harder on your successors, so be sure you’re actually going to let go.
2) Do it when other people think you should.
Just as you are the last person to realize when you’re too tired to be coding, you will often be the last person to realize that you ought to be moving on, so ask people and pay attention to what they say.
3) Be open about what, when, and why.
Tell people that you’re leaving and what the succession plan is as soon as possible (which in practice means “as soon as you think you won’t have to revise what you have said publicly”).
4) Leave for something.
People who start things usually aren’t good with idleness, and idleness tends not to be good for them, so when you leave, leave for something, even if it’s something small.
5) Don’t choose your successor on your own.
You may have strong opinions about who should succeed you, but you should still check those opinions with someone more objective.
6) Train your successor.
Share tasks with your successor for a few days or weeks: they will get to see how things actually work, and you’ll discover things you would otherwise forget to tell them. Go on holiday for a week and leave your successor temporarily in charge. You’ll discover even more things you would otherwise forget to pass on.
7) Celebrate.
Many people are uncomfortable being praised, but give the organization a chance to celebrate what you accomplished and thank you for it.
8) Leave.
It may be tempting to continue to have a role in the organization, but that usually leads to confusion, since people are used to looking to you for answers. It will be easier for your successor, particularly if they weren’t a founder as well, but the best thing you can do to help them is to find something else to do for a year.
9) Learn from your mistakes.
Whatever you have left will almost certainly not be the last thing you ever do. Take some time to think about what you could have done differently, write it down, and then move on: obsessing over only-ifs and might-have-beens won’t help anyone.
10) Do something before you go.
Everything comes to an end, but you have time before then to do something. What are you waiting for?

Teaching

See Teaching Tech Together for a much longer discussion.

  1. Be kind: all else is details.

  2. Remember that you are not your learners…

  3. …that most people would rather fail than change…

  4. …and that ninety percent of magic consists of knowing one extra thing.

  5. Never teach alone.

  6. Never hesitate to sacrifice truth for clarity.

  7. Make every mistake a lesson.

  8. Remember that no lesson survives first contact with learners…

  9. …that every lesson is too short from the teacher’s point of view and too long from the learner’s…

  10. …and that nobody will be more excited about the lesson than you are.

Supervising Undergraduate Software Projects

These tips are based on the dozens of projects I supervised at the University of Toronto between 2000 and 2010.

  1. Insist that they adopt a code of conduct for team members. Tell them how to report violations and how reports will be handled.

  2. Teach them how to run a meeting and how to make decisions as a group.

  3. Teach them how to manage conflict within the team.

  4. Make sure everyone understands and agrees to the evaluation scheme, particularly parts related to group work.

  5. Teach them how to allocate responsibilities.

  6. Spend an hour showing them how to use tools properly (e.g., linter, debugger, profiler) and then give them a performance exam to check that they have become proficient.

  7. Don’t try to insist that they use an agile development methodology (or any other): they are probably juggling several courses at once, each of which is setting deadlines without consulting the others, so it’s unfair to expect them to work as if they weren’t constantly being interrupted.

  8. Require them to create and close issues for tasks so that you can review their progress and they have a record of it when the time comes to write their final report.

  9. Tell them that they’re allowed to cut corners when they have to.

  10. Celebrate their victories, however small, because shared pain is lessened and shared joy increased.