Appendix F — Checklists and Templates

(Gawande 2007) popularized the idea that using checklists can save lives, and more recent studies have generally supported their effectiveness Ramsay et al. (2019). We find them useful, particularly when bringing new teachers onto a team; the ones given below can be used as starting points for developing your own.

F.1 Teaching Evaluation

This rubric is designed to assess people teaching for 5–10 minutes with slides, live coding, or a mix of both. Rate each item as “Yes,” “Iffy,” “No,” or “Not Applicable.”

• Opening
  • Exists (use N/A for other responses if not)
  • Good length (10–30 seconds)
  • Introduces self
  • Introduces topics to be covered
  • Describes prerequisites
• Content
  • Clear goal/narrative arc
  • Inclusive language
  • Authentic tasks/examples
  • Teaches best practices/uses idiomatic code
  • Steers a path between the Scylla of jargon and the Charybdis of over-simplification
• Delivery
  • Clear, intelligible voice (use “Iffy” or “No” for strong accent)
  • Rhythm: not too fast or too slow, no long pauses or self-interruption, not obviously reading from a script
  • Self-assured: does not stray into the icky tarpit of uncertainty or the dungheap of condescension
• Slides
  • Exist (use N/A for other responses if not)
  • Slides and speech complement one another (dual coding)
  • Readable fonts and colors/no overwhelming slabs of text
  • Frequent change on screen (something every 30 seconds)
  • Good use of graphics
• Live Coding
  • Used (use N/A for other responses if not)
  • Code and speech complement one another
  • Readable fonts and colors/right amount of code on the screen
  • Proficient use of tools
  • Highlights key features of code
  • Dissects errors
• Closing
  • Exists (use N/A for other responses if it doesn’t)
  • Good length (10–30 seconds)
  • Summarizes key points
  • Outlines next steps
• Overall
  • Points clearly connected/logical flow
  • Make the topic interesting (i.e. not boring)
  • Knowledgeable

F.2 Teamwork Evaluation

This rubric is designed to assess individual performance within a team. You can use it as a starting point for creating a rubric of your own. Rate each item as “Yes,” “Iffy,” “No,” or “Not Applicable.”

• Communication
  • Listens attentively to others without interrupting
  • Clarifies what others have said to ensure understanding
  • Articulates ideas clearly and concisely
  • Gives good reasons for ideas
  • Wins support from others
• Decision Making
  • Analyzes problems from different points of view
  • Applies logic in solving problems
  • Offers solutions based on facts rather than “gut feel” or intuition
  • Solicits new ideas from others
  • Generates new ideas
  • Accepts change
• Collaboration
  • Acknowledges issues that the team needs to confront and resolve
  • Works toward solutions that are acceptable to all involved
  • Shares credit for success with others
  • Encourages participation among all participants
  • Accepts criticism openly and non-defensively
  • Cooperates with others
• Self-Management
  • Monitors progress to ensure that goals are met
  • Puts top priority on getting results
  • Defines task priorities for work sessions
  • Encourages others to express views even when they are contrary
  • Stays focused on the task during meetings
  • Uses meeting time efficiently
  • Suggests ways to proceed during work sessions

F.3 Event Setup

The checklists below are used before, during, and after events.

Scheduling the Event

• Decide if it will be in person, online for one site, or online for several sites.

• Talk through expectations with the host and make sure everyone agrees who is covering travel costs.

• Determine who is allowed to take part: is the event open to all comers, restricted to members of one organization, or something in between?

• Arrange teachers.

• Arrange space, including breakout rooms if needed.

• Choose dates. If it is in person, book travel.

• Get names and email addresses of attendees from host.

• Make sure everyone is registered.

Setting Up

• Set up a web page with details on the workshop, including date, location, and what participants need to bring.

• Check whether any attendees have special needs.

• If the workshop is online, test the video conferencing—twice.

• Make sure attendees will have network access.

• Create a page for sharing notes and exercise solutions (e.g. a Google Doc).

• Email attendees a welcome message with a link to the workshop page, background readings, a description of any setup they need to do, a list of what they need to bring, and a way to contact the host or teacher on the day.

At the Start of the Event

• Remind everyone of the code of conduct.

• Take attendance and create a list of names to paste into the shared page.

• Distribute sticky notes.

• Make sure everyone can get online.

• Make sure everyone can access the shared page.

• Collect any relevant online account IDs.

At the End of the Event

• Update the attendance list.

• Collect feedback from participants.

• Make a copy of the shared page.

Travel Kit

Here are a few things teachers take with them to workshops:

sticky notes	cough drops
comfortable shoes	a small notepad
a spare power adapter	a spare shirt
deodorant	video adapters
laptop stickers	their notes (printed or on a tablet)
a granola bar or some other snack	antacid (because road food)
business cards	spare glasses/contacts
a notebook and pen	a laser pointer
an insulated cup for tea/coffee	extra whiteboard markers
a toothbrush or mouthwash	wet wipes (because spills happen)

When travelling, many teachers also take running shoes, a bathing suit, a yoga mat, or whatever else they exercise in or with. Some also bring a portable WiFi hub in case the room’s network isn’t working and some USB drives with installers for the software learners need.

F.4 Lesson Design

This section summarizes the backward design method developed independently by Fink (2013). It lays out a step-by-step progression to help you think about the right things in the right order and provides spaced deliverables so you can re-scope or redirect effort without too many unpleasant surprises.

Everything from Step 2 onward goes into your final lesson, so there is no wasted effort; as described in Chapter 6, writing sample exercises early helps ensure that everything you ask your learners to do contributes to the lesson’s goals and that everything they need to know is covered.

The steps are described in order of increasing detail, but the process itself is always iterative. You will frequently go back to revise earlier work as you learn something from your answers to later questions or realize that your initial plan isn’t going to play out the way you first thought.

Who is this lesson for?

Create some learner personas (Section 6.1) or (preferably) choose ones that you and your colleagues have drawn up for general use. Each persona should have:

1. the person’s general background;

2. what they already know;

3. what they think they want to do; and

4. any special needs they have.

Deliverable: brief summaries of who you are trying to help.

What’s the big idea?

Write point-form answers to three or four of the questions below to help you figure out the scope of the lesson. You don’t need to answer all of these questions, and you may pose and answer others if you think it’s helpful, but you should always include a couple of answers to the first one. You may also create a concept map at this stage (Section 3.1).

• What problems will people learn how to solve?

• What concepts and techniques will people learn?

• What technologies, packages, or functions will people use?

• What terms or jargon will you define?

• What analogies will you use to explain concepts?

• What mistakes or misconceptions do you expect?

• What datasets will you use?

Deliverable: a rough scope for the lesson. Share this with a colleague—a little bit of feedback at this point can save hours of wasted effort later on.

What will learners do along the way?

Make the goals in Step 2 firmer by writing full descriptions of a couple of exercises that learners will be able to do toward the end of the lesson. Doing this is analogous to test-driven development: rather than working forward from a (probably ambiguous) set of learning objectives, work backward from concrete examples of where you want your learners to end up. Doing this also helps uncover technical requirements that might otherwise not be found until uncomfortably late.

To complement the full exercise descriptions, write brief point-form descriptions of one or two exercises per lecture hour to show how quickly you expect learners to progress. Again, these serve as a good reality check on how much you’re assuming and help uncover technical requirements. One way to create these “extra” exercises is to make a point-form list of the skills needed to solve the major exercises and create an exercise that targets each.

Deliverable: 1–2 fully explained exercises that use the skills people are to learn, plus half a dozen point-form exercise outlines. Include complete solutions so that you can make sure the software you want learners to use actually works.

How are concepts connected?

Put the exercises you have created in a logical order and then derive a point-form lesson outline from them. The outline should have 3–4 bullet points for each hour with a formative assessment of some kind for each. It is common to change assessments in this stage so that they can build on each other.

Deliverable: a lesson outline. You are likely to discover things you forgot to list earlier during this stage, so don’t be surprised if you have to double back a few times.

Lesson overview

You can now write a lesson overview with:

• a one-paragraph description (i.e. a sales pitch to learners);

• half a dozen learning objectives; and

• a summary of prerequisites.

Doing this earlier often wastes effort, since material is usually added, cut, or moved around in earlier steps.

Deliverable: course description, learning objectives, and prerequisites.

F.5 Pre-Assessment Questionnaire

This questionnaire helps teachers gauge the prior programming knowledge of participants in an introductory JavaScript workshop. The questions and answers are concrete, and the whole thing is short so that respondents won’t find it intimidating.

1. Which of these best describes your experience with programming in general?

   1. I have none.

   2. I have written a few lines now and again.

   3. I have written programs for my own use that are a couple of pages long.

   4. I have written and maintained larger pieces of software.

2. Which of these best describes your experience with programming in JavaScript?

   1. I have none.

   2. I have written a few lines now and again.

   3. I have written programs for my own use that are a couple of pages long.

   4. I have written and maintained larger pieces of software.

3. Which of these best describes how easily you could write a program in any language to find the largest number in a list?

   1. I wouldn’t know where to start.

   2. I could struggle through by trial and error with a lot of web searches.

   3. I could do it quickly with little or no use of external help.

4. Which of these best describes how easily you could write a JavaScript program to find and capitalize all of the titles in a web page?

   1. I wouldn’t know where to start.

   2. I could struggle through by trial and error with a lot of web searches.

   3. I could do it quickly with little or no use of external help.

5. What do you want to know or be able to do after this class that you don’t know or can’t do right now?

Aveling, Emma-Louise, Peter McCulloch, and Mary Dixon-Woods. 2013. “A Qualitative Study Comparing Experiences of the Surgical Safety Checklist in Hospitals in High-Income and Low-Income Countries.” BMJ Open 3 (8). https://doi.org/10.1136/bmjopen-2013-003039. Reports the effectiveness of surgical checklist implementations in the UK and Africa.

Biggs, John, and Catherine Tang. 2011. Teaching for Quality Learning at University. Open University Press. ISBN 0335242758. A step-by-step guide to lesson development, delivery, and evaluation for people working in higher education.

Fink, L. Dee. 2013. Creating Significant Learning Experiences: An Integrated Approach to Designing College Courses. Jossey-Bass. ISBN 1118124251. A step-by-step guide to a systematic lesson design process.

Gawande, Atul. 2007. “The Checklist.” The New Yorker, December 10. Describes the life-saving effects of simple checklists.

Ramsay, G., A. B. Haynes, S. R. Lipsitz, et al. 2019. “Reducing Surgical Mortality in Scotland by Use of the WHO Surgical Safety Checklist.” BJS, ahead of print, April. https://doi.org/10.1002/bjs.11151. Found that the introduction of surgical checklists in Scottish hospitals significantly reduced mortality rates.

Urbach, David R., Anand Govindarajan, Refik Saskin, Andrew S. Wilton, and Nancy N. Baxter. 2014. “Introduction of Surgical Safety Checklists in Ontario, Canada.” New England Journal of Medicine 370 (11): 1029–38. https://doi.org/10.1056/nejmsa1308261. Reports a study showing that the introduction of surgical checklists did not have a significant effect on operative outcomes.

Wiggins, Grant, and Jay McTighe. 2005. Understanding by Design. Association for Supervision & Curriculum Development (ASCD). ISBN 1416600353. A lengthy presentation of reverse instructional design.