Book cover

  1. Introduction
  2. Systems Programming
  3. Asynchronous Programming
  4. Unit Testing
  5. File Backup
  6. Data Tables
  7. Pattern Matching
  8. Parsing Expressions
  9. Page Templates
  10. Build Manager
  11. Layout Engine
  12. File Interpolator
  13. Module Loader
  14. Style Checker
  15. Code Generator
  16. Documentation Generator
  17. Module Bundler
  18. Package Manager
  19. Virtual Machine
  20. Debugger
  21. Conclusion
  1. License
  2. Code of Conduct
  3. Contributing
  4. Bibliography
  5. Glossary
  6. Index

download examples

Chapter 1: Introduction

The best way to learn design is to study examples [Schon1984, Petre2016], and some of the best examples of software design come from the tools programmers use in their own work. In these lessons we build small versions of things like file backup systems, testing frameworks, regular expression matchers, and browser layout engines both to demystify them and to give some insights into how experienced programmers think. We draw inspiration from [Brown2011, Brown2012, Brown2016], Mary Rose Cook’s Gitlet, and the books that introduced the Unix philosophy to an entire generation of programmers [Kernighan1979, Kernighan1981, Kernighan1983, Kernighan1988].

All of the written material in this project can be freely reused under the terms of the Creative Commons - Attribution - NonCommercial license, while all of the software is made available under the terms of the Hippocratic License. All proceeds from this project will go to support the Red Door Family Shelter.

Section 1.1: Who is our audience?

Every lesson should be written with specific learners in mind. These three personas describe ours:

  • Aïsha started writing VB macros for Excel in an accounting course and never looked back. After spending three years doing front-end JavaScript work she now wants to learn how to build back-end applications. This material will fill in some gaps in her programming knowledge and teach her some common design patterns.

  • Rupinder is studying computer science at college. He has learned a lot about the theory of algorithms, and while he uses Git and unit testing tools in his assignments, he doesn’t feel he understands how they work. This material will give him a better understanding of those tools and of how to design new ones.

  • Yim builds mobile apps for a living but also teaches two college courses: one on full-stack web development using JavaScript and Node and another titled “Software Design”. They are happy with the former, but frustrated that so many books about the latter subject talk about it in the abstract and use examples that their students can’t relate to. This material will fill those gaps and give them starting points for a wide variety of course assignments.

Like these three personas, readers should be able to:

  • Write JavaScript programs using loops, arrays, functions, and classes.

  • Create static web pages using HTML and CSS.

  • Install Node on their computer and run programs with it from the command line.

  • Use Git to save and share files. (It’s OK not to know the more obscure commands.)

  • Explain what a tree is and how to process one recursively. (This is the most complicated data structure and algorithm we don’t explain.)

This book can be read on its own or used as a classroom resource. If you are looking for a project to do in a software design course, adding a tool to those covered here would be fun as well as educational. Please see Chapter 21 for more details.

Section 1.2: What tools and ideas do we cover?

Programmers have invented a lot of tools to make their lives easier. This volume focuses on a few that individual developers use while writing software; we hope future volumes will explore those used in the applications that programmers build.

Appendix E defines the terms we introduce in these lessons, which in turn define their scope:

  • How to process a program like any other piece of text.

  • How to turn a program into a data structure that can be analyzed and modified.

  • What design patterns are and which ones are used most often.

  • How programs are executed and how we can control and inspect their execution.

  • How we can analyze programs’ performance in order to make sensible design tradeoffs.

  • How to find and run code modules on the fly.

Section 1.3: How are these lessons laid out?

We display JavaScript source code like this:

for (const thing in collection) {
  console.log(thing)
}

Unix shell commands are shown like this:

for filename in *.dat
do
    cut -d , -f 10 $filename
done

and data and output like this:

Package,Releases
0,1
0-0,0
0-0-1,1
00print-lol,2
00smalinux,0
01changer,0

We occasionally wrap lines in source code in unnatural ways to make listings fit the printed page, and sometimes use ... to show where lines have been omitted. Where we need to break lines of output for the same reason, we end all but the last line with a single backslash \. The full listings are all available in our Git repository and on our website.

Finally, we write functions as functionName rather than functionName(); the latter is more common, but people don’t use objectName{} for objects or arrayName[] for arrays, and the empty parentheses makes it hard to tell whether we’re talking about “the function itself” or “a call to the function with no parameters”.

Section 1.4: How did we get here?

In the early 2000s, the University of Toronto asked Greg Wilson to teach an undergraduate course on software architecture. After delivering the course three times he told the university they should cancel it: between them, the dozen textbooks he had purchased with the phrase “software architecture” in their titles devoted a total of less than 30 pages to describing the designs of actual systems.

Frustrated by that, he and Andy Oram persuaded some well-known programmers to contribute a chapter each to a collection called Beautiful Code [Oram2007], which went on to win the Jolt Award in 2007. Entries in the book described everything from figuring out whether three points are on a line to core components of Linux and the software for the Mars Rover, but the breadth that made them fun to read also meant they weren’t particularly useful for teaching.

To fix that, Greg Wilson, Amy Brown, Tavish Armstrong, and Mike DiBernardo edited a four-book series between 2011 and 2016 called The Architecture of Open Source Applications. In the first two volumes, the creators of fifty open source projects described their systems’ designs; the third book explored the performance of those systems, while in the fourth volume contributors built scale models of common tools as a way of demonstrating how those tools worked. These books were closer to what an instructor would need for an undergraduate class on software design, but still not quite right: the intended audience would probably not be familiar with many of the problem domains, and since each author used the programming language of their choice, much of the code would be hard to understand.

Software Tools in JavaScript is meant to address these shortcomings: all of the code is written in one language, and the examples are all tools that programmers use daily. Most of the programs are less than 60 lines long and the longest is less than 200; we believe each chapter can be covered in class in 1-2 hours, while the exercises range in difficulty from a few minutes to a couple of days.

Section 1.5: How can people use and contribute to this material?

All of the written material on this site is made available under the Creative Commons - Attribution - NonCommercial 4.0 International license (CC-BY-NC-4.0), while the software is made available under the Hippocratic License. The first allows you to use and remix this material for non-commercial purposes, as-is or in adapted form, provided you cite its original source; the second allows you to use and remix the software on this site provided you do not violate international agreements governing human rights. Please see Appendix A for details.

If you would like to improve what we have or add new material, please see the Code of Conduct in Appendix B and the contributor guidelines in Appendix C. If you have questions or would like to use this material in a course, please file an issue in our GitHub repository or send us email.

Section 1.6: Who helped us?

I am grateful to the creators of diagrams.net, Emacs, ESLint, Glosario, GNU Make, LaTeX, Node, NPM, Standard JS, SVG Screenshot, WAVE, and all the other open source tools used in creating these lessons: if we all give a little, we all get a lot. I would also like to thank Darren McElligott, Evan Schultz, and Juanan Pereira for their feedback; any errors, omissions, or misunderstandings that remain are entirely my fault.