When I first saw Starling Software’s Programmer Competency Matrix, I was struck by the parallels between its four levels and the first four of the five that Dr. Patricia Benner identified in her landmark book From Novice to Expert: Excellence and Power in Clinical Nursing Practice. Her five levels—which are grounded in a ton of empirical data—are as follows:
- Stage 1: Novice
- Novices have had no experience of the situations in which they are expected to perform. Novices are taught rules to help them perform. The rules are context-free and independent of specific cases; hence the rules tend to be applied universally. The rule-governed behavior typical of the novice is extremely limited and inflexible. As such, novices have no “life experience” in the application of rules.
- Stage 2: Advanced Beginner
- Advanced beginners are those who can demonstrate marginally acceptable performance, those who have coped with enough real situations to note, or to have pointed out to them by a mentor, the recurring meaningful situational components. These components require prior experience in actual situations for recognition. Principles to guide actions begin to be formulated. The principles are based on experience.
- Stage 3: Competent
- Competence develops when the practitioner begins to see his or her actions in terms of long-range goals or plans of which he or she is consciously aware. A plan establishes a perspective, and the plan is based on considerable conscious, abstract, analytic contemplation of the problem. The conscious, deliberate planning that is characteristic of this skill level helps achieve efficiency and organization. The competent practitioner lacks the speed and flexibility of a proficient one but does have a feeling of mastery and the ability to cope with and manage the many contingencies of the real world. The competent person does not yet have enough experience to recognize a situation in terms of an overall picture or in terms of which aspects are most salient, most important.
- Stage 4: Proficient
- The proficient practitioner perceives situations as wholes rather than in terms of chopped up parts or aspects, and performance is guided by maxims. Proficient practitioners understand a situation as a whole because they perceive its meaning in terms of long-term goals. The proficient practitioner learns from experience what typical events to expect in a given situation and how plans need to be modified in response to these events. The proficient practitioner can now recognize when the expected normal picture does not materialize. This holistic understanding improves the proficient practitioner’s decision making; it becomes less labored because the practitioner now has a perspective on which of the many existing attributes and aspects in the present situation are the important ones. The proficient practitioner uses maxims as guides which reflect what would appear to the competent or novice performer as unintelligible nuances of the situation; they can mean one thing at one time and quite another thing later.
- Stage 5: Expert
- The expert practitioner no longer relies on rules to connect his or her understanding of the situation to an appropriate action. The expert practitioner, with an enormous background of experience, now has an intuitive grasp of each situation and zeroes in on the accurate region of the problem without wasteful consideration of a large range of unfruitful, alternative diagnoses and solutions. The expert operates from a deep understanding of the total situation. He or she is no longer aware of features and rules; his/her performance becomes fluid, flexible, and highly proficient. Experts do use analytic tools in situations where they have had no previous experience, or when they make a wrong initial assessment of the situation.
I’d like to find out what knowledge programmers think they acquire at each of these five stages. My first guess is in the table shown below; what else do you think should be there? And what shouldn’t, or is in the wrong place? (Note that this table starts at Novice level, which is not the same as an absolute beginner who has seen nothing at all. For us, a novice is someone who has done a standard first-year university “Computing 101″ course, and can hack together code to (more or less) pass an assignment, but in Benner’s terms, has no “life experience” of their own on which to base decisions when faced with real-world problems.)
| Topic | Level 1
(Novice) |
Level 2
(Advanced Beginner) |
Level 3
(Competent) |
Level 4
(Proficient) |
Level 5
(Expert) |
| Description | Our assumed starting point | If they learn just one thing | If they learn everything in our course | Beyond the scope of this course (but in the next one) | Large-scale development for computational science |
| Data Structures | list | list of lists | dictionary, queue | tree (recursion) | cyclic graphs (and too many others to mention) |
| Control Flow | for while if |
raise/except | overriding methods first-class functions |
recursion | closures decorators |
| Modularization | function | libraries (import) | first-class functions classes |
dynamic import introspection |
cross-language programming |
| Patterns | what’s a pattern? | forall case sentinel most valuable flag |
reduce filter |
factory vs. prototype template method visitor |
composite command strategy proxy decorator |
| Text | readline substring escape characters tokenization using split |
wildcard regular expressions | character encoding (Unicode) | — | recursive descent parsing |
| Relational Databases | select where |
aggregation | join foreign keys one-to-one one-to-many |
many-to-many NULL transactions |
triggers stored procedures |
| Data Management | shared drive | version control structured vs. unstructured data compression |
dirty/missing data metadata (format specifiers, provenance) discipline-scale databases |
— | ontologies |
| Design | functions for code re-use | entities and relationships encapsulation interface vs. implementation |
polymorphism inheritance Open/Closed Principle |
Liskov Substitution Principle frameworks OO design patterns |
— |
| Version Control | checkout update commit |
merge | rollback | branch management | — |
| Build | “javac *.java” | “make program” | patterns and rules | macros conditional builds |
continuous integration |
| Documentation | README.txt | Javadoc and equivalents | — | — | — |
| Coordination | mailing list shared spreadsheet project blog |
bug tracker IRC/chat (archived) |
— | — | |
| Testing | hand-run tests (interactive or scripted) | unit tests using a framework “make test” debugger |
boundary case analysis | static analysis coverage |
stubs/mock objects |
| Packaging and Deployment | tar and email | version control labels “make tarball” |
installation scripts | RPMs, Eggs, autoconf | — |
| Web Programming | download files by clicking on them | wget url.open.read |
the HTTP protocol public/private key pairs |
CGI to serve static content | partial failure service composition |
| Performance Engineering | timing | profiling big-O algorithms vs. tweaks Amdahl’s Law |
memory hierarchy task farming |
processor pipeline data parallelism |
threading message passing |
V intersting table Greg.
what comes to mind based on my current preoccupations is the Data section could be filled out some.
eg level 2
– read and write from csv and text files
– basic knowledge of XML, ability to write code to grab basic data from xml files
lev 3
– knowledge of HDF5, NetCDF, and other standards for large binary datasets
Advanced:
– ability to carefully design the right data solution for an app task: appropriate mix of XML, csv, binary files, databases.
– includes whether to use ORM, and yes things like ontologies, which seem to be moving to RDF & OO databases.
(I’m more of a parallel algorithms guy by background but see data stuff as more & more a key competence these days in HPC)
Thanks for the thought-provoking post.
Benner: “The expert operates from a deep understanding of the total situation. He or she is no longer aware of features and rules”
You: “I’d like to find out what knowledge programmers think they acquire at each of these five stages”
It seems to me that Benner is saying, importantly and correctly, that knowledge acquisition as such is for stages earlier than 5. Some of your boxes seem to in effect grant this (“cross-language programming”, “ontologies”), while others seem to argue against it (“triggers/stored procedures”, “stubs/mock objects”). In any event, aren’t you struck by how trivial the latter two are in comparison with the former two?
I would have put “modifies test harness for each significant new project” in place of mock/stub, btw.
Yegge’s “Done and Gets Things Smart” is still my favorite exposition of the tricky issues involved in evaluating programmer expertise. Not coincidentally, it is itself a tricky piece of rhetoric that invites slow and careful reading.
A thought I suspect you will reject out of hand: perhaps what Benner calls expertise is what lies beyond what you call engineering. The choice by Starling to label columns with polynomial degree is IMHO a big giveaway that they don’t get it; novices may put O(n)^3 algorithms in their inner loops, but that does not mean that the best programmers spend the bulk of their time trying to reduce the asymptotic run time performance of their functions. Per Benner, they’ve long since moved on to more important concerns.