Suppose you have a room full of scientists—hundreds of ‘em—and want to find out how they actually use computers in their work. There isn’t time to interview them individually, or to record their desktops during a typical working week, so you’ve decided to ask them to self-asses their understanding of some key terms on a scale of:
- No idea what it is.
- Use it/have used it infrequently.
- Use it regularly.
- Couldn’t get through the day without it.
My list is below; what have I forgotten, and (more importantly) how would you criticize this assessment method?
- A command-line shell
- Shell scripts
- Version control system (e.g., CVS, Subversion)
- Bug tracker
- Build system (e.g., Make, Ant)
- Debugger (e.g., GDB)
- Integrated Development Environment (e.g., Eclipse, Visual Studio)
- Numerical Computing Environment (e.g., MATLAB, Mathematica)
- Inverse analyzer (e.g., Inane)
- Spreadsheet (e.g., Excel)
- Relational database (e.g., SQLite, MySQL, Oracle)
- Layout-based document formatting (e.g., LaTeX, HTML)
- WYSIWYG document formatting (e.g., Word, PowerPoint, OpenOffice)
Now, you have the same room full of scientists, and you want to find out how much they know about software development. There still isn’t time to interview them or have them solve some programming problems, so again you’re falling back on self-assessment. This time, the scale is:
- No idea what it means.
- Have heard the term but couldn’t explain it.
- Could explain it correctly to a junior colleague.
- Expert-level understanding.
and the terms themselves are:
|
|
Once again, my questions are (a) what have I forgotten, and (b) how “fair” is this as an assessment method?
I think your first scale is missing something between “Haven’t heard of it” and “use it infrequently”. Perhaps something like “I know what it is, but I don’t use it”
Howdy from Texas.
Is Q9 a trick question? Not only have I never heard of an inverse analyzer, Google hasn’t either. The first hit on “inane” “inverse analyzer” is this blog entry, and the pickin’s are mighty slim after that.
The upper scale doesn’t have a good response if they know perfectly well what X is, but have never used it.
I’d suggest that you add a number into your first scale:
0. No idea what it is.
1. Know what it is, but never used it.
2. Use it/have used it infrequently.
3. Use it regularly.
4. Couldn’t get through the day without it.
What I don’t like about your scale is that it’s loaded and doesn’t cover all the options. There’s pretty much a built-in assumption that if you know about this stuff you will use it. What about people who have good knowledge of something but either have chosen not to use it or have used it a lot and stopped because it was bad (or bad for them)? For example, I’ve not only lived in debuggers, I’ve written them; but I haven’t used one for years.
Personally, I think the most obvious missing thing on the (first) list is a text editor…but I know you think emacs users would still be using punched card if only there decent readers available
(And on a different note, I can’t resist asking, if Word is WYSIWYG system, why does it have a preview button?)
I might add something like “automated builds” to the first list. And perhaps release management or “how do you deploy the software?”
> Perhaps something like “I know what it is, but I don’t use it”
Or even, “I don’t know what it is, but Greg told me I need to use it”
Some trick questions could be useful for filtering bozo responses. “And all of you who said that they used back-scattering inheritance in their code are big ol’ liars!”
~Matt
For the first list you’ve missed out:
14. Computerised Simulation e.g. Monte Carlo methods, discrete event simulations.
15. Computerised visualization software e.g. Thermal analysis.
On the second list, you could add:
* Recursion
* OO
* functional programming style.
* declarative programming style.
* Profiling.
* Refactoring.
The last list doesn’t address the team management side of software development though.
- Paddy.
* Style guides
*
Your first order question is define ‘scientist’. I would hazard that there are many disciplines that not pass any of these tests. eg. Medical researcher. But that does not mean that they aren’t a scientist.
Fact much of this I would pose to the team IT guy not the Principal Investigator.
Just an observation.
“I know all about it and you couldn’t pay me enough to use it again”.
This post is also a high Google hit for “Variant digression”.
define:”variant digression” and define:”variant regression” return nothing.
Yahoo returns little of interest. So … what is it?