Accidental Horizons


A quarter of a century ago, in The Mythical Man Month, Fred Brooks pointed out the difference between intrinsic complexity and accidental complexity. Intrinsic complexity is how hard a problem really is; accidental complexity is how much harder we make it by creating inconsistent APIs, fragile configuration files, or GUIs that don't accurately reflect the state of the world.

Over the past couple of years, I've realized there's a similar difference between intrinsic horizons and accidental horizons. A "horizon" defines what you can see from where you're standing; in a technical field, it is the boundary between problems you know how to solve, and ones you don't. There are therefore two ways a field can make progress: by inventing new ways to solve problems, or by pushing out the horizons of the ways they already have. When we do the latter, we often discover that the horizon was accidental. There was never anything to stop us from solving those problems before—we just didn't think to try.

Which brings me to Hibernate, the object/relational mapping package we're using in Hippo. A Hibernate-based application has three parts:

When everything is working as it should, this is an excellent division of labor. The problem is that when things aren't working, there is no easy way to figure out why not [1]. I have Hibernate's log messages, but they don't tell me why my attempt to create a session object throws an exception. I have Hibernate's source, so I could step into the session's constructor and trace the fault if I wanted to, but I don't think I should have to. Hibernate is supposed to abstract away the details of working with databases directly; as far as I'm concerned, it should also abstract away the details of debugging my interactions with the database.

Which brings us back to the notion of an accidental horizon. The IDE I'm using, Eclipse, is really just a framework for hosting plugins. Some of these (like the syntax-aware Java editor) come bundled with it, but many others (like the Checkstyle plugin) have been developed by third parties. The Apache web server is built in a similar way: its core only knows how to load plugins, and route message traffic through them. All the stuff we use it for—resolving paths, launching CGI scripts, and so on—is implemented in a plugin that ships with the distribution, but hundreds of others exist as well.

So why aren't debuggers built this way? When I something as complex as Hibernate, why can't I also create a debugging plugin to help Eclipse show me what Hibernate is doing? There are no intrinsic technical obstacles [2]; as far as I can see, the only reason it isn't supported is that, well, no-one else supports it either. It is a purely accidental horizon, and the sooner it's eliminated, the more productive we will all be [3].

Now, if you'll excuse me, I have some Hibernate source code to step through…

[1] See also Joel Spolsky's article on leaky abstractions.

[2] Which isn't to say that it wouldn't take a lot of thoughtful design and hard work to create the right hooks in Eclipse for developers to plug into. Master's thesis, anyone?

[3] See also my article on extensible programming systems.