Agility and Romanticism
According to Wikipedia, Romanticism was “…a secular and intellectual movement in the history of ideas that originated in late 18th century Western Europe. It…was in part inspired by a revolt against aristocratic social and political norms of the previous period, as well as a reaction against the rationalization of nature by the Enlightenment…It stressed strong emotion…, legitimized the individual imagination as a critical authority…, and overturned some previous social conventions.”
That’s one view, anyway. Another is that Romanticism was not so much a reaction against the Enlightenment as a counter-strike by those who couldn’t keep up with it. Before the birth of quantitative experimental science, people could organize their world-view based on whatever patterns or similarities struck their fancy; anyone could have an opinion on the nature of motion or the origin of lightning.
Newton, Lavoisier, and others changed that: if you wanted your opinions taken seriously in 1800, you had to be intelligent enough to master calculus, and patient enough to measure foul-smelling gases a gram at a time. Most of the fine young things of the time were neither, so they did what fine young things always do when they’re losing an argument: they moved the goalposts. “Numbers? Equations? Pfah! My dear sir, they’re so cold. And this ‘utility’ you’re so obsessed with—that’s really a matter for tradesmen, isn’t it? A true gentleman relies on intuition and courage to comprehend the universe!” Now, I’m not suggesting that the early Romantics gathered some foggy night in a smoke-filled cellar in Paris to plot the defeat of Reason over a bottle of absinthe. Instead, natural selection acted on memes to produce the same result. Any fine young thing of the early 1800s who took on the Enlightenment on its own terms almost certainly lost; only those who adopted a “move the goalposts” strategy were rewarded with a chuckle at the college high table, attracted the interest of the FYT of opposite gender, got their work published, and so on. I think that something similar to the rise of Romanticism lies behind the popularity of agile software development methods today. Everyone wants to be “agile” these days: even Doug Rosenberg, co-author of Extreme Programming Refactored: The Case Against XP, has given it a nod in his latest book. I don’t think it’s a coincidence that this is happening at the same time as an increasing number of development jobs are moving offshore to India and China. Two of agility’s distinguishing features are being intimate with customers, and not leaving much behind you in the way of transferable documentation; neither works when you’re eight or ten timezones away from the people who are paying for your work. People who can convince their customers that agile practices are the way to go are therefore more offshore-proof that people who stick to higher-ceremony processes. It’s neither a conspiracy nor the “right” way to do things; it’s just a survival strategy that happens to work. And it’s not necessarily a bad thing. As Jorge Aranda pointed out, the quantitative experimental approaches that characterized the Enlightenment weren’t appropriate for tackling problems in psychology, sociology, and related fields. It was therefore only natural for Romantics to react negatively to people who thought that everything could and should be explained quantitatively, just as it is perfectly defensible for agilistas to rebel against processes that seem to emphasize red tape over creativity. Jorge’s other observation was interesting as well:
In science as in software development, you're differentiating between methods that are process- and number-heavy, and those that are people-oriented and unstructured, and you're putting the "Romantics" with the second group. But the "Romantics" should be seen as a separate entity: a third group that doesn't really care about building functional systems, or their users or implications of their work, but about their "art", in the most selfish and capricious sense of the word. The thing is, they hide more easily in an agile environment, and that's why you'll find them there. They ignore most of the precepts of agile development, but since these precepts are harder to track than the process-heavy ones, they get away with it; just as it's easier for a mediocre scientist to get away with qualitative than with quantitative studies.