They're All Small Devices Now

I noticed a book on the shelves at Scimatic last week called Small Memory Software: Patterns for systems with limited memory, by Noble and Weir. It took me a couple of minutes to realize why it seemed familiar: I reviewed it for Doctor Dobb's Journal when it first came out. The topics are what you'd expect from the title–memory limits, adaptive compression, copy-on-write, and so on.

I liked the book when it first came out, and like it even more now. What struck me leafing through it, though, is how many of the patterns it presents are now important for large-scale computing because of the explosion in data set sizes over the past decade. (See for example Titus Brown's recent post "The sky is falling! The sky is falling!".) Nobody's data fits in RAM any more, and everyone managing a data center has to worry about power consumption. In a way, it reminds me of how cosmology and subatomic physics are intimately intertwined.