Jon Udell has posted an interview he did with me last week at IT Conversations. The title is “High Performance Computing Considered Harmful”, and slides from a talk of the same name that I gave in Austin are available.
Later: here’s Jon’s summary of the interview. Curmudgeonly? Moi? 
I liked the interview and agreed with everything you said. Your implication that it is impossible to validate CFD is one I disagree with however. The best validation approach for complex codes is …. you write down (in closed form) any flow field that satisfies the boundary/initial conditions. Substitute this into the governing field equations (analytically) and whatever residual you get, you then introduce as a (closed form) body force in your code. You then have a perfectly good analytical closed-form solution (albeit with an obscure body force) that confirms your code is solving the equations you think it is, and enables you to perform convergence studies to ensure it is converging in a manner appropriate for the chosen discretization scheme.
Rich: and how do you deal with the influence and validation of subscale physical parameterisations? (p.s. haven’t listened to the interview … yet)
Terrific interview. I completely agree that most scientists would be better off learning about version control than MPI. In my field, biostatistics, I don’t see so many people distracted by HPC, but I do see serious problems with reproducibility.