Continuous Installation Checking

This one started with me trying and failing to install some bioinformatics software on my Mac, then turned into a Twitter rant:

• Everyone: please start doing installation-driven development. Not test-driven, not feature-driven: installation-driven.
• Best possible use of AI: a program that simulates a naive user trying to install your software. "Reads" docs, follows steps…
• …gets lost, makes mistakes… we could feed traces from actual install attempts into a Markov chain generator, then couple it to…
• …Travis-CI or something like that so that "user can install" becomes part of the pre-commit check and the world's a better place

But I'm serious. (I'm always serious.) Given how formulaic installation instructions tend to be, could a graduate student or startup build:

1. an NLP tool to extract actions from installation and configuration instructions,
2. a monitor to gather traces of the steps taken by actual users (not professional sys admins or package authors) as they try to install the software, and
3. some sort of Markov chain generator to follow the steps found in #1 while making mistakes similar to those detected by #2?

I don't think this would be easy: the first two parts are each worthy of a thesis (though I suspect the third will be relatively straightforward by comparison). But imagine how useful this would be. Imagine what it would be like if Travis-CI or other continuous integration tools could fire up a lightweight VM and go through your installation instructions every time you committed a change and say, "Nope, sorry, it looks like you've missed a step, and that bit there, the one we've been telling you about for the last three months? Most people are still going to find it confusing, so how about you try explaining it differently?"