Comments on: Show Me http://third-bit.com/blog/archives/1271.html Data is ones and zeroes | Software is ones and zeroes and hard work. Tue, 22 May 2012 16:23:22 +0000 hourly 1 http://wordpress.org/?v=3.3.2 By: Lorin Hochstein http://third-bit.com/blog/archives/1271.html#comment-1395 Lorin Hochstein Thu, 20 Dec 2007 00:49:45 +0000 http://pyre.third-bit.com/blog/archives/1271.html#comment-1395 Don't overlook IEEE Software, which now has an excellent column called Voice of Evidence, edited by Forrest Shull, which summarizes results of empirical software engineering studies in a format that's more accessible to practitioners than the journals. Don’t overlook IEEE Software, which now has an excellent column called Voice of Evidence, edited by Forrest Shull, which summarizes results of empirical software engineering studies in a format that’s more accessible to practitioners than the journals.

]]> By: Personal experience vs. Science « Catenary http://third-bit.com/blog/archives/1271.html#comment-1394 Personal experience vs. Science « Catenary Wed, 19 Dec 2007 13:12:53 +0000 http://pyre.third-bit.com/blog/archives/1271.html#comment-1394 [...] experience vs. Science Jump to Comments Greg Wilson has an interesting post regarding both the lack of evidence and the lack of regard forevidence that is prevalent in software engineering. [...] [...] experience vs. Science Jump to Comments Greg Wilson has an interesting post regarding both the lack of evidence and the lack of regard forevidence that is prevalent in software engineering. [...]

]]> By: Neil Bartlett http://third-bit.com/blog/archives/1271.html#comment-1393 Neil Bartlett Tue, 18 Dec 2007 21:25:55 +0000 http://pyre.third-bit.com/blog/archives/1271.html#comment-1393 Greg, it's not that I am disinterested in or would ignore objective evidence. But such evidence so rarely exists that working programmers have no choice but to generate our own evidence, by actually trying the language or framework in question. If I try a technique and it works then that is the ultimate in hard evidence _for me_. It is the evidence of my own senses. I know that for you and for other programmers, it's just another anecdote... so they have to go and generate their own evidence. The pharmaceutical analogy is flawed because I am unlikely to be killed by a dangerous programming language. Personal experimentation in our field has no downside risk other than opportunity cost. Greg, it’s not that I am disinterested in or would ignore objective evidence. But such evidence so rarely exists that working programmers have no choice but to generate our own evidence, by actually trying the language or framework in question.

If I try a technique and it works then that is the ultimate in hard evidence _for me_. It is the evidence of my own senses. I know that for you and for other programmers, it’s just another anecdote… so they have to go and generate their own evidence.

The pharmaceutical analogy is flawed because I am unlikely to be killed by a dangerous programming language. Personal experimentation in our field has no downside risk other than opportunity cost.

]]> By: Titus Brown http://third-bit.com/blog/archives/1271.html#comment-1392 Titus Brown Tue, 18 Dec 2007 19:25:38 +0000 http://pyre.third-bit.com/blog/archives/1271.html#comment-1392 Speaking as a biologist / programmer, I couldn't agree with you more. I've been thinking about moderating my gung ho rants about testing a bit, because I'm not really sure it works for everyone -- even though I can tell you that it *should*, if done properly. (heh.) Anecdotal evidence drives quite a bit of 'net wrangling about languages, OSes, etc. and it does CS quite a bit of harm. Speaking as a biologist / programmer, I couldn’t agree with you more. I’ve been thinking about moderating my gung ho rants about testing a bit, because I’m not really sure it works for everyone — even though I can tell you that it *should*, if done properly. (heh.)

Anecdotal evidence drives quite a bit of ‘net wrangling about languages, OSes, etc. and it does CS quite a bit of harm.

]]> By: David Wolever http://third-bit.com/blog/archives/1271.html#comment-1391 David Wolever Tue, 18 Dec 2007 17:13:58 +0000 http://pyre.third-bit.com/blog/archives/1271.html#comment-1391 The problem with scientific studies, though, is they are notoriously hard to conduct. So when you don't have any science on hand for a particular situation, you've got to base your opinions on _something_. At least, that's what really irks me. No doubt, though, programming community would be much better off if all programmers were as rational as the machines they work with. The problem with scientific studies, though, is they are notoriously hard to conduct. So when you don’t have any science on hand for a particular situation, you’ve got to base your opinions on _something_. At least, that’s what really irks me.

No doubt, though, programming community would be much better off if all programmers were as rational as the machines they work with.

]]> By: Neil http://third-bit.com/blog/archives/1271.html#comment-1390 Neil Tue, 18 Dec 2007 16:09:08 +0000 http://pyre.third-bit.com/blog/archives/1271.html#comment-1390 Great post Greg. I follow this blog: http://badscience.net/, and I'm often struck by how similar many software 'engineering' claims are to the claims of homeopathy, naturopathy, etc. Even an empirical study may have faults: selection biases, not controlled, not blind, etc. I would say that such studies seem easier in medicine than software. How easy is it to do a double-blind controlled trial with UML class diagrams in an industrially relevant manner? My other question is, to what extent are working programmers relying on anecdotal experience because they are subject to so much hype? E.g., .Net will do this better, use RUP, etc... Eventually I imagine one would just ignore it and use what works (Vi, C, what have you). One difference between medicine and software is that medical devices/drugs cannot make unsubstantiated claims the way software vendors can. Great post Greg. I follow this blog: http://badscience.net/, and I’m often struck by how similar many software ‘engineering’ claims are to the claims of homeopathy, naturopathy, etc. Even an empirical study may have faults: selection biases, not controlled, not blind, etc.

I would say that such studies seem easier in medicine than software. How easy is it to do a double-blind controlled trial with UML class diagrams in an industrially relevant manner?

My other question is, to what extent are working programmers relying on anecdotal experience because they are subject to so much hype? E.g., .Net will do this better, use RUP, etc… Eventually I imagine one would just ignore it and use what works (Vi, C, what have you).

One difference between medicine and software is that medical devices/drugs cannot make unsubstantiated claims the way software vendors can.

]]>