Thou shalt have no other gods before the ANSI C standard 1477

CBFalconer

Ok, fair enough, though that was orthogonal to the point I wanted to get at. When we say that such-and-such methodology reduces the prevalence of bugs, we rarely measure quanbreastatively how much it reduces bugs. If you don't measure, you don't really know whether it is actually reducing bugs.

(I say "we", because I'm just as guilty of not measuring.)

In the history of software engineering, all too often techniques that were initially claimed to yield big wins later turned out to provide much smaller improvement, or even no improvement at all.

The medical field at least tries to do controlled trials. Imagine a medical process that didn't do any controlled trials. Imagine what the state of medical care would be like if all we had to go on were anecdotes from a few doctors. That's about where we are today in software engineering.

I'm not saying this is easy; on the contrary, it is very expensive, particularly for a field as new and rapidly changing as this. I'm not trying to criticize anyone. I'm just saying that we should expect limits on how certain we can be about which development methodologies will work and which ones won't, under changing conditions.