Thou shalt have no other gods before the ANSI C standard 1583

Trevor L. Jackson, III

That would be great, of course. I don't doubt that great staff make a big difference, but I am curious where you got those numbers from. Has someone tried the experiment? Have they tried measuring the defect rate, the cost, the productivity? My one concern is that if no one has ever tried to measure, I'm not sure how we could know for sure exactly how much benefit it gives (is it 1.2x? 2x? 8x? how do we know?).

I'm not trying to give you a hard time, but I tend to think it is important that we measure these things rather than just guess. The history of software engineering shows many examples of new development processes that were supposed to provide an order of magnitude improvement. (structured programming, object-oriented programming, software re-use, etc.) When they were actually measured, there often turned out to be some improvement, but no where near what was claimed. This is why I think it behooves us to measure these things.

And I suspect the security community could benefit from placing more effort on measurement. We could probably learn a lot by tracking our errors, identifying root causes, counting them, and using this as a management technique. Unfortunately, right now it is very rare for this kind of information to be shared outside of any single organization, and that greatly slows down the learning process.

Good point. Lacking motivation or incentive, change is very unlikely. Organizations respond to incentives -- or, at least, they certainly are unlikely to insbreastute expensive changes when there is no motivation or incentive to do so.

And even with strong motivation and enormous incentives, it can still be difficult. Look at Microsoft for an example: they are spending hundreds of millions on improving their security culture (they say), so they must feel that they have a strong incentive for change. Even then, their progress so far has been constrained by their backwards compatibility problem. Change could take time.