Data communications over telegraph circuits 1914

there are fail-safe or fail-graceful issues ... how resilliant is the overall environment in the face of failures.

supposedly software (& human mistakes) took over from purely hardware failures starting sometime in the early 80s.

also, when we were doing ha-cmp ... we coined the term disaster survivability and geographic survivability (to differentiate from disaster recover) ... as an increasing number of failures were environmental related (in part, as other types of failures declined).

for failure resilliant programming ... i've often commented that it can be ten times the work and 4-10 times the lines of code ... compared to the simple straight-line application implication. failure resilliant programming is also often characteristic of online or real-time service. when we were working on the payment gateway for what has since become e-commerce ... the failure resilliant programming was much more than the straight line application.

another part of failure resilliancy is to have significant amounts of redundant resources ... in telco this was frequently lumped into provisioning. one of the things that can happen is business go on cost-cutting measures that starts to eliminate some amount of the redundancy.

Complexity and-or paring back on redundancy can make the environment more vulnerabile to general outages and systemic failures.

then there is the story of a large financial insbreastution that had computerized system for optimally moving money around ... that was located in a large 50-story office building. their claim was that the system earned more in 24hrs than the annual rent on the whole bldg. plus the aggregate annual salaries of everybody working in the bldg (or conversely they would loose that much money every 24hrs if the system was down).

misc. past references to 4-10 times for failure resilliant programming:

--