How Did Software Get So Reliable Without Proof

1
How Did Software Get So Reliable Without Proof?

• C.A.R. Hoare
• Oxford University
• Keynote address
• Formal Methods Europe 1996

2
Answer

• Rigorous management of procedures for design
  inspection and review
• Quality assurance based on a wide variety of
  targeted tests
• Continuous evolution by removal of errors from
  products already in widespread use
• Defensive programming

3
Gloom and doom has been predicted for years

• False predictions are common in science and
  engineering, but ...
• Many projects justify the gloom and doom!
• Some developers have become very careful, causing
  the predictions not to come true in some cases
• Software can be simultaneously fragile and robust

4
Formal methods

• Play a small direct part
• But they do provide
• a conceptual framework
• basic understanding to promote the best of
  current practice
• point directions for future improvement

5
Worst kinds of errors subtly misleading effects
6
Several thousand deaths attributed to computers,
but only 10 or so to software
7
There are software systems with tens of millions
of lines of code, updated on the fly thousands
of times a year,
that almost never fail
8
Technology transfer is extremely slow, even in
software
typically twenty years
9
There is a wider recognition that software
development can be predicted, planned, managed
and controlled in the same way as any other
branch of engineering
10
Projects that fail can be identified as failures
very early
a lack of up-front effort on requirements and
design
11
Judgement and experience must be applied early
Decisions and their reasons must be open to review
12
Mathematical reasoning belongs in the review and
inspection process

• especially in the verification of interfaces
  among components

13
Use assertions, preconditions, postconditions,
invariants
14
Long periods of intellectual drudgery are needed
to achieve a thoroughly professional result
15
Testing is inductive proof of correctness
16
Testing tests the methods of production
17
Tests must be planned as far in advance as
possible, and must be as severe as possible

• Encourages early exploration, simplification and
  clarification of assumptions

18
Design more tests than can ever be run

• Helpful even if no tests are ever run
• Tests should be selected at random from the large
  set available

19
Tests that are designed early are necessarily
black box tests
20
Daily/weekly builds and regression (sanity) tests
21
Bring costs back from the customer to the
perpetrator
22
Bimodal distribution of bugs gentle and vicious
mosquitos

• Swat the vicious ones and live in peace

23
Comprehensive testing costs four times as much as
less rigourous testing
24
Rate of error discovery declines precipitously

• only a tiny fractions of possible paths is ever
  executed, and the rate of opening new paths after
  the initial rush is very low

25
Engineers build safety factors into designs

• avoid sophistication and optimization
• no pointers, no dynamic memory allocation
• pre-allocate more resources than are needed
• no global variables
• execution efficiency should not be a concern
  unless it really is!
• Wear a belt and suspenders check bounds before
  and after calling

26
Only 20 of the code in large systems is ever
executed

• Overengineered by a factor of 5?
• Causes a real problem all this code must be
  designed, written, tested and maintained
• problem is especially severe with clones

27
Auditing data structures improves mean time to
crashing from hours to months
28
Need rapid and invisible warm restart
29
Engineering research into programming methodology

• establish a framework and a theoretical basis for
  derivation and justification of every design
  decision

30
Bohm-Jacopini Theorem

• dont need gotos

31
Strict type-checking
32
Application of research in progress is dangerous

• Theory can take 20 years to move into research on
  applicability, and then there can be another 20
  years before widespread adoption

33
Obscure time-dependent errors, deadlocks and
livelocks (thrashing) can lurk untestable for
many years, and then trigger a failure costing
many millions

• Formal methods may be close to the point of being
  able to detect such conditions

34
Education and toolsets are limiting in the
effective application of formal methods
35
Research on improved techniques is made more
difficult by fragmentation

• Hoare is trying for a unified theory