Critical Systems Development

1
Critical Systems Development
2
Motivation

• Nuclear Plant Emergency Shutdown System.
• Navigational Systems
• Radiation Therapy Machines.

3
Critical Systems Development

• Fault Avoidance
• Fault Tolerance
• Fault Detection

4
Fault-free software development

• Needs a precise specification.
• Organizational committment to quality.
• Information hiding and encapsulation.
• Strict typing and run-time checking
• Error-prone constructs should be avoided
• Dependable and repeatable development process
• Irrespective of the people involved in the
  process, it will be succesful (???)

5
Error-prone constructs

• Floating-point numbers
• Pointers
• Hanging pointers
• Readability
• Dynamic memory allocation
• Parallelism
• Recursion

6
Error-prone constructs

• Interrupts and Gotos
• Preemption
• Readability
• Inheritance
• Code is not localised. This can result in
  unexpected behaviour when changes are made and
  problems of understanding
• These constructs dont have to be avoided but
  they must be used with great care. (???)

7
Error-minimizing constructs

• Information Hiding
• the probability of accidental corruption of
  information is low
• the information is surrounded by firewalls so
  that problems are less likely to spread to other
  parts of the program
• as all information is localised, the programmer
  is less likely to make errors and reviewers are
  more likely to find errors

8
Critical Systems Development

• Fault Avoidance
• Fault Tolerance
• Fault Detection

9
Hardware fault tolerance

• Triple-Modular Redundancy (TMR)

10
Going to Software

• TMR only works if
• Faults come from component failure, not design
  faults.
• Low probability of simultaneous component failure

11
Software fault tolerance

• Recovery blocks

12
Software fault tolerance

• N-version programming
• Often used in navigational systems e.g. in Airbus
  320.

13
Achieving Design Diversity

• Different approaches to design (e.g
  object-oriented and function oriented)
• Implementation in different programming languages
• Use of different tools and development
  environments
• Use of different algorithms.

14
Critical Systems Development

• Fault Avoidance
• Fault Tolerance
• Fault Detection

15
Fault Detection

• Detect fault
• Access damage
• Recover

16
Damage assessment

• Assess extent of data corruption
• Assess extent of state corruption.
• Generally based on validity functions
• Interface CheckableObject
• public boolean check()

17
Damage assessment techniques

• Checksums for data transmission
• Redundant pointers for data structures.
• Watch dog timers

18
Fault Detection

• Detect fault
• Access damage
• Recover

19
Recovery

• Domain specific
• Error codes
• Used in data transmission
• Redundant pointers
• Used for database and filesystem repair
• Rollbacks and checkpoints
• Backword recovery. Heavily used in database
  systems.

20
Fault Detection

• Exceptions ?
• Detect fault
• Access damage
• Recover

21
Questions To Be Answered

• Is it theoretically possible to remove all faults
  from a software system?
• It has been suggested that the control software
  for a radiation therapy machine should be
  implemented using N-Version programming. Is this
  a good idea?

22
Questions To Be Answered

• Producing safe software obviously involves
  considerable exta costs. What extra costs can be
  justified if 100 lives will be saved over the
  15-year lifetime of a system? Would the same
  costs be justified if 10 lives were saved? Does
  the earning capability of the people affeted make
  a difference to this judgement?

23
Trying is the first step toward failure.