Critical Systems Validation

1
Critical Systems Validation

• Need to verify validate

• Reliability
• How likely that system behaves correctly?

• Safety
• How likely is system to cause damage?

• Security
• How likely can system defend against accidental
  or deliberate intrusion?

2
Validating reliability

• Idealized approach
• Identify operational profiles
• Probabilistic mix of classes of inputs
• Based on profile of existing, similar systems
• Prepare test data
• Manual and generated
• Apply tests
• Compute observed reliability

3
Validating reliability (cont)

• But
• Operational profiles may be inaccurate
• Particularly difficult for new systems
• Test data generation may be expensive
• Higher reliability standard
• more accurate statistics
• more failures
• longer and longer runs

4
Safety assurance

• Cheap form of formal proof
• Demonstrate that program cannot arrive at unsafe
  state
• Proof by contradiction
• For each hazard condition
• Assume the unsafe state
• Backtrack through program, demonstrating that
  post-conditions of each path contradict the
  pre-conditions of the unsafe state

5
Insulin delivery code

• // The insulin dose to be delivered is a function
  of blood sugar level, the previous dose //
  delivered and the time of delivery of the
  previous dose
• currentDose computeInsulin ()
• // Safety check - adjust currentDose if
  necessary
• if (previousDose 0) // if statement 1
• if (currentDose gt 16)
• currentDose 16
• else
• if (currentDose gt (previousDose 2) )
• currentDose previousDose 2
• if ( currentDose lt minimumDose ) // if
  statement 2
• currentDose 0 //
  then branch
• else if ( currentDose gt maxDose ) // else
  branch
• currentDose maxDose
• administerInsulin (currentDose)

6
Run-time safety checking

• Defensive programming
• Insert inline assertions to catch dangerous
  faults
• Can reduce runtime efficiency

7
Assessing system security

• Experience-based validation
• Apply well-known exploits
• Tool-based validation
• E.g., password checkers
• Tiger teams
• Ad-hoc attacks
• Formal verification
• Not widely used