Study Findings Part 1

1
Study Findings Part 1

• Carolyn Salmon
• carolyn.salmon_at_era.co.uk

2
Content of Presentation

• Why is DO-178B so important
• US Civil aerospace approach to certification
• Current UK Military approach to re-certification
• Perceived problems of DO-178B

3
Why is Do-178B so important and the context in
which software developed to DO-178B is certified
4
Why is DO-178B important?

• Global Aviation Traffic Management (GATM) require
  all commercial aircraft to comply with FAA
  regulations
• All aircraft to fly in civil airspace will need
  to comply with FAR 25.1309
• AC 25-115B specifies use of DO-178B
• De-facto document for obtaining approval of
  software

5
Implication for UK MoD

• Re-use of civil applications
• Will have software developed to DO-178B
• But how do MoD re-certify new application in a
  cost effective manner?

6
US Civil approach to certification of systems
containing software

• Software not certified stand alone
• DO-178B does not address safety management
• Carried out at system level

7
US Civil approach to certification of systems
containing software

• To achieve certification system must comply with
  requirements of FAA regulations
• FAR 25.1309
• Safe design of the system
• Demonstration of safety
• Need to do a system hazard analysis
• Identify how software behaviour could lead to the
  hazards
• Not covered by DO-178B

8
Safety Assessment Process ARP 4761
Function, Failure Safety Information
Intended Aircraft Function
System Design
System Development Process ED-79/ARP 4754
Functional system
Allocated Functions Requirements
Implementation
Hardware Development Life Cycle DO-254/ED-80
ED-94B FAQ 3.24
Software Development Life Cycle ED-12B/DO-178B
9
UK MOD Approach to Certification

• In the past, minimal acceptance of existing
  process
• Make no assumption on sufficient of supplied
  equipment
• Have to undertaken additional analysis
• e.g. Undertake post development (retrospective)
  Static Code Analysis (SCA)
• Expensive
• Time consuming
• Questionable benefit

10
What is Static Code Analysis?

• Technique for analysing code statically
• No need to simulate environment
• Requirement of Def-Stan 00-55 (withdrawn)
• Manual or automated
• Range of techniques
• Data, Control and Information Flow analysis
• Formal proofs

11
SCA Example

• Used in earnest on Hercules C-130J programme

12
SCA Example

• Number of avionics LRUs analysed
• Found 11,590 anomalies
• 6 of these could threaten safety
• But.
• Only one small, non-random sample
• Estimates of cost per aircraft type 13 million
• Concern retrospective SCA not cost effective
• Particularly for current off-the-shelf
  applications

13
Are there benefits to SCA?

• 60 of errors found during walkthroughs/
  familiarisation
• Effective
• Does not require use of tools
• Can be done at any time
• SCA undertaken during development
• Very effective if at code level
• Number of tools now available
• At complication e.g. PCLint
• Flow, control and data analysis e.g. LDRA
  Testbed
• Formal Proof Polyspace, MALPAS, Spark Ada

14
Should we do SCA?

• SCA not just technique to be carried out
  retrospectively
• Carry out at right time and huge benefits
• Suggest any competent organisation now doing as
  default
• Reduces benefit of retrospective SCA further

15
Perceived Problems with DO-178B
16
DO-178B Perceptions/Misconceptions

• Much criticism of DO-178B
• Are these appropriate?
• Perception that DO-178B is not sufficient and
  suitable
• For UK military applications has lead to a need
  to undertake additional activities to gain
  certification

17
Published Research Findings

• From published research
• Comparison with other standards
• Some key issues
• Lack of Safety Management
• Lack of formality in definition of software
  requirements
• Number of differences between 00-55 and DO-178B

18
Lack of Safety Management

• Never intended to be covered by DO-178B
• Software is certified as part of system not stand
  alone
• Some elements of DO-178B do generate evidence
• Safety Accomplishment Summary
• Derived requirements

19
Specification of Requirements

• Def Stan 00-55 required more rigour
• DO-178B includes
• Review
• Analysis
• But, emphasis in guidance on testing
• Difference is safety culture between US and UK?

20
Differences between DO-178B and Def-Stan 00-55 - 1

• YSE Paper 1999
• Clause by clause comparison
• Def Stan 00-55 includes
• Lots of additional documentation
• Extra requirements for choice of language
• More rigour in defining safety requirements
• Verification of the assignment of the software
  level

21
Differences between DO-178B and Def-Stan 00-55 - 2

• DO-178B includes
• Consideration of system architecture
• Ensuring higher level requirements not
  compromised by derived requirements
• Consideration of target computer
• More rigorous configuration control requirements

22
US Market View 1998

• Streamlining Software Aspects of Certification
• 292 questionnaires
• Based on experience with DO-178B
• Key issues
• Service Bulletin/Airworthiness Directives issued
• Lack of formality in safety requirements
• Need to relate software development and safety
  activities
• Funding withdrawn

23
Conclusions
24
Conclusions from investigations

• Following DO-178B capable of producing both good
  and bad software
• Weaknesses being addressed
• DO-178C
• Relationship between safety and software
  development
• Need to re-certify in context of new application

25
Questions