Certificate Management: A Practitioners Perspective

1
Certificate Management A Practitioners
Perspective

• Dr. Michael W. Whalen
• Advanced Computing Systems
• Rockwell Collins
• 400 Collins Road NE, MS 108-206
• Cedar Rapids, Iowa 52498
• mwwhalen_at_rockwellcollins.com

2
Who Are We?
A World Leader In Aviation Electronics And
Airborne/ Mobile Communications Systems For
Commercial And Military Applications
3
Advanced Technology Center
Government Systems
Commercial Systems
Advanced Technology Center

• The Advanced Technology Center (ATC) identifies,
  acquires, develops and transitions value-driven
  technologies to support the continued growth of
  Rockwell Collins.
• The Automated Analysis group applies mathematical
  tools and reasoning to the problem of producing
  high assurance systems.

4
Automated Analysis Section
1992
AAMP5 Microcode Verification (PVS)
NASA LaRC Funded
NSA Funded
AAMP-FV Microcode Verification (PVS)
1994
AFRL Funded
AAMP5 Partitioning (PVS)
Tech Transfer
1996
JEM Java Virtual Machine (PVS)
FGS Mode Confusion Study (PVS)
1998
FCP 2002 Microcode (ACL2)
2000
AvSSP
AAMP7 Separation Kernel (ACL2)
NASA
FGS Safety Analysis (RSML-e)
FGS Mode Confusion (RSML-e)
NSA
AFRL
2002
vFaat (ACL2, PVS)
FCS 5000 FGS Verification (NuSMV)
SHADE (ACL2)
GreenHills Integrity RTOS (ACL2)
2004
Displays Verification (NuSMV)
2006
5
What are Our Problems?

• Safety-Critical Software Is Too Expensive
• Safety-Critical Software Is Often Wrong
• FAA DO-178B Software Certification Is Too
  Expensive

Cut Development Costs/Cycle Time in Half
Find 10x More Errors than Current Methods
Technology Must Be Applicable to DO-178B
Development
6
Convergence of Two Trends
Model-Based Development
Automated Analysis
A Revolutionary Change in How We Design and
Build Systems
7
How Do We Reduce Costsand Improve Quality?
Requirements Elicitation
Reuse
Autotest/ Autocert
20
Modeling
5
Simulation
Autocode
Automated Analysis
10 - 20
Cheaper Than Manual Analysis
Finds the Really Hard Errors
8
RCI/UMN Simulink Analysis Tool Chain
NuSMV
SCADE
ACL/2
Lustre
PVS
Safe State Machines
Design Verifier
Rockwell Collins
Esterel Technologies
SRI International
MathWorks
Reactive Systems
9
Example - ADGS-2100 Adaptive Display Guidance
System
4000 Subsystem Instances 16,000 Simulink Blocks
Requirement Drive the Maximum Number of Display
Units Given the Available Graphics Processors
Counterexample Found in 5 Seconds!
Checking 563 Properties Found 98 Errors
10
Tech Transfer Process Improvements
Dev. Group(Blue)
ATC Group(Beige)
Translation Time 10 MinutesTurnaround 3 Hours
to 2 Days
Translation Time 1-4 HoursTurnaround 1 Day to
1 Week
Translation Time 10 MinutesTurnaround 10
Minutes
11
What IS a Software Certificate?
Software certification demonstrates the
reliability, safety, or security of software
systems in such a way that it can be checked by
an independent authority with minimal trust in
the techniques and tools used in the
certification process itself.

• Certificates derive authority from
• Mathematical relationships between software
  artifacts
• Personal authority
• Its o.k. because I say so
• Process authority
• Its o.k. because we followed the steps of
  process X
• Informal certificates are currently used for
  critical software (DO178B documentation process)
• Research agenda here centers on formal
  certificates

12
DO178B and Formal Certificates

• DO178B Guidelines call out several design/code
  safety properties
• Objective A-4.2 Low-Level Requirements are
  Accurate and Consistent
• No arithmetic overflow/underflow
• No division by zero
• All variables are initialized-before-use
• No unsafe use of equality operators with floating
  point types
• Correct and consistent use of physical units
• Objective A-4.3 Low-Level Requirements are
  Compatible with Target Computer
• Numeric types used in requirements are supported
• WCET of code generated from model is acceptable
  in target HW
• Precision of floating and fix-point numbers is
  sufficient

13
DO178B and Formal Certificates

• Functional correctness
• Objective A-4.7 Low-Level Requirements are
  Traceable to High-Level Requirements
• We are using model checkers extensively to show
  correspondence
• However, no evidence is generated if checker
  returns true
• Correctness of code-generation tools
• Development tool qualification is extremely
  difficult
• Evidence-based Equivalence of Generated Code to
  Model would be VERY interesting
• Entire tables (A.5, A.6) could be eliminated with
  proper evidence

14
What Are Certificate Management Systems?

• In a Critical Software Development Process, there
  are several related tools
• System build
• Configuration management
• Bug tracking
• Workflow and process
• Reporting
• Traceability Management
• Development process generates a mix of formal and
  informal artifacts
• Informal artifacts tend to predominate
• Where do Certificate Management systems fit?

15
What Are Certificate Management Systems?
16
One View They Do It All

• View from the web page
• Manage workflow process
• Formally describe development process
• Manage system build process
• Manage traceability between artifacts
• Manage audits and reporting
• Incremental recertification
• Butthis is a lot of engineering
• I believe that current tools provide necessary
  support for informal certificates

17
My View Certificate Management System as Plugin
Certification Mgmt.
Requirements Management Tool
Configuration Management Tool
Generated Config.
18
My View Certificate Management System as Plugin

• Let Certificate Management checkers act as new
  gatekeeper/make tool with existing tools
• It makes sense to support more than one
  Certificate Management checker
• Artifacts (including, now, proofs) have to live
  for 30 years on commercial aircraft
• Checkers will probably evolve over course of dev.
• It certainly makes sense to have several
  certificate generators

19
Some Cautions on Limits of Formality

• Using Software to Certify Software
• Several translation steps get swept under the
  rug
• Even with proof checkers, issues with is
  property correct? Is formalism correct?
• Recertification Infrastructure / Make tools are
  they correct?
• IBM Study 20 of properties submitted to a model
  checker are at least partially vacuous
• Formal Proofs get more difficult as we get closer
  to the bare metal
• Software Runs on a Platform
• Operating System / HW
• Communications Media

20
Research Directions

• Generating Formal Evidence
• Automated Deduction for PCC-style properties
• Proof-Generating Model Checking Tools
• Evidence-Generating Compilers
• Formulating New Safety Policies to Match Critical
  Concerns
• How do we formulate resource boundedness?
• Numeric precision?
• Proof Representation and Storage
• Incremental Re-verification
• Meta Proofs about Evidence?

21
Discussion
22
My Off Topic Slide
Rockwell Collins is a World Leader in the
Industrial Use of Formal Methodsand Were
Looking for Good People!

• We Have 15 Years of FM Industrial Experience
• Thriving Automated Analysis Section
• Doing Extensive Work for NASA and the NSA
• Broad Tool Expertise
• PVS, ACL2, NuSMV, Prover, SAL, Simulink, SCADE,
  SCR,
• Focus on Application to Real Systems