Lusine Sarkisian

1
Automated Software Module Testing for FAA
Certification

• Lusine Sarkisian

2
Outline

• Introduction
• Tools and testing process
• FAA Certification testing
• TSE Process
• Benefits of TSE
• Limitations of TSE
• Summary

3
Introduction

• Automated module testing process for software
  written in Ada
• using Test Set Editor (TSE)
• TSE consists of set of programs that works in
  combination with Microsoft Excel spreadsheet
  program and home grown test scripts written in
  Tcl/tk.
• Automation aims to achieve both reduced costs and
  a more thorough analysis, testing and
  verification.

4
Tools and testing process

• TSE helps to achieve shorter turnaround time and
  cut cost during regression testing.
• TSE is a multi-purpose test tool set designed for
  use with Ada.
• TSE includes tools
• To identify inputs/outputs of the module
• Building a driver program to apply a set of test
  cases to the module
• Measuring coverage

5
FAA Certification testing

• Testing safety-critical software begins with a
  set of requirements-based cases that should also
  achieve structural coverage.
• Three levels of structural coverage
• statement coverage
• Decision coverage
• Modified condition-decision coverage (MCDC)

6
TSE Process

• Identifying inputs/outputs of a module
• Selecting test cases and tagging them with
  requirements
• Constructing a test driver and stubs for
  subprograms called by the module under test
• Compiling and running the tests
• Analyzing coverage

7
Example
8

9

10
Test results
11
Report
12
Benefits of TSE

• TSE doesnt require to be an expert programmer in
  Ada.
• Analyzes the source code and identifies
  inputs/outputs of a given module
• Coverage information is provided two ways
• tabular form,
• HTML form
• Easy to maintain

13
Benefits of TSE (cont)

• Self-verifying
• Repeatable
• Domain-independent
• Only requirement is to create test cases in
  spreadsheet file.

14
Limitations of TSE

• It is not possible to construct one generic
  module test to verify all instantiations.
• TSE driver is not suitable for testing an
  independent thread.
• Difficult for private type parameters.
• TSE cant test subprograms nested deeper as
  independent modules.

15
Summary

• As the quality of software is becoming
  increasingly important, the testing of that
  software becomes crucial for ensuring high
  quality for safety-critical software.
• TSE is used in the structural testing of FAA
  certified commercial avionics software as well as
  the qualification of tools used in the production
  of such software.