Managing Verification Activities Using SVM System Verification Manager

1
Managing Verification Activities Using SVM
System Verification Manager

• Bruce H. Krogh
• Carnegie Mellon University

Funded by the DARPA MoBIES Program
Model-Based Integration of Embedded Software
2
The System Verification Manager (SVM)

• A single portal into interconnected views of
• system requirements
• system architecture
• system models
• application of verification methods
• results of verification activities
• Target Users
• Embedded system development teams in production
  environments advocating model-based methods

3
SVM Features

• Requirement traceability
• Associates requirements and system architecture
  with multiple external model representations
• Verification activity management
• identifies verification activities that need to
  be re-executed when changes are made
• Verification result management
• access to the status and results of
  requirements-driven verification activities
• Extensible, open framework
• Flexible definition and reuse of models and
  verification methods

4
The SVM Team

• Bruce KroghAnsgar Fehnker
• Zhi Han
• Jim KapinskiRajesh Kumar
• Peter Feiler
• John Walker
• Gopalan Raghavachari
• Shiva N. Sivashankar
• Swami Gopalswamy
• Jit Ken Tan
• Bill Aldrich
• Eric Lim
• Mehran Mestchian

CarnegieMellon
5
Fundamental Views in SVM
Requirements
System Architecture
System Models
Block Diagrams
Consolidated Window
6
Requirements

• Imported from requirements documents
• Verification driven from requirements nodes
• Displays verification status
• Verification status updated when models are
  changed

7
Models

• SVM model objects point to external models
• models can be associated with
• requirements for verification activities
• elements of the system architecture
• models can be hierarchical

8
Importing Existing Simulink Models

• SVM extracts model information library
  dependencies
• user identifies relevant verification
  parameters, inputs, and outputs

9
Configurable Model Attributes
10
Attribute Definition Interface
11
Configurable Model Attributes

• Annotate system models with user-defined
  information
• Attribute definition contains name, data type,
  default value, list of value choices
• Edit attribute value in-place in the System
  Models View
• Show/hide attribute columns
• Future enhancements that will use attributes
• Filter viewable models by attribute values
• Data dictionary

12
System Architecture

• two views tree and block diagram
• reference for model variants of the same system
• supports consistency checking on annotations

13
Dealing With Multiple Models

• Multiple models for system architecture
• Associate multiple Simulink versions
• Associate Checkmate model
• Associate source code
• Maintain consistency between models
• Validate all models against architecture

14
Deriving A System Architecture
Creates an architecture from asingle model or
collection of models
15
Verification Methods - Registration

• methods implemented as MATLAB m-code
• registration builds the VM object
• identifies VM variables
• provides user cues

16
Verification Activities

• registered VMs can are applied to specific
  requirements as verification activites (VAs)
• identifies
• models
• variables
• results

17
Executing Verification Activities
Automatic execution of verifications Recording
result status and results Filtered views
viewing external result representations
18
Requirements Logic

• verification folders support requirements logic
  for groups of verification activities

19
Test vector generation in SVM

• Provided a SVM project that illustrates TVG
• Ford powertrain example
• GE (SF coverage)
• Gear shift example

Defined a TVG verification method for SVM
20
Model-order reduction in SVM

• Various model order reduction algorithms
• Order of the reduced block given by user or
  determined automatically from the given
  tolerance.
• Validate the reduced-order model by comparing
  simulation trajectories

21
Model Checking in SVM
Executing a modelchecking activity

• uses Cadence SMV
• user specifies input and output files
• activity returns true if all properties specified
  in SMV file are verified

22
Change Propagation Reverification

• Recognize changes in external models
• Recognize changes in verification parameters
  data sets
• Handle model library dependencies
• Invalidate reverify verification activities
• Identify potentially impacted related models

23
Application Signal Classification System

• signal processing system to classify types of
  incoming signals
• algorithms designed from interconnected signal
  processing elements
• prototyped in MATLAB M-code OR Simulink
• implemented in C for target multi-processor
  architectures

24
Component Designs in Simulink
25
Verification of implementations

• verification task demonstrate implementation
  realizes the prototype
• verification method compare results for hundreds
  of test cases

26
SVM Requirements and Activities
27
SwRI Automated OpBlock Verification
Legacy System Test Files
Reference Signal Analyzer
Clone Verification Activity
Instrumented Signal Analyzer
Compare Outputs
Captured Reference OpBlock I/O
28
OpBlock Verification Method Results
Specifying Parameters
Viewing Results
29
Repeat Verifications for Different Data
Cloned verification activities
Specify multiple data files using wild cards
30
Ford Application

• Verification of Embedded Software in Fords first
  Hybrid Electric Vehicle Escape HEV
• Ford engineers are manually verifying embedded
  controller software in closed-loop in a dSPACE
  based HIL system
• For an experienced engineer, execution of
  Target Tests (smaller subset of the complete set)
  takes around 8 hours
• SVM Usage for production software testing
• SVM used to automate and increase the number of
  test dimensions for the Target Tests
• Target Test execution using SVM around 1 hour
• Test execution using SVM does not require an
  experienced engineer
• Interface to dSPACE Real-Time platform is
  available as one of the SVM Verification Methods

31
Test Setup for SVM at Ford
Laptop
Desktop PC
Calibration Software ATI Vision
HIL Software dSPACE ControlDesk
ASAP3
serial
API
Test Engineer launches tests via SVM
custom
custom
Signal Conditioning BreakoutBox
HIL System (Plant Simulator)
Production Controller
32
(No Transcript)
33
Why is this paper in this conference?

• Formal methods use lots of models and
  abstractions
• SVM provides an environment for managing the
  models and results
• Next step Reasoning about the results from
  multiple verification activities

34
Heterogeneous Verification
SVM Single source for all verification analyses
related information of interest.
Heterogeneous database

• store verification data in processable form.
• Heterogeneous data represented in a verification
  ontology derived representation scheme.
• Extensible scheme to enable new aspects of
• verification to be represented.

Query interface
Reasoning on heterogeneous information

• Queries on heterogeneous information for
  verification, consistency, assumption tracking,
  what-if analysis.
• E.g. show the assumptions which violated cause
  rise time requirement to be violated.
• Querying language based on logic programming
  and a purpose specific logic.