THE SCR TOOLSET

1

• THE SCR TOOLSET

2
TOOLS FOR SPECIFYING AND ANALYZING REQUIREMENTS

• The SPECIFICATION EDITOR supports the creation of
  precise, unambiguous requirements specs.
• The DEPENDENCY GRAPH BROWSER displays the
  relationship between variables in spec.
• The CONSISTENCY CHECKER checks that the specs
  satisfy application-independent properties.
• The SIMULATOR allows developers to ensure that
  the specs satisfy their intent.
• The VERIFICATION TOOLS support checking of
  application properties.

SPECIFICATION EDITOR
DEPENDENCY GRAPH BROWSER

tabular
specs

formal requirements specs
modes
terms
monitored variables
events
controlled variables
conditions
CONSISTENCY CHECKER
SIMULATOR
VERIFICATION TOOLS
supports validation
support verification
3
THE REQUIREMENTS ARE ORGANIZED INTO DICTIONARIES
AND TABLES

Dictionaries

Tables
4
A VARIABLE DICTIONARY
Lists each monitored variable, controlled
variable, and term along with its type, initial
value, and required precision.
5
A MODE CLASS DICTIONARY

• Lists each mode class along with its
• member modes
• initial mode

6
CONSTANT AND TYPE DICTIONARIES

• Lists each mode class along with its
• member modes
• initial mode

7
DEPENDENCY GRAPH BROWSER
GRAPH OF VARIABLE DEPENDENCIES FOR SAFETY
INJECTION SYSTEM
8
CONSISTENCY CHECKING

• Tests application-independent properties --
• consistency of spec with our requirements model
• Most individual checks require a single table and
  the type definitions

• Proper syntax
• Type correctness
• Completeness of entity definitions
• Disjointness
• Coverage
• Initial values
• Reachability
• Lack of circularity

9
COVERAGE ERROR IN A CONDITION TABLE
Some missing cases, namely,
Ù
(
)
Sta1Rdy
yes
Sta8Rdy
yes

Ú

WeapType

0
10
DISJOINTNESS ERROR IN AN EVENT TABLE

• In this example, the tool has detected an
  instance of nondeterminism.

Ù
Ù
11
DETECTION OF A CYCLE
A Circular Definition
Overridden is defined in terms of Pressure
and Pressure is defined in terms of
Overridden
12
APPLYING CONSISTENCY CHECKING TO A-7 REQ.
DOCUMENT RESULTS
EXPERIMENT 1

• Checked All 36 condition tables, a total of 98
  rows
• Results 17 rows in 11 tables in error

EXPERIMENT 2
Checked Three mode transition tables, a total of
700 rows (4319 logical expressions) Results 57
instances of nondeterminism detected!
Consistency checking for Coverage and
Disjointness errors is very cheap in comparison
with other analysis techniques, such as
reachability analysis
13
DETECTING AMBIGUITY WITH THE CONSISTENCY CHECKER

• Proper syntax
• Type correctness
• Complete entity definitions
• Disjointness
• Coverage
• Initial values
• Mode reachability
• Lack of circularity

Excerpt from 14-page table

Arrows point to ambiguity


counterexample
For each error detected, the consistency checker
displays 1. The table containing the error
with erroneous entries highlighted 2. A
state pair demonstrating the error
Event that could trigger either transition
_at_T(Doppler_up) WHEN NOT CA_stage_complete AND
latitude gt 70 deg. AND NOT
present_position_entered AND NOT latitude gt
80 deg. AND IMSMODEGndal
14
USING THE SIMULATOR FOR VALIDATION
Simulator Display
Simulator Log
System State
Next Event
Executed Events
Dependent Vars
Monitored Vars
15
CHECKING ASSERTIONS WITH THE SIMULATOR
Simulator Display
At step 4, the simulator has detected a
violation of an assertion. Clicking on the
warning msg. in the log highlights the failed
assertion.
Simulator Log
Assertion Dictionary
ASSERTION BombRelease on ??ReleaseEnable on
16
A FRONT-END FOR THE SIMULATOR



17
MODELING TIME INSCR SPECIFICATIONS
18
EXAMPLE SAFETY INJECTION IN A NUCLEAR PLANT
REQ and NAT
Low
time
0
SafetyInjection
On
time
Off

• Use a Duration operator to represent the time a
  given condition has
• been true. E.g., Duration(WaterPresLow)
  2sec.
• Need to specify the time that an output event
  must occur.
• E.g., if?? is the time that _at_T(WaterPresLow)
  occurred, then the time
• that _at_T(SafetyInjOn) occurs must be in
  ??????950ms, where ?????ms.

19
USING Duration TO EXPRESS TIMEIN A WEAPONS
SYSTEM SPEC

• In Nattack mode,
• the pilots press of
• ReleaseEnable when
• MissDistance ?????
• causes the system to
• turn BombRelease on.
• When BombRelease
• has been on for at
• least FirePulseWidth
• ms, the system turns
• BombRelease to off.

20
SIMULATING TIME PASSAGE IN SCR
Effect of Time Passage on Bomb Release
Create clock variable _DUR_ReleaseEnab for
ReleaseEnableon and set it to zero.
Time passage of 1000 ms


System in NAttack mode and FirePulseWidth
becomes 27

Pilot presses ReleaseEnable to on. When
MissDistance ??????system turns BombRelease to
on. Clock variable _DUR_ReleaseEnab starts
count.
After 27 ms, _DUR_ReleaseEnab becomes 27
(1027-1000), causing the system to turn
BombRelease off.
21
VERIFICATION OF SCR SPECIFICATIONS
22
VERIFICATION TOOLS
VERIFICATION TOOLS
DEPENDENCY GRAPH BROWSER
SPECIFICATION EDITOR

• The MODEL CHECKERS check properties of finite
  state models.
• The INVARIANT GENERATOR generates state
  invariants automatically.
• The INVARIANT CHECKER analyzes properties using
  decision procedures.
• The GENERAL-PURPOSE PROVER is for more
  general-purpose theorem proving.

formal requirements specs
modes
terms
monitored variables
events
controlled variables
conditions
CONSISTENCY CHECKER
SIMULATOR
SIMULATOR
MODEL CHECKER (SPIN)
GENERAL-PURPOSE PROVER (PVS)
Integrated into toolset
Have applied to SCR specs but not integrated
MODEL CHECKER (SMV)
INVARIANT GENERATOR
INVARIANT CHECKER
23
CURRENT PROBLEMS IN MODEL CHECKING

• Recently, model checking has proven highly
  effective in detecting errors, especially in
  hardware designs and communication protocols
• Major problem is state explosion
• Model checking is only feasible for abstractions
  of the specification
• In most approaches, however, establishing the
  correspondence between the abstraction and the
  original specification is based on intuitive
  arguments
• In other approaches, user ingenuity is required
  to produce the abstraction

MODEL CHECKING
finite state machine model
Abstract Machine ?A
property q
Currently, this step is nearly always ad hoc
logic formula
Specification of State Machine ?
?
?
?
???
?
? ?A and ?A q implies ? q
24
REASONING ABOUT A SPECIFICATION WITH AN ABSTRACT
MACHINE
Abstract machine ?? state space S? initial
state predicate ?? next-state relation ??
State machine ? state space S initial state
predicate ? next-state relation ?
????S ??S??
S?
S
abstraction mapping
property q
property qA

• Given a property q, we want the following to
  hold
• q? is an invariant of ????implies that q is an
  invariant of ??(soundness)
• q is an invariant of ???iff qA is an invariant
  of ???(completeness)
• (thus, a counterexample to qA is found
  in ?? implies q is not an invariant of ??
• Two kinds of invariants are of interest
• properties of each reachable state (state
  invariants)
• properties of each pair of reachable states in
  relation ? (transition invariants)

25
THREE AUTOMATABLEABSTRACTION METHODS
Opening the vent valve shall be prevented unless
the differential pressure is within safe limits
ABSTRACTION METHOD 1REMOVE IRRELEVANT VARIABLES
ABSTRACTION METHOD 3 CREATE NEW ABSTRACTIONS
Pressure
WaterPres
SafetyInjection
Reset
Overridden
Block
ABSTRACTION METHOD 2 USE EXISTING ABSTRACTIONS

• Each abstraction method is sound
• We have identified conditions which
• often hold that imply completeness
• for Methods 2 and 3

WaterPres
Pressure
SafetyInjection
Reset
Overridden
Block
R. Bharadwaj and C. Heitmeyer, Model checking
complete requirements specifications using
abstraction, Journal of Automated Software
Eng., Jan., 1999.
26
ABSTRACTION METHOD 1ELIMINATE IRRELEVANT
VARIABLES

• Include in a set O all variables that appear in
  the formula
• Take the reflexive and transitive closure O of
  the dependency sets of
• all variables in O

O Reset, Pressure, Overridden
O Block, Reset, WaterPres, Pressure,
Overridden
Pressure
WaterPres
SafetyInjection
Reset
Overridden
Extracting the SCR spec for ???from the SCR spec
for ? is extremely simple.
Block
27
ABSTRACTION METHOD 2 USE EXISTING ABSTRACTIONS
OF MONITORED VARS
In this example, we can use the mode class
Pressure as an abstract model of WaterPres
Pressure
WaterPres
SafetyInjection
Reset
Overridden
The dependence of Pressure on WaterPres is easily
removed by eliminating all references
to WaterPres from the table defining Pressure.
Block
28
ABSTRACTION METHOD 3 CREATE NEW ABSTRACTIONS
OF MONITORED VARS
Opening the Torpedo Tube Vent Valve shall be
prevented unless the Missile-to-Torpedo-Tube
differential pressure is within safe limits.
Suppose a monitored var represents the real
number read by a transducer. Based on the
transition relation and the above property, this
detailed variable can be replaced with a new
abstract variable whose type set is I0, I1 ,
..., I6.
7.7
9.2
14.8
21.0
1.8
15.3
I0
I2
I1
I3
I4
I6
I5
assumed transducer failure
assumed transducer failure
minimum allowable for launch
maximum allowable for launch
29
SOUNDNESS OF THE METHODS
THE FIRST TWO METHODS USE VARIABLE RESTRICTION
The values of state sA in SA
RFA values
The values of state s in S

• q depends only on these
• syntactically, qA q

RF-RFA values
Variable restriction methods are always sound
THE THIRD METHOD USES VARIABLE ABSTRACTION
Equiv. class of states in S, one for each value
of r
RF - r values
Same RF - r values
A value set for r
One value of r

• q depends only on these
• syntactically, qA is the same
• as q except for uses of r and r

Variable abstraction methods are always sound
30
COMPLETENESS OF THE METHODS

• Using our theorem, it can be shown that
• the first abstraction method is complete
• the second abstraction method is complete if two
  given
• equivalent states are always connected by
  transitions in ???i.e.,
• the third abstraction method is complete if one
  has the same
• picture as above, where all r values have the
  same f(r) value

s?
S?
?
S
??
s
s??
?
s?
abstraction mapping
some r value
Same RF - r values
Note Entity r is an input variable
31
TAME A SPECIALIZED INTERFACE FOR PVS

• Objective
• Reduce human effort needed to encode
  specifications and reason about them with a
  mechanical theorem prover

• Design Goals
• TAME specs should be easy to create from typical
  automaton descriptions
• The formulation of properties should be natural,
  not biased for the prover
• Proof steps should match in size and kind steps
  used in hand proofs
• TAME proofs should be recognizably similar to
  hand proofs

TAME
Timed Automata Modeling Environment

• Why build upon the general-purpose theorem prover
  PVS?
• Avoid reinventing well-known, existing techniques
• Simplify extension of types of proof support for
  automata models
• Take advantage of increased flexibility for proof
  support from higher-order logic
• State automata properties in the expressive logic
  of the general-purpose prover

32
TAME PROVIDES A TEMPLATE FOR DESCRIBING AN
AUTOMATON MODEL

• To define a timed automaton, the TAME user simply
  fills in the following
• Any relationships among the constants in the
  specification
• (optional defaults to true)
• Declarations of the non-time-passage actions of
  the automaton
• A basic state record type whose fields represent
  the state variables
• Any state predicate desired to restrict the state
  space (optional defaults to true)
• The specified preconditions for the
  non-time-passage actions
• The effects (if any) of time passage on the basic
  state variables
• The effects of the non-time-passage actions on
  state variables other than now
• The initial state predicate, which must imply now
  0

Usually, some auxiliary declarations of types and
constants are also needed.
33
HUMAN-STYLE HAND PROOFS VS. PROOFS DONE USING
PVS
HUMAN-STYLE

Close human?PVS match
Bad PVS ? human match

High level human steps need PVS strategies
34
MAJOR TAME STRATEGIES
35
TECHNICAL PROBLEMANALYZING SPECS CONTAINING
NUMBERS

• Generally, one cannot use a single decision
  procedure to decide formulas
• in typical Navy applications.
• Examples
• Steering_StageOTS???Range_to_Rmaxlt0.0???A/C_fac
  ing_tgt
• ?Iphase4.0???Mach?MSW1???Contoff)????MM?on???Co
  nton???EOWstat)
• These formulas contain variables of type real,
  integer, boolean, and enumerated.
• Such formulas cannot be decided by conventional
  decision procedures which
• analyze formulas whose variables range over
  some but not all of these types.

EXAMPLE CONSISTENCY CHECKING OF SCR SPECS

• Consistency checker (CC) analyzes formulas of
  the form
• c1 ? c2 false
• c1?????c2?????...???cn true
• CC analyzes 95 of the formulas in SRSs
  correctly and efficiently.
• CC has difficulty with the remaining 5
• returns dont know (rather than yes or no
  with a counterexample)
• requires excessive computing resources (e.g.,
  runs out of memory)

DISJOINTNESS PROPERTY COVERAGE PROPERTY
MOST PROBLEMATIC FORMULAS CONTAIN NUMBERS!
36
SALSA, AN INVARIANT CHECKER, COMBINES DECISION
PROCEDURES
SCR tabular spec
Salsa contains 30,000 lines of source code
Parser
The UNSATISFIABILITY CHECKER integrates a BDD
algoroithm and an integer linear constraint
solver.
Term Rewriting
Generate Formula
Reduce Formula
formula is valid
enumerated types
booleans
integers
formula is invalid counterexample
UNSATISFIABILITY CHECKER
R. Bharadwaj and S. Sims, Salsa Combining
Constraint Solvers with BDDs for Automatic
Invariant Checking, TACAS 2000 , Berlin,
Germany, March 2000 (to appear)
37
EVALUATING FORMULAS USING BDDs AND A CONSTRAINT
SOLVER
To evaluate whether the following formula is
satisfiable, i.e., has any solutions

• Create BDD variables encoding the
  terms as follows
• Build a BDD for
• Determine paths to true, i.e.,
• Check whether the formula
• is satisfiable, etc.

a
true
false
b
c
true
false

• This BDD is unsatisfiable if
• it is false OR
• all paths from the root to true are infeasible

38
THE INTEGER LINEAR CONSTRAINT SOLVER

• Based on algorithm originally proposed by Boudet
  and Comon for solving
• linear Diophantine equations
• Uses FSMs to determine whether a relation
  defined on integers has any solutions

Example Check whether
is satisfiable
accepts bb0000
accepts all strings except bbb000
0
1
0,1
0,1
0,1
0,1
0,1
1
0
FSM checking constraint
FSM checking constraint
Because the intersection of the accepting states
of the two FSMs is empty, the formula is
unsatisfiable.
The emptiness of the language accepted by an
automaton is decidable.
39
AUTOMATIC INVARIANTGENERATION

• We have developed an algorithm that automatically
• generates state invariants from SCR
  specifications
• Uses of automatically generated invariants
• to validate the specification
• to use as auxiliary invariants in proving other
  system properties

MOff ??IgnOn M Cruise ??IgnOn ??EngRunning ??
Brake ??Lever off M
Override ??IgnOn ??EngRunning M Inactive
??IgnOn
/
MODE INVARIANTS IN A CRUISE CONTROL SPEC
40
ALGORITHM FOR GENERATING MODE INVARIANTS
Compute Mode Entry Conditions C is the
disjunction c1?c2?... ?cm of conditions true
when mode entered
Compute Unconditional Exit Set Set
d1,d2,...,dn of simple Boolean conditions
whose falsification causes unconditional exit
from mode

• To strengthen invariant, use
• initial state predicate
• environmental constraints
• invariants computed on
• earlier passes

Compute Mode Invariant Keep just
the di found in C
REPEAT THESE THREE STEPS UNTIL A FIXPOINT IS
REACHED
R. Jeffords and C. Heitmeyer, "Automatic
Generation of State Invariants from Requirements
Specifications," 6th International Symposium
on the Foundations of Software Engineering
(FSE-6), Orlando FL, Nov. 3-5, 1998.
41
MODE INVARIANTS GENERATED FROM A-7 REQUIREMENTS
DOCUMENT
MUNone ??true M HUDUpd ??StnSelected ??ACAirb
??ModeRotarySwPresPos ?? GunSel
?? PresPos Update??? UpdateTW
HUD ??MFSWNone MAFlyUpd ??ACAirb
??ModeRotarySwPresPos ??PresPos Update
??UpdateTW Flyover ??Weapons_ModeBOC
??MFSWBOC M RadarUpd ??StnSelected
??ModeRotarySwPresPos ?? PresPos Update?
?? UpdateTW Radar
??GunSel ??MFSWNone M TacUpd
??ModeRotarySwPresPos??? UpdateTW TacL-L
??GunSel ??
PresPos Update????MFSWNone or MFSWTF) M
MapUpd ??ModeRotarySwPresPos ?? UpdateTW
Flyover ??GunSel
??PresPos Update????MFSWNone or MFSWTF) M
FlyUpd ???ModeRotarySwPresPos ?? UpdateTW
Flyover ??GunSel
??PresPos Update????MFSWNone or MFSWTF)
MODE CLASS M Navigation_Update
42
SPECIFICATION-BASED TESTINGBASED ON SCR
43
USING THE SCR SPECIFICATIONS DURING THE LATER
DEVELOPMENT PHASES
SCR TOOLSET
DEVELOPMENT PHASE
tabular specs
SPEC EDITOR
REQUIREMENTS
SIMULATOR
. . .
CONS. CHECKER
REQUIREMENTS SPEC
REQUIREMENTS SPEC
LATER PHASES
DESIGN
AUTOM. CODE GENERATOR
CODING
AUTOM. TEST CASE GENERATOR
TESTING
44
EXTENDING SCR TO AUTOMATIC TEST SET GENERATION
SYSTEM REQ. SPEC
OUR FOCUS
Test Set

• Our approach to software testing
• specification-based
• blackbox--does the software satisfy
• the requirements specification?

45
SOME BASICS

• Test Sequence Sequence of inputs, each paired
  with a set of outputs
• Test Set A collection of test sequences
• Goals of Test Set Generation
• The number of test sequences in the test set
  should be as small
• as possible
• The test set should cover all errors that any
  implementation
• may contain
• Our approach to generating test sequences Use a
  model checker
• To construct the test input data (sequence of
  inputs)
• As an oracle--given a sequence of inputs, we use
  the model checker
• to compute the set of expected outputs

How to generate test sequences? USE TRAP
PROPERTIES
46
CONSTRUCTING TRAP PROPERTIES FROM SYSTEM
PROPERTIES
Suppose the SCR spec of SIS satisfies the
following safety property
_at_T(WaterPres lt Low) WHEN Block On ? Reset
Off ? SafetyInjection Off

• How do we generate a test sequence from this
  property?
• Translate the SCR specification into the lang.
  of a model checker, say SMV
• Translate the negation of the above propertys
  hypothesis into CTL

AG!( EX(WaterPresltLow) ! WaterPresltLow
Block On Reset Off
The above property is an example of a TRAP
PROPERTY
47
GENERATING A TEST SEQUENCE FROM A SYSTEM PROPERTY

• If Spec ? P, then Spec ? P will generate a
  counterexample
• This counterexample is the basis for a test
  sequence that can be used to test the correctness
  of the software

Test Sequence Constructed from SMV Counterexample
The above test sequence may be represented more
concisely as lt (r,off -), (w,5 -), (w,8 -),
(w,10 s,off), (b,on -), (w,8-) gt
48
WEAKNESSES OF THIS APPROACH

• A set of system properties may not be available
• The approach is based on the assumption that both
  the specification and the properties are correct
• In our experience, producing a high-quality
  specification is feasible
• Verifying that the spec satisfies a set of
  critical properties is more difficult
• The set of system properties may be incomplete
  --i.e., all possible system behaviors may not be
  covered

49
EXAMPLE AN EVENT TABLE FROM THE SCR SPEC OF THE
SIS
Event Table Defining the Mode Class Pressure
if Pressure TooLow ? _at_T(WaterPres ? Low)
? Pressure Permitted Pressure
Permitted ? _at_T(WaterPres ? Permit) ?
Pressure High Pressure Permitted ?
_at_T(WaterPres lt Low) ? Pressure TooLow
Pressure High ? _at_T(WaterPres lt Permit) ?
Pressure Permitted (else) ?
Pressure Pressure fi
Total Function That the Table Defines (Single
else Clause)
50
CONSTRUCTING CASES FROM AN EVENT TABLE (1)
if Pressure TooLow if _at_T(WaterPres ?
Low) ? Pressure Permitted C1 (else) ?
Pressure Pressure C2 fi
Pressure Permitted if _at_T(WaterPres ?
Permit) ? Pressure High C3
_at_T(WaterPres lt Low) ? Pressure TooLow
C4 (else) ? Pressure Pressure
C5 fi Pressure High if
_at_T(WaterPres lt Permit) ? Pressure Permitted
C6 (else) ? Pressure Pressure
C7 fi fi
Alternate Representation of the Function With One
else Clause per Mode
51
CONSTRUCTING CASES FROM AN EVENT TABLE (2)

• Each part of the function definition is called a
  case
• Each case defines a set of state transitions
• Because each function is total, the set of cases
  cover the entire state space
• Because the cases are mutually exclusive, each
  case is an equivalence class of state transitions

For example, case C1 defines the set of state
transitions that satisfy the following property
Pressure TooLow ? _at_T(WaterPres ? Low) ?
Pressure Permitted
52
OTHER APPROACHES
53
REQUIREMENTS STATE MACHINE LANGUAGE (RSML)

• combines graphical and tabular notations
• system modeled as set of concurrent finite
  state machines (patterned after Statecharts)
• each table describes the conditions on the
  transitions under a single triggering event in
  a given state machine
• there is only one source state, dest. state,
  and triggering event per table
• no use of modes
• an output action may be associated with a
  transition
• input and outputs are represented as signals
• specifies all behavior explicitly (including
  no change)

Heimdahl and Leveson, Completeness and
Consistency Analysis of State-Based
Requirements, 17th Intern. Conf. on Software
Eng., Seattle, WA,1995, pp. 3-14.
54
TABLEWISE

• latest in a series of approaches that use
  decision
• tables to specify the required software
  behavior
• improvement over previous approaches because
• it handles nonbooleans
• uses a single table to specify the entire system
• behavior
• the table looks like a condition table
• an output action is associated with each
  condition
• can generate code from the tables
• no use of modes

Hoover and Chen, Tablewise, A Decision Table
Tool, 10th Annual Conf. on Computer Assurance,
Gaithersburg, MD, June, 1995, pp. 97-108.