Compositional Verification part II

1
Compositional Verification part II

• Dimitra Giannakopoulou and Corina Pasareanu
• CMU / NASA Ames Research Center

2
recap from part I

• Compositional Verification
• Assume-guarantee reasoning
• Weakest assumption
• Learning framework for reasoning about 2
  components

3
compositional verification
Does system made up of M1 and M2 satisfy property
P?
M1

• Check P on entire system too many states!
• Use the natural decomposition of the system into
  its components to break-up the verification task
• Check components in isolation
• Does M1 satisfy P?
• Typically a component is designed to satisfy its
  requirements in specific contexts / environments
• Assume-guarantee reasoning
• Introduces assumption A representing M1s
  context

A
M2
4
assume-guarantee reasoning

• Reason about triples
• ?A? M ?P?
• The formula is true if whenever M is part of a
  system that satisfies A, then the system must
  also guarantee P

M1

• Simplest assume-guarantee rule ASYM

A
M2
How do we come up with the assumption A? (usually
a difficult manual process) Solution synthesize
A automatically
5
the weakest assumption

• Given component M, property P, and the interface
  of M with its environment, generate the weakest
  environment assumption WA such that ?WA? M ?P?
  holds
• Weakest means that for all environments E
• ?true? M E ?P? IFF ?true? E ?WA?

6
assumption generation ASE02
property true! (all environments)

• STEP 1 composition, hiding, minimization

STEP 2 backward propagation of error along ?
transitions
property false! (all environments)
STEP 3 property extraction (subset construction
completion)
assumption
7
learning for assume-guarantee reasoning

• Use an off-the-shelf learning algorithm to build
  appropriate assumption for rule ASYM
• Process is iterative
• Assumptions are generated by querying the system,
  and are gradually refined
• Queries are answered by model checking
• Refinement is based on counterexamples obtained
  by model checking
• Termination is guaranteed

8
learning assumptions

• Use L to generate candidate assumptions
• ?A (?M1 ? ?P) ? ?M2

Model Checking
L
string c ??A
false cex c
conjecture Ai
?Ai? M1 ?P?
true
true
?true? M2 ?Ai?
P holds in M1 M2
false cex c
false
?c??A? M1 ?P?
P violated in M1 M2
string c ??A
true

• Guaranteed to terminate
• Reaches weakest assumption or terminates earlier

9
part II

• compositional verification
• assume-guarantee reasoning
• weakest assumption
• learning framework for reasoning about 2
  components
• extensions
• reasoning about n gt 2 components
• symmetric and circular assume-guarantee rules
• alphabet refinement
• reasoning about code

10
extension to n components

• To check if M1 M2 Mn satisfies P
• decompose it into M1 and M2 M2 Mn
• apply learning framework recursively for 2nd
  premise of rule
• A plays the role of the property
• At each recursive invocation for Mj and Mj
  Mj1 Mn
• use learning to compute Aj such that
• ?Ai? Mj ?Aj-1? is true
• ?true? Mj1 Mn?Aj? is true

11
example

• Model derived from Mars Exploration Rover (MER)
  Resource Arbiter
• Local management of resource contention between
  resource consumers (e.g. science instruments,
  communication systems)
• Consists of k user threads and one server thread
  (arbiter)
• Checked mutual exclusion between resources
• E.g. driving while capturing a camera image are
  mutually incompatible
• Compositional verification scaled to gt5 users vs.
  monolithic verification ran out of memory
  SPIN06

Resource Arbiter
12
recursive invocation

• Compute A1 A5 s.t.
• ?A1? U1 ?P?
• ?true? U2 U3 U4 U5 ARB ?A1?
• ?A2? U2 ?A1?
• ?true? U3 U4 U5 ARB ?A2?
• ?A3? U3 ?A2?
• ?true? U4 U5 ARB ?A2?
• ?A4? U4 ?A3?
• ?true? U5 ARB ?A4?
• ?A5? U5 ?A4?
• ?true? ARB ?A5?
• Result
• ?true? U1 .. U5 ARB ?P?

13
analysis results
Analysis
Assumption generation
LTSA tool
14
symmetric rules motivation
Ordererr
Output
Input
send
in
send
out
out
out
in
ack
M1 Input, M2 Output, P Order
M1 Output, M2 Input, P Order
in
A2
A1
ack
send
in
ack
15
symmetric rules

• Assumptions for both components at the same time
• Early termination smaller assumptions
• Example symmetric rule SYM
• coAi complement of Ai, for i1,2
• Requirements for alphabets
• ?P ? ?M1 ? ?M2 ?Ai ? (?M1 ? ?M2) ? ?P, for i
  1,2
• The rule is sound and complete
• Completeness needed to guarantee termination
• Straightforward extension to n components

Ensure that any common trace ruled out by both
assumptions satisfies P.
16
learning framework for rule SYM
add counterex.
add counterex.
L
L
remove counterex.
remove counterex.
A2
A1
?A1? M1 ?P?
?A2? M2 ?P?
false
false
true
true
L(coA1 coA2) ? L(P)
true
P holds in M1M2
false
counterex. analysis
P violated in M1M2
17
circular rule

• Rule CIRC from GrumbergLong Concur91
• Similar to rule ASYM applied recursively to 3
  components
• First and last component coincide
• Hence learning framework similar
• Straightforward extension to n components

18
assumption alphabet refinement
M1
P

• Rule ASYM
• Assumption alphabet was fixed during learning
• ?A (?M1 ? ?P) ? ?M2
• SPIN06 A subset alphabet
• May be sufficient to prove the desired property
• May lead to smaller assumption
• How do we compute a good subset of the assumption
  alphabet?
• Solution iterative alphabet refinement
• Start with small alphabet
• Add actions as necessary
• Discovered by analysis of counterexamples
  obtained from model checking

M2
19
learning with alphabet refinement

• 1. Initialize S to subset of alphabet ?A (?M1 ?
  ?P) ? ?M2
• 2. If learning with S returns true, return true
  and go to 4. (END)
• 3. If learning returns false (with counterexample
  c), perform extended counterexample analysis on
  c.
• If c is real, return false and go to 4. (END)
• If c is spurious, add more actions from ?A to S
  and go to 2.
• 4. END

20
extended counterexample analysis
query
?A (?M1 ? ?P) ? ?M2 S ? ?A is the current
alphabet
?s? M1 ?P?
L
c ??A
false
conjecture Ai
?Ai? M1 ?P?
?true? M2 ?Ai?
P holds
false cex c
false
false cex t
true
?c?S? M1 ?P?
P violated
?c??A? M1 ?P?
c ??A
true
Refiner compare c??A and t??A
Add actions to S and restart learning
21
alphabet refinement
Ordererr
Output
Input
in
out
send
out
out
in
ack
?A send, out, ack
? out
c?S ?out?
false with c ?send, out?
?true? Output ?Ai?
c??A ?send, out?
?c?S? Input ?P?
false with counterex. t ?out ?
?c??A? Input ?P?
true
compare ?out? with ?send, out?
add send to ?
22
characteristics

• Initialization of S
• Empty set or property alphabet ?P ? ?A
• Refiner
• Compares t??A and c??A
• Heuristics
• AllDiff adds all actions in the symmetric
  difference of the trace alphabets
• Forward scans traces in parallel forward adding
  first action that differs
• Backward symmetric to previous
• Termination
• Refinement produces at least one new action and
  the interface is finite
• Generalization to n components
• Through recursive invocation
• See also learning with optimal alphabet
  refinement
• Developed independently by Chaki Strichman 07

23
implementation experiments

• Implementation in the LTSA tool
• Learning using rules ASYM, SYM and CIRC
• Supports reasoning about two and n components
• Alphabet refinement for all the rules
• Experiments
• Compare effectiveness of different rules
• Measure effect of alphabet refinement
• Measure scalability as compared to
  non-compositional verification
• Extensions for
• SPIN
• JavaPathFinder
• http//javapathfinder.sourceforge.net

24
case studies
K9 Rover

• Model of Ames K9 Rover Executive
• Executes flexible plans for autonomy
• Consists of main Executive thread and
  ExecCondChecker thread for monitoring state
  conditions
• Checked for specific shared variable if the
  Executive reads its value, the ExecCondChecker
  should not read it before the Executive clears it
• Model of JPL MER Resource Arbiter
• Local management of resource contention between
  resource consumers (e.g. science instruments,
  communication systems)
• Consists of k user threads and one server thread
  (arbiter)
• Checked mutual exclusion between resources

MER Rover
25
results

• Rule ASYM more effective than rules SYM and CIRC
• Recursive version of ASYM the most effective
• When reasoning about more than two components
• Alphabet refinement improves learning based
  assume guarantee verification significantly
• Backward refinement slightly better than other
  refinement heuristics
• Learning based assume guarantee reasoning
• Can incur significant time penalties
• Not always better than non-compositional
  (monolithic) verification
• Sometimes, significantly better in terms of memory

26
analysis data
ASYM
ASYM refinement
Monolithic
A assumption size Mem memory (MB) Time
(seconds) -- reached time (30min) or memory
limit (1GB)
27
design/code level analysis
M1
M2
P
A
Design
C1
C2
A
P
Code

• Does M1 M2 satisfy P? Model check build
  assumption A
• Does C1 C2 satisfy P? Model check use
  assumption A
• ICSE2004 good results but may not scale
• TEST!

28
compositional verification for C

• Check composition of software C components
• C1C2 P

C1
C2
refine
refine
predicate abstraction
predicate abstraction
M1
M2
true
learning framework
C1C2 P
false
counterexample analysis
spurious
spurious
C1C2P
29
compositional verification for C (details)
weaken A1
weaken A2
learning
learning
strengthen A1
strengthen A2
A2
A1
?A1? M1 ?P?
?A2? M2 ?P?
false
false
true
true
M1
M2
?true? M2 ?A1? ?true? M1 ?A2?
M1
M2
predicate abstraction
predicate abstraction
OR
false
strengthen M1
strengthen M2
counterexample analysis
C1C2P
30
end of part 1I
please ask LOTS of questions!