Compositional Verification of Evolving Component-based Systems

1
Compositional Verification of Evolving
Component-based Systems

• Nishant Sinha
• ECE, CMU
• PhD Proposal Presentation

2
Overview

• Problem of Re-validating Evolving Systems
• Our Solution Substitutability Check
• Containment
• Compatibility
• Assume-Guarantee for Simulation
• Related Approaches
• Proposed Research
• Timeline

3
Evolving Systems/Designs

• Evolution is inevitable in any real
  hardware/software system
• Changing requirements, designs
• Bug fixes
• Product changes (e.g., underlying platform)
• Upgrades of vendor-provided components

4
Re-validation on Evolution
Assembly A
?
Component C (1.0)
Component C (2.0)
5
Re-validating Evolving Systems

• Component assembly re-validation
• Necessary on upgrade of any component
• High costs of complete global verification
• For example, a software system of 25 components
  (20K LOC) and 100 properties might take up to
  a month of verification effort
• Require efficient formal techniques for
  re-validation upon evolution

6
Re-validation on Evolution

• Pervasive problem in Software and Hardware
  Engineering communities
• Evolution
• Upgrades, Bug Fixes, Re-designs ..
• Re-validation appears in numerous forms
• Regression checks, Upgrade compatibility checks,
  Re-design validation
• Our proposed formulation/solution
• Substitutability Check

7
Subst. Check - Requirements

• Substitutability Check for Upgrades
• Formal technique for statically re-validating
  upgrades
• Local and Incremental nature
• Should avoid complete re-validation
• Local focus on modified components
• Incremental reuse previous validation results
• A make for component-based systems

8
Subst. Check - Our Solution

• Our Substitutability Check procedure
• Focus on Static Re-validation of upgrades
• Local and Incremental
• Supports changes in Component Interfaces and
  Behaviors
• Supports multiple component upgrades

9
Component Upgrade

• Two-phase Subst. Check
• Local Containment
• Global Compatibility

• Involves both addition and removal of behaviors

10
Substitutability Check
11
Substitutability Check
FM05

• Containment
• Detects if any useful behaviors are lost
• Locally checks if behaviors (services) of the
  previous component are still present in new one
• C ? C
• Compatibility
• Safety with respect to other components in
  assembly
• Detects if the added behaviors cause any
  assembly-level violations
• Does C1 C2 .. Cn ? P hold?

FM05 Dynamic Component Substitutability
Analysis, S. Chaki, E. Clarke, N. Sharygina, N.
Sinha, Formal Methods Conference, 2005.
12
Subst. Check for Software Components

• We have formalized Substitutability Check in
  context of Component-based Software Systems
  FM05
• Components are communicating C programs
• Obtain a finite behavioral model of all
  components by abstraction State-event
  communicating automata
• Containment
• Use both Under- and Over- approximations of
  components
• Compatibility
• Use Dynamic Assume-Guarantee Reasoning
• Focus of proposed research

13
State-event (SE) automata
p

• Both state and actions labeled
• ltQ,?,T,P,Lgt
• Composition semantics
• Synchronize on shared actions
• Represents abstractions

?
?
q
?
!q
!p
C Component
Component SE Automaton abstraction
Predicate Abstraction
14
Component Assembly

• A set of communicating concurrent C programs
• No recursion, procedures inlined
• Communication is abstracted into interface
  actions

Component Assembly
Predicate Abstraction
Abstract Assembly
15
Component Assembly Example

• Inter-process communication module

Lock server
Queue
Enter_crit_section, Leave_crit_section
ReadMQ
WriteMQ
Write_to_queue
Read_from_queue
16
Containment Check
17
Containment Check

• Goal Check C ? C
• All behaviors retained after upgrade
• Cannot check directly need approximations
• Idea Use both under- and over- approximations
• Solution
• Compute M C ? M
• Compute M M ? C
• Check for M ? M

C
C
18
Containment (contd.)
C
C
over-approx
under-approx
M
M
M ? M ?
True
False, CE
C ? C
False, Refine M
CE ? C ?
True, Refine M
! ( C ? C), CE provided as feedback
True
CE ? C ?
False
19
Containment (contd.)

• Computing over- (may) approximation
• Conventional predicate abstraction
• Amay(a,a)
• ?s, ?s. R(s,s) ? a h(s) ? a h(s)
• Computing under- (must) approximation
• Modified predicate abstraction
• Universal abstraction
• Amust(a,a)
• ? s, ? s. R(s,s) ? a h(s) ? a h(s)

20
Compatibility Check
21
Compatibility Check

• Goal
• Detect system-level spec violations due to added
  behavior
• Approach Dynamic Compositional Reasoning
• Use compositional reasoning to obtain
  verification proofs for the initial assembly
• Identify parts of the proof that need to be
  re-validated on upgrades
• Re-validate those parts while reusing as much of
  the previous proof as possible

22
Compositional Reasoning

• Assume-Guarantee Reasoning (AGR)

M1 A ? P M2 ? A M1
M2 ? P
AG - Non Circular

• Automatically generate assumption A
• Cobleigh et. al. at NASA Ames
• Use learning algorithm for regular languages, L

23
Learning Regular languages L

• Proposed by D. Angluin, improved by Rivest et al.
  
• Learning regular sets from queries and
  counterexamples, Information and Computation,
  75(2), 1987.
• Learns the minimal DFA corresponding to an
  unknown regular lang.

Minimally adequate Teacher
L learner
IsMember( trace ? )
IsCandidate( DFA D )
Modelchecker
Minimum DFA

• Polynomial in the number of states and length of
  max counterexample

24
L Data Structures
? a ab

• Observation table empty initially
• Rows have access traces
• Columns have distinguishing traces
• Table entries show membership of traces in U
• Iteratively updates the table using provided
  counterexample samples
• Finally, builds a DFA from a closed table and
  presents it as a candidate

q0 q1 q1 q1 q0
? a b aa ab

• 1 0
• 1 0
• 1 1 0
• 1 1 0
• 0 1 0

25
Automated AG Reasoning using L
Cobleigh et al.
-CE for A
Teacher
R1 M1 A ? P
L Assumption Generation
A
true
true
R2 M2 ? A
M1 M2 ? P
CE
Actual CE M1 M2 ? P
CE Analysis
CE for A
26
Handling Multiple Components
R1 M1 A ? P R2 M2 ? A
M1 M2 ? P

• AG-NC can be applied recursively
• (Cobleigh et al.)

M1 M2 M3 ? P

• AG Proof Tree
• Each Ai computed by a separate L instantiation

M2 M3 ? A1
M1 A1 ? P
M2 A2 ? A1
M3 ? A2
27
Compatibility of Upgrades
C
C
Identical
New
Lost

• Suppose assumptions are available from the old
  assembly
• Re-validate an upgrade
• We propose Dynamic AGR Incremental AGR
  technique
• Reuse previous verification results

Upgrade
M1 M2 ? P
M1 A1 ? P
M2 ? A1
Reuse?
Violates L invariant

• Can we reuse previous assumptions directly?
• NO upgrades may change the unknown U to be
  learned
• Therefore, Dynamic AGR needs Dynamic L

28
Dynamic L

• Learn assumption A corresponding to U
• Corresponding Observation Table T
• Unknown language U changes to U
• Goal Continue learning from previous model A
• Start learning U by reusing table T
• Central Idea Re-validate T against U to obtain
  T
• T is valid with respect to U

29
Dynamic L
Table valid for U
Table valid for U
U U\b Re-validation against U
Assumption A
Assumption A
30
Dynamic AG
Upgrade
M1 M2 ? P
M1 M2 ? P
M1 A1 ? P
M2 ? A1
M1 A1 ? P
M2 ? A1
(Dynamic L) Re-Validate! and Reuse

• Multiple Components
• Heuristics for choosing order of re-validating
  Ais in proof tree

31
Implementation

• ComFoRT framework for verifying concurrent C
  programs
• Industrial benchmark
• ABB Inter-process Communication (IPC) software
• 4 main components CriticalSection, IPCQueue,
  ReadMQ, WriteMQ
• Evaluated on single and simultaneous upgrades
• WriteMQ and IPCQueue components
• Properties
• P1 Write after obtaining CS lock
• P2 Correct protocol to write to IPCQueue

32
Experimental Results
Upgrade (Property) Torig (msec) (AGR from scratch) Tug (msec) (Dynamic AGR)
Ipc1 (P1) 2260 13
Ipc1 (P2) 1694 14
Ipc2 (P1) 3286 17
Ipc2 (P2) 805 10
Ipc3 (P1) 3624 17
Ipc3 (P2) 1649 14
Ipc4 (P1) 1102 24
33
Summary Till Now

• Automated Substitutability Checking
• Containment and Compatibility
• Containment is a local check
• Compatibility is a global check and reuses
  previous verification results
• Handles multiple upgrades
• Implementation
• ComFoRT framework
• Compatibility Check applied to IPC example

34
AGR for Simulation Conformance
CAV05

• Till now during AGR
• Notion of conformance was trace containment
• Simulation conformance local, non-det.
• M1 M2 P ? M1 A P M2 A
• Counterexamples are trees
• We proposed automated AGR for simulation
  conformance of LTSs
• Learn from tree CEs instead of trace CEs
• Tree language of an LTS set of trees that it
  simulates

CAV05 Automated Assume-Guarantee Reasoning for
Simulation Conformance, S. Chaki, E. Clarke, N.
Sinha, P. Thati, Conference on Computer-Aided
Verification, 2005.
35
AGR for Simulation Conformance

• Key contributions
• Formulate simulation check as regular tree
  language containment M N ? T(M) ? T(N)
• Represent tree languages with det. tree automata
• Learning algorithm LT for an unknown tree lang.
  using teacher
• For AGR Teacher uses tree language containment
  to answer queries
• Experiments with C implementations of Secure
  Socket Layer (SSL) Protocol client and server

36
Results
Direct Direct AG AG Gain
Verif. Result T1 Mem1 T2 Mem2 Mem1/Mem2 A MQ CQ
Invalid 2146 325 207 10.4 8 265 3
Valid 2080 309 163 12.8 8 279 3
Valid 2077 309 163 12.7 16 279 3
Valid 2076 976 167 12.4 16 770 4
Valid 2075 969 167 12.4 16 767 4
Invalid 2074 3009 234 8.9 24 1514 5
Invalid 2075 3059 234 8.9 24 1514 5
Invalid 2072 3048 234 8.9 24 1514 5
37
Related Work
38
Related Work

• Behavioral Subtyping Liskov ,Wing 01
• Extend OO subtyping to behaviors
• Subtype Req If ?(x) is provable about object
  xT, then ?(y) must hold for yS, where S is a
  subtype of T
• Add invariant and history constraints to object
  specifications
• Local substitution check for an object,
  independent of its use by the environment

39
Related Work

• Interface automata formalism Henzinger et al.
  01-03
• Variant of Input/Output automata used to
  represent component abstractions and interfaces
• Composition and Refinement theory
• Composition is well-defined only if under some
  environment two automata dont deadlock
• M refines N if M allows more inputs and exhibits
  less outputs than N
• Refinement is used as a notion of Substitution
  does not capture environment use of a component

40
Related Work

• Checking compatibility of Upgrades McCamant,
  Ernst04
• Compute observational summaries of components
  before upgrades using sample execution behaviors
• Internal and External summaries
• Pre-, Post-conditions to component method calls
• On an upgrade, check if new summaries refine old
  ones using a theorem prover
• Unsound component behavior abstraction
• No temporal information about component behaviors

41
Related Work

• Automated Assume-Guarantee Reasoning (AGR)
• Weakest assumption GPB02
• Learning assumptions automatically CGP03
• Circular Rules for Automated AGR BGP03
• Predicate Abstraction and AGR CCGP04
• Symbolically learning assumptions AMN04

42
Related Work

• Learning for Verification
• Black-box model checking PVY99,GPY02
• Learning for infinite-state model checking
  HV04, VSVA04
• Synthesizing interfaces of Java classes ACMN05
• Behavior-based model construction HS04
• Inductive learning for shape analysis SLR05

43
Proposed Research
44
Proposed Research

• Substitutability Framework for SystemC
• AGR for Liveness
• Improving Substitutability Check
• Learning-assisted CEGAR

45
SystemC

• C library to simulate concurrency
• SystemC constructs are macros in C
• Being adopted for System-level Design by industry
• Modular computation structure
• Has Communication primitives Channels/Events
• Executable specifications
• Supports HW/SW Co-Design
• Goal Develop a Compatibility Check Framework for
  SystemC Designs

46
SystemC Example
47
SystemC Schematic
48
Program Structure

• program lt processes , channels , variables ,
  eventsgt
• process
• ( comm ctrl flow )
• comm
• event.notify() event.notify_delayed()
• signal.read() signal.write()
• channel.lttransaction_namegt()
• port.read() port.write()
• port.lttransaction_namegt()
• ctrl flow
• if (exp) then lts1gt else lts2gt while (exp) lts1gt
• variable types
• bool int bitvector

49
Verifying SystemC Hardware Designs

• Synchronous model of execution
• Based on Discrete-event simulation
• Verification easier than C
• Finite number of concurrent processes
• No dynamic objects
• Available open-source parsers
• PINAPA

50
Plan Subst. Check for SystemC
Flatten and Translate to Synch. Intermediate
Language
SystemC Design
Events, Channels, Shared Vars, Processes
Build Verifier for Intermediate Language
Delta-cycle semantics
Assumption automata over ltInput,Outputgt alphabet
Build Compositional Verifier for Intermediate
Language
Perform Compatibility Check
51
AGR for Liveness

• Till now, safety and simulation conformance
• Checking liveness specs involves reasoning about
  infinite system behaviors
• Consider ?-regular language containment
• M1 M2 ?? P ? M1 A ?? P M2 ?? A
• A corresponds to an ?-regular language
• Weakest assmp. language exists is ?-regular

52
AGR for Liveness

• Develop a learning algorithm for general -regular
  languages
• Previous algorithms target a subclass
• Such languages have non-canonical finite
  representations
• Cannot extend L directly Myhill-Nerode
  congruence is insufficient
• Use a right-congruence theorem for ?-regular
  languages (Maler and Staiger)

53
Improving Substitutability Check

• Till now, only behavioral changes allowed during
  upgrades
• Interface ports of modules remain the same
• Consider changes in alphabet during Dynamic Check
• Extend Dynamic L to handle both unknown language
  change and alphabet change

54
Learning-assisted CEGAR

• Compositional Reasoning for Software Components
  C1 and C2
• C1 C2 ? P

55
Learning-assisted CEGAR

• Compositional Reasoning for Software Components
  C1 and C2
• C1 C2 ? P

M1 M2 ? P
C1 C2
M1 M2
Predicate Abstraction
M1 A ? P M2 ? A
Constrains
Learner
Spurious CE
Provides Multiple Spurious CEs
Builds spurious CE language automata