Model Checking: From Hardware to Software and Back Again

1
Model CheckingFrom Hardware to Softwareand
Back Again

• Edmund Clarke
• Computer Science Department
• Carnegie Mellon University
• (Joint work with H. Jain, D. Kroening, and N.
  Sharygina)

2
Motivation

• Hardware design is typically done at a higher
  level than in the past.
• Need to reason about hardware languages like
  Verilog, VHDL, SystemVerilog, SpecC, SystemC,
  which are close to software.
• Verification tools must reason about
• Programming languages constructs
• Bit-vector semantics (concatenation, extraction)
• Concurrency, Objects, Templates
• Fortunately, no recursion, no pointers, and no
  heap !!

3
Temporal Logic Model Checking

• Model checking is an automatic verification
  technique for finite state concurrent systems.
• Developed independently by Clarke and Emerson and
  by Queille and Sifakis in early 1980s.
• Specifications are written in propositional
  temporal logic.
• Verification procedure is an exhaustive search of
  the state space of the design.

4
Advantages of Model Checking

• No proofs!!!
• Fast (compared to other rigorous methods such as
  theorem proving)
• Diagnostic counterexamples
• No problem with partial specifications
• Logics can easily express many concurrency
  properties

5
Main Disadvantage

• State Explosion Problem
• Too many processes
• Complex data structures

6
LTL - Linear Time Logic

• Determines Patterns on Infinite Traces
• Atomic Propositions
• Boolean Operations
• Temporal operators
• a a is true nowX a a is true in the
  neXt stateFa a will be true in the
  FutureGa a will be Globally true in the
  futurea U b a will hold true Until b
  becomes true

a
7
LTL - Linear Time Logic

• Determines Patterns on Infinite Traces
• Atomic Propositions
• Boolean Operations
• Temporal operators
• a a is true nowX a a is true in the
  neXt stateFa a will be true in the
  FutureGa a will be Globally true in the
  futurea U b a will hold true Until b
  becomes true

a
8
LTL - Linear Time Logic

• Determines Patterns on Infinite Traces
• Atomic Propositions
• Boolean Operations
• Temporal operators
• a a is true nowX a a is true in the
  neXt stateFa a will be true in the
  FutureGa a will be Globally true in the
  futurea U b a will hold true Until b
  becomes true

a
9
LTL - Linear Time Logic

• Determines Patterns on Infinite Traces
• Atomic Propositions
• Boolean Operations
• Temporal operators
• a a is true nowX a a is true in the
  neXt stateFa a will be true in the
  FutureGa a will be Globally true in the
  futurea U b a will hold true Until b
  becomes true

a
a
a
a
a
10
LTL - Linear Time Logic

• Determines Patterns on Infinite Traces
• Atomic Propositions
• Boolean Operations
• Temporal operators
• a a is true nowX a a is true in the
  neXt stateFa a will be true in the
  FutureGa a will be Globally true in the
  futurea U b a will hold true Until b
  becomes true

a
a
a
a
b
11
Branching Time
12
CTL Computation Tree Logic
EF g g will possibly become true
13
CTL Computation Tree Logic
AF g g will necessarily become true
14
CTL Computation Tree Logic
AG g g is an invariant
15
CTL Computation Tree Logic
EG g g is a potential invariant
16
CTL Computation Tree Logic

• CTL uses the temporal operators
• AX, AG, AF, AU
• EX, EG, EF, EU
• CTL allows complex nestings such as
• AXX, AGX, EXF, ...
• CTL linear model checking algorithm !

17
Model Checking Problem

• Let M be a state-transition graph.
• Let be the specification in temporal logic.
• Find all states s of M such that M, s ???.

18
Model of computation
Microwave Oven Example
State-transition graph describes system
evolving over time.
Start Close Heat Error
Start Close Heat Error
Start Close Heat Error
Start Close Heat Error
Start Close Heat Error
Start Close Heat Error
Start Close Heat Error
19
Temporal Logic and Model Checking

• The oven doesnt heat up until the door is
  closed.
• Not heat_up holds until door_closed
• ( heat_up) U door_closed

20
Model Checking
Hardware Description (VERILOG, VHDL, SMV)
Informal Specification
Transition System (Automaton, Kripke structure)
Temporal Logic Formula (CTL, LTL, etc.)
21
Hardware Example IEEE Futurebus

• In 1992 Clarke and his students at CMU used SMV
  to verify the IEEE Future cache coherence
  protocol.
• They found a number of previously undetected
  errors in the design of the protocol.
• This was the first time that formal methods have
  been used to find errors in an IEEE standard.
• Although the development of the protocol began in
  1988, all previous attempts to validate it were
  based entirely on informal techniques.

22
From Hardware to Software

• Natural Question Is it possible to model
  check software?
• According to Wired News on Nov 10, 2005
• When Bill Gates announced that the technology
  was under development at the 2002 Windows
  Engineering Conference, he called it the holy
  grail of computer science

23
Grand ChallengeModel Check Software !

• What makes Software Model Checking different ?

24
What Makes Software Model Checking Different ?

• Large/unbounded base types int, float, string
• User-defined types/classes
• Pointers/aliasing unbounded s of
  heap-allocated cells
• Procedure calls/recursion/calls through
  pointers/dynamic method lookup/overloading
• Concurrency unbounded s of threads

25
What Makes Software Model Checking Different ?

• Templates/generics/include files
• Interrupts/exceptions/callbacks
• Use of secondary storage files, databases
• Absent source code for libraries, system calls,
  mobile code
• Esoteric features continuations, self-modifying
  code
• Size (e.g., MS Word 1.4 MLOC)

26
What Does It Mean to Model Check Software?

• Combine static analysis and model checking
• Use static analysis to extract a model K
  from a boolean abstraction of the program.
• Then check that f is true in K (K ² f),
  where f is the specification of the program.
• ² SLAM (Microsoft)
• ² Bandera (Kansas State)
• ² MAGIC (CMU)

27
What Does It Mean to Model Check Software?

• Simulate program along all paths in computation
  tree
• ² Java PathFinder (NASA Ames)
• ² Source code backtracking (e.g., Verisoft)
• ² Source code symbolic execution
  backtracking
• (e.g., MS/Intrinsa Prefix)

• Use finite-state machine to look for patterns in
  control-flow graph Engler

28
What Does It Mean to Model Check Software?

• Design with Finite-State Software Models
• Finite state software models can act as
  missing link
• between transition graphs and complex
  software.
• ² Statecharts
• ² Esterel

29
What Does It Mean to Model Check Software?

• Use Bounded Model Checking and SAT Kroening
• ² Problem How to compute set of reachable
  states?
• Fixpoint computation is too expensive.
• ² Restrict search to states that are reachable
  from initial
• state within fixed number n of
  transitions
• ² Implemented by unwinding program and using
• SAT solver

30
Key techniques for Software Model Checking

• Counterexample Guided Abstraction Refinement
• (Kurshan, Yuan Lu, Clarke et al JACM, Ball et
  al.)
• - Uses counterexamples to refine
  abstraction
• Predicate Abstraction
• (Graf and Saidi, Ball et al, Chaki et al,
  Kroening)
• - Keeps track of certain predicates on data
• - Captures relationship between variables

31
Counterexamples
Informal Specification
Program
Transition System
Temporal Logic Formula (CTL, LTL, etc.)
32
Counterexamples
Informal Specification
Program
Transition System
Temporal Logic Formula (CTL, LTL, etc.)
Safety Property bad state unreachable
Counterexample
33
Counterexamples
Informal Specification
Program
Transition System
Temporal Logic Formula (CTL, LTL, etc.)
Safety Property bad state unreachable
Counterexample
34
Existential Abstraction
Given an abstraction function ? S ? S?, the
concrete states are grouped and mapped into
abstract states
M?
Preservation Theorem ?
M
35
Simulation Relation
Assume N simulates M (M ? N) Let f be a
universal CTL formula (ACTL)

M
N
a
a
b
b
b
c
d
c
d
N f
36
Preservation Theorem

• Atomic formula f respects ? if f does not
  distinguish concrete states within abstract
  state.
• Theorem (Clarke, Grumberg, Long) If ? is an
  ACTL specification where the atomic formulas
  respect ?, then M ? M?.
• Corollary Preservation Theorem applicable
• M? ? implies M ?.
• Converse implication is not valid !

37
Spurious Behavior
AGAF red Every path necessarily leads back to
red.
Spurious Counterexample ltgogtltgogtltgogtltgogt ...
Artifact of the abstraction !
38
How to define Abstraction Functions?

• Abstraction too fine? State Explosion
• Abstraction too coarse? Information Loss
• Automatic Abstraction Methodology

39
Automatic Abstraction
Spurious
Spurious counterexample
Validation or Counterexample
Correct !
M
Original Model
40
CEGAR CounterExample-Guided Abstraction
Refinement
CProgram
Abstract model
Simulator
41
Software Example Device Driver Code

• Also according to Wired News
• Microsoft has developed a tool called Static
  Device Verifier or SDV, that uses Model
  Checking to analyze the source code for Windows
  drivers and see if the code that the programmer
  wrote matches a mathematical model of what a
  Windows device driver should do. If the driver
  doesnt match the model, the SDV warns that the
  driver might contain a bug.

42
Back to Hardware!
Formal verification support
Ease of design increases
43
Register Level Verilog module
counter_cell(clk, carry_in,
carry_out) input clk input
carry_in output carry_out reg value assign
carry_out value carry_in initial value
0 always _at_(posedge clk) begin // value (value
carry_in) 2 case(value)
0 value carry_in 1 if
(carry_in 0) value
1 else value 0
endcase end endmodule
Gate Level (netlist) .model counter_cell .inputs
carry_in .outputs carry_out .names value
carry_in _n2 .def 0 1 1 1 .names _n2
carry_outraw_n1 - _n2 .names valueraw_n3 0 .nam
es _n6 0 .names value _n6 _n7 .def 0 0 1 1 1 0
1 .r valueraw_n3 value 0 0 1 1 .. (120 lines)
44
Lack of verification support
use techniques from software verification
Must be automatic and scalable!!
45
This work
Model check
?
46
Abstraction-Refinement loop (CEGAR)
Initial Abstraction
Verification
No erroror bug found
VerilogProgram
ModelChecker
Abstract model
Property holds
Counterexample
Refinement
Simulator
Simulation sucessful
Abstraction refinement
Bug found
Spurious counterexample
47
Our approach

• Apply predicate abstraction at RTL Level
• Allows abstraction using word-level predicates
• Example x lt y z, x z,z
• Use a SAT solver for computing abstraction
• Semantics of bit-wise operators taken into
  account
• Obtaining suitable word level predicates
• Syntactic weakest pre-conditions of Verilog
  statements (Kurshan and Namjoshi)

48
Related work

• SAT-Based Predicate Abstraction Wang et al.
• Works at netlist level
• Refinement introduces bit-level predicates
• Vapor tool Andraus et al.
• Works on RTL level designs
• Abstraction to CLU models (equality of terms,
  uninterpreted functions, predicates)
• Lots of other related work

49
An example
module main (clk) input clk reg 100 x,
y initial x 100, y 200 always _at_ (posedge
clk) begin x lt y y lt
x end endmodule
Property AG (x 100 Ç x 200)
Verilog program
50
CEGAR loop
Initial Abstraction
Verification
No erroror bug found
VerilogProgram
ModelChecker
Abstract model
Property holds
Counterexample
Refinement
Simulator
Simulation sucessful
Abstraction refinement
Bug found
Spurious counterexample
51
Predicate Abstraction
module main (clk) input clk reg 100 x,
y initial x 100, y 200 always _at_ (posedge
clk) begin x lt y y lt
x end endmodule
Property AG (x 100 Ç x 200)
Initial set of predicates x 100, x 200
Word Level
Transition relation x y Æ y x
Verilog program
52
Computing Most Precise Abstraction
Next state
Current state
Transition Relation

ltx 100, x 200gt

ltx 100, x 200gt
x y y x
53
Obtain transitions

Computing abstract transitions
ltp1,p2gt
1,0
0,0
0,1
and so on
1,1
54
Abstract Model
module main (clk) input clk reg 100 x,
y initial x 100, y 200 always _at_ (posedge
clk) begin x lt y y lt
x end endmodule
Property AG (x 100 or x 200) Initial set
of predicates x 100, x 200
Failure state
Initial state
Verilog program
55
CEGAR loop
Initial Abstraction
Verification
No erroror bug found
VerilogProgram
ModelChecker
Abstract model
Property holds
Counterexample
Refinement
Simulator
Simulation sucessful
Abstraction refinement
Bug found
Spurious counterexample
56
Model checking
module main (clk) input clk reg 100 x,
y initial x 100, y 200 always _at_ (posedge
clk) begin x lt y y lt
x end endmodule
Abstract Model
Initial state
Failure state
1,0
0,0
0,1
1,1
Verilog program
57
Model checking
module main (clk) input clk reg 100 x,
y initial x 100, y 200 always _at_ (posedge
clk) begin x lt y y lt
x end endmodule
Abstract Model
Abstract counterexample
Initial state
Failure state
1,0
0,0
0,1
1,1
Verilog program
58
CEGAR loop
Initial Abstraction
Verification
No erroror bug found
VerilogProgram
ModelChecker
Abstract model
Property holds
Counterexample
Refinement
Simulator
Simulation sucessful
Abstraction refinement
Bug found
Spurious counterexample
59
Simulation
module main (clk) input clk reg 100 x,
y initial x 100, y 200 always _at_ (posedge
clk) begin x lt y y lt
x end endmodule
Abstract counterexample
Initial state
Failure state
1,0
0,0
Verilog program
Counterexample is spurious
60
CEGAR loop
Initial Abstraction
Verification
No erroror bug found
VerilogProgram
ModelChecker
Abstract model
Property holds
Counterexample
Refinement
Simulator
Simulation sucessful
Abstraction refinement
Bug found
Spurious counterexample
61
Refinement (Kurshan and Namjoshi)

• Let length of spurious counterexample be k
• Take weakest pre-condition of property for k
  steps with respect to transition function

62
Refinement
spurious counterexample
Property
New predicates y 100, y 200
AG (x 100 Ç x 200)
length 1

63
Abstract again
module main (clk) input clk reg 100 x,
y initial x 100, y 200 always _at_ (posedge
clk) begin x lt y y lt
x end endmodule
Property AG (x 100 or x 200) Updated set
of predicates x 100, x 200, y100, y200
New abstraction
Initial state
1,0,0,1
0,1,1,0
Model check
Verilog program
64
Model checking
module main (clk) input clk reg 100 x,
y initial x 100, y 200 always _at_ (posedge
clk) begin x lt y y lt
x end endmodule
Property AG (x 100 or x 200) Updated set
of predicates x 100, x 200, y100, y200
New abstraction
Initial state
1,0,0,1
0,1,1,0
Verilog program
65
Result
module main (clk) input clk reg 100 x,
y initial x 100, y 200 always _at_ (posedge
clk) begin x lt y y lt
x end endmodule
Property AG (x 100 or x 200)
Property holds!
Verilog program
66
Making it work in practice

• Predicate Abstraction Computation
• Handling large number of predicates
• Refinement
• Sufficient predicates but inexact abstraction
  (due to over approximation of most precise
  abstraction)
• Insufficient predicates

67
Predicate Clustering
Next state predicates
Current state predicates
Transition relation
Æ
68
Predicate Clustering

• Speeds up abstraction computation
• Introduces over-approximation
• Refinement
• Handles over-approximation due to predicate
  clustering
• Generate new predicates

69
Refinement by generating new predicates
Identify predicates whose weakest pre-condition
needs to be computed
Init x1, y2
(x1) Æ (x2)
PROOF OF UNSATISFIABILITY
x y y x
Predicate whose Weakest pre-condition is needed
(x1) Æ(x2)
70
Refinement by generating new predicates
Blowup in weakest pre-condition size
Only add atomic predicates
((x lt 5) ? (x 2) x ) 2
Add x lt5 as a new predicate and run the loop
again
x (x lt 5) ? (x 2) x
x 2
71
A counterexample of same length
But now we get a value for x lt 5 from abstract
counterexample (say true)
New predicate
((x lt 5) ? (x 2) x ) 2
x 2 2
x x 2
x (x lt 5) ? (x 2) x
simplify
x 2
72
Experimental results
73
Experimental results (VIS benchmarks)
74
Summary

• Verification at Register level without going to
  netlists
• Predicate abstraction using word-level predicates
• Handling large no. of predicates (predicate
  clustering)
• Weakest pre-conditions for obtaining new
  predicates
• Techniques are completely automatic
• VCEGAR
• Encouraging results on industrial benchmarks
• http//www.cs.cmu.edu/modelcheck/vcegar

75
Questions?