Efficient Software Model Checking of Data Structure Properties

1
Efficient Software Model Checking of Data
Structure Properties

• Paul T. Darga
• Chandrasekhar Boyapati
• The University of Michigan

2
Software Model Checking

• Exhaustively test code
• On all possible schedules
• On all possible inputs
• Within a bounded finite domain

3
Software Model Checking
State Space Explosion!
4
State Space Reduction

• Many software model checkers
• Verisoft, JPF, CMC, SLAM, Blast, Magic,
• Many state space reduction techniques
• Partial order reduction
• Predicate abstraction
• Effective for control-oriented properties
• Our work focuses on data-oriented properties

5
Our Approach Tree Example

• Our system detects that it suffices to check
• Every operation on every tree path
• Rather than every operation on every tree
• Red-black tree O(n3) paths, O(nn) trees
• Significant speedup to model checking

6
Glass Box Model Checking
back
front
enqueue
dequeue
0,0
0,1
1,0
0,2
2,0
1,1
3,0
2,1
1,2
0,3
4,0
3,1
0,4
2,2
1,3
Glass Box model checker
Traditional (black-box) model checker
O(n2) transitions
O(n) transitions
O(n2) states
O(n) states
7
Outline

• Introduction to glass box model checking
• Challenges
• Experimental results
• Related work
• Future work and conclusion

8
Glass Box Challenges

• State space organization

9
State Space Reachability
State is disconnected, but we still need to check
one of its transitions!

• We cannot use reachability through transitions
  (black-box approach)
• Programmers must provide a class invariant
• State space the set of all type-correct states
  within a specified bound which satisfy the
  invariant

10
Invariants Specification

• Singly-linked list absence of cycles

java.util.TreeMap 1670 lines of code 20 lines of
invariant
class LinkedList static class Node Node
next Object value Node head
boolean repOk() Set visited new
java.util.HashSet() for (Node n head n !
null n n.next) if (!visited.add(n))
return false return true
class LinkedList static class Node tree
Node next Object value tree Node
head boolean repOk() // writing
invariants is easy and fun! return true

11
Glass Box Search Algorithm

• I states satisfying the invariant
• S I transitions
• while (S is not empty)
• t any transition in S
• run t
• verify the post-condition
• T transitions similar to t
• S S T

How do we represent these sets, and perform
operations on them, efficiently?
12
Glass Box Challenges

• State space organization
• Class invariants
• State space representation
• Binary decision diagrams

13
Binary Decision Diagrams
?
root is null
?
left is null
right is null
right is null
?
?
root is red
root is red
root is red
?
?

• Compact representation of exponentially large yet
  structured sets
• Perform set operations directly

left is red
left is red
left is red
?
?
?
right is red
right is red
?
?
?
?
14
BDDs Red-Black Trees
Max Height Set Size BDD Size
1 2 0
2 7 12
3 34 58
4 597 226
5 238526 744
6 42715248230 2367
7 1123387228727905854061 7359
15
Glass Box Challenges

• State space organization
• Class invariants
• State space representation
• Binary decision diagrams
• State space reduction
• Monitoring field access
• Monitoring information flow
• Pruning isomorphic structures
• Ensuring soundness

16
Monitoring Field Access
Object pop() if (head null) return null
Object v head.value head head.next return
v
3
7
4
2
n0
head
n1
n2
n3

• t op pop ? head n0 ? n0.value 3 ?
  n0.next n1 ? n1.value 7 ? n1.next n2 ?
  n2.value 4 ? n2.next n3 ? n3.value 2 ?
  n3.next null

T op pop ? head n0 ? n0.value 3 ?
n0.next n1
T op pop ? head n0 ? n0.next n1
17
Outline

• Introduction to glass box model checking
• Challenges
• Experimental results
• Related work
• Future work and conclusion

18
Performance Stack
Max Size BDD Nodes Transitions Transitions Transitions
Max Size BDD Nodes Glass Box JPF Black Box
1 9 4 33 9
2 14 5 141 32
3 16 5 1033 115
4 18 5 10949 450
8 26 5 - 264420
16 42 5 - -
32 74 5 - -
64 140 5 - -
128 268 5 - -
19
Performance Stack
Max Size BDD Nodes Time (s) Time (s) Time (s)
Max Size BDD Nodes Glass Box JPF Black Box
1 9 0.016 0.533 0.008
2 14 0.016 0.669 0.009
3 16 0.017 1.349 0.012
4 18 0.017 7.233 0.030
8 26 0.017 - 2.242
16 42 0.018 - -
32 74 0.018 - -
64 140 0.020 - -
128 268 0.023 - -
20
Performance Queue
Max Size BDD Nodes Transitions Transitions Transitions
Max Size BDD Nodes Glass Box JPF Black Box
1 17 5 601 27
2 36 7 89756 356
3 46 8 - 6610
4 61 9 - 176430
8 114 13 - -
16 272 21 - -
32 834 37 - -
64 2873 69 - -
128 12292 133 - -
21
Performance Queue
Max Size BDD Nodes Time (s) Time (s) Time (s)
Max Size BDD Nodes Glass Box JPF Black Box
1 17 0.019 1.121 0.011
2 36 0.020 47.598 0.044
3 46 0.020 - 0.161
4 61 0.020 - 2.126
8 114 0.022 - -
16 272 0.028 - -
32 834 0.050 - -
64 2873 0.086 - -
128 12292 0.118 - -
22
Performance Red-Black Tree
Max Height BDD Nodes Transitions Transitions Transitions
Max Height BDD Nodes Glass Box JPF Black Box
1 28 6 49 12
2 493 28 - 936
3 3860 108 - 18143370
4 24400 366 - -
5 128314 1094 - -
6 781369 2968 - -
7 6263228 7524 - -
Only 10 seconds to verify over 270 red-black
trees!
23
Performance Red-Black Tree
Max Height BDD Nodes Time (s) Time (s) Time (s)
Max Height BDD Nodes Glass Box JPF Black Box
1 28 0.022 0.617 0.013
2 493 0.028 - 0.084
3 3860 0.066 - 431.682
4 24400 0.134 - -
5 128314 0.340 - -
6 781369 1.543 - -
7 6263228 10.340 - -
Only 10 seconds to verify over 270 red-black
trees!
24
Performance File System
Max Height BDD Nodes Transitions Transitions Transitions
Max Height BDD Nodes Glass Box JPF Black Box
2 784 62 12901 570
3 1764 135 - 14820
4 3728 240 - 552900
5 6531 380 - -
6 9984 558 - -
7 13467 777 - -
8 21755 1040 - -
9 31031 1350 - -
10 40954 1710 - -
25
Performance File System
Max Height BDD Nodes Time (s) Time (s) Time (s)
Max Height BDD Nodes Glass Box JPF Black Box
2 784 0.035 6.883 0.045
3 1764 0.057 - 0.201
4 3728 0.075 - 6.267
5 6531 0.096 - -
6 9984 0.109 - -
7 13467 0.125 - -
8 21755 0.147 - -
9 31031 0.167 - -
10 40954 0.203 - -
26
Outline

• Introduction to glass box model checking
• Challenges
• Experimental results
• Related work
• Future work and conclusion

27
Related Work

• Software model checkers
• Verisoft Godefroid
• Java PathFinder Visser et al
• CMC Musuvathi, Park, Chou, Engler, Dill
• Bandera Corbett, Dwyer, Hatcliff, Robby, et al
• Bogor Dwyer, Hatcliff, Hoosier, Robby
• SLAM Ball, Majumdar, Millstein, Rajamani
• Blast Henzinger, Jhala, Majumdar
• Magic Chaki, Clarke, Groce, Jha, Veith
• XRT Grieskamp, Tillmann, Shulte
• JCAT DeMartini, Iosif, Sisto

28
Related Work

• State space reduction techniques
• Abstraction refinement SLAM Blast Magic
• Partial order reduction Godefroid Flanagan
• Heap canonicalization Musuvathi, Dill Iosif
• Symmetry reduction Ip, Dill

29
Related Work

• Static analysis tools
• TVLA Sagiv, Reps, Wilhelm
• PALE Moeller, Schwartzbach
• Formal verification using theorem provers
• ESC/Java Nelson et al
• ACL2 Kaufmann, Moore, et al

30
Outline

• Introduction to glass box model checking
• Challenges
• Experimental results
• Related work
• Future work and conclusion

31
Future Work

• Data structures are just the beginning
• Applicable to any system where we can
• Describe the state space using invariants
• Transitions depend on a small part of the state
• Can significantly speedup model checking