Automatic Verification of Pointer Programs using Grammar-based Shape Analysis

1
Automatic Verification of Pointer Programs using
Grammar-based Shape Analysis

• Hongseok Yang
• Seoul National University
• (Joint Work with Oukseh Lee and Kwangkeun Yi)

2
Automatic Verification of Pointer Programs

• Inference of program invariants
• crucial for automatic verification.
• Difficulty unboundedly many new heap cells.

h
hnil while () xnew(nil,nil) if
(hnil) hx else x-gtnexth
h-gtprevx hx
nil
h
Need to summarize heap cells.
nil
nil
h
nil
nil
h
nil
nil
h
nil
nil
3
Goal Precise High-level Invariants
h

• Existing technology Shape analysisSaReWi96,99.
• Idea Use a grammar to find a good abstraction of
  each heap object (i.e., cells and their
  pointers).

nil
nil
h
nil
nil
nil
h
nil
p
dlist(p) nil p
dlist(c)
c
dlist(p)
4
Demo

• Binomial heap construction (all pointers to nil
  are omitted.)
• Our immediate goal was to handle the binomial
  heap construction algorithm.

5
Structure of Our Analysis
while(B) C1 C2 C3 C4
Embed
Nk
D
Abstract Execution
D
Nk
Normalize
6
Abstract Domain D Grammar

• D Pf(Graph) x Grammar T
• A grammar R is a set of following rules
• ?(x) nil O
• where V1, V2 2 nil, self, x, ?(_), ?(self),
  ?(nil), ?(x)
• Examples
• tree(x) nil O
• dList(x) nil O

V1
V2
tree(_)
tree(_)
x
dList(self)
7
Abstract Domain D Shape Graph

• D Pf(Graph) x Grammar T
• Shape graph
• Each node is concrete (a), annotated with nil
  (d), or annotated with a nonterminal (c and
  b).
• An element (S,G) in D is called abstract state.

y
Stack
x
Heap
a
ctree(_)
bdList(a)
dnil
8
Normalized Abstract Domain Nk

• Idealized version of normalization
• Group nodes according to heap objects
• Compute the best grammar that describes each
  group
• Ensure that each shape graph doesnt use more
  than k nodes.
• Example
• Nk (µ D) consists of normalized abstract
  states.
• The actual definition of Nk is not algorithmic.

x
x
x
normalize
a
anil
atree(_)
cnil
b
dnil
enil
tree(_)
tree(_) nil O
tree(_)
9
Definition of Analysis

• Analysis of programs without loops
• Forward analysis C D ! D
• Case pruning B D ! D
• while B CA FixAv F tnFn(normalize(A)).
• F Nk ! Nk
• F ?A. normalize(A t B(CA)))

10
Doubly Linked List Construction

• h nil
• while ()
• var x
• x new
• if (h nil)
• h x
• else
• x-gtnext h
• h-gtprev x
• h x

11
Inferred Loop Invariant

• Inferred abstract state (i.e., shape-graph set
  and grammar)

prev
nil
?(x) nil O
h
?(self)
next
prev
x
?(x) nil O
a?(_)
?(self)
next
12
3rd Iteration Step

• Abstract state A2 after the 2rd iteration
• Inferred invariant A
• A normalize(A2 t LoopBody(A2))

prev
nil
?(x) nil O
h
?(self)
next
prev
x
?(x) nil O
a?(_)
nil
next
prev
nil
?(x) nil O
h
?(self)
next
prev
x
?(x) nil O
a?(_)
?(self)
next
13
Computation of A2tLoopBody(A2)
prev
prev
nil
x
?(x) nil O
?(x) nil O
?(self)
nil
next
next
h
h
x
xnew
if(hnil)
prev
next
aa(_)
aa(_)
e
fnil
gnil

1. Unroll ?.
2. Prune cases.
3. Execute.
4. Join results.
5. Collect garbage.

x
h
True Branch
prev
next
anil
e
fnil
gnil
x
h
False Branch
next
next
prev
next
a
c?(a)
e
fnil
gnil
bnil
prev
prev
14
Normalization 1 Identify Heap Objects

• Identify data structures, and express them by
  nonterminals.

h
h
h
prev
next
prev
next
next
a
a
aa(_)
bnil
cnil
bnil
c
d?(c)
prev
prev
nil
?(x) nil O
?(self)
next
prev
x
?(x) nil O
nil
next
prev
nil
?(x)O
nil
next
15
Normalization 1 Identify Heap Objects

• Identify data structures, and express them by
  nonterminals.

h
h
h
prev
next
next
a
aa(_)
bnil
c
d?(c)
a?(_)
prev
prev
prev
nil
x
g(x)O
?(x) nil O
?(self)
?(self)
next
next
prev
x
?(x) nil O
nil
next
prev
nil
?(x)O
nil
next
16
Normalization 1 Identify Heap Objects

• Identify data structures, and express them by
  nonterminals.

h
h
h
prev
next
a
aa(_)
bnil
c?(a)
a?(_)
prev
prev
nil
x
g(x)O
?(x) nil O
?(self)
?(self)
next
next
prev
prev
x
nil
?(x) nil O
p(x)O
nil
g(self)
next
next
prev
nil
?(x)O
nil
next
17
Normalization 1 Identify Heap Objects

• Identify data structures, and express them by
  nonterminals.

h
h
h
aa(_)
a?(_)
a?(_)
prev
prev
nil
x
g(x)O
?(x) nil O
?(self)
?(self)
next
next
prev
prev
x
nil
?(x) nil O
p(x)O
nil
g(self)
next
next
prev
nil
?(x)O
nil
next
18
Normalization 2 Unify Similar Shape Graphs

• Roughly, two shape graphs are similar iff they
  coincide except the use of nonterminals.

h
h
h
h
aa(_)
a?(_)
a?(_)
at(_)
prev
prev
nil
x
g(x)O
?(x) nil O
?(self)
?(self)
next
next
prev
prev
x
nil
?(x) nil O
p(x)O
nil
g(self)
next
next
prev
prev
prev
nil
nil
nil
prev
nil
O
O
t(x) nil O
?(x)O
?(self)
nil
g(self)
nil
next
next
next
next
19
Normalization 3 Collect Garbage

• Eliminate the definitions of unused nonterminals
  from the grammar.

h
at(_)
prev
prev
nil
x
?,?,? are not used
g(x)O
?(x) nil O
?(self)
?(self)
next
next
prev
prev
x
nil
?(x) nil O
p(x)O
nil
g(self)
next
next
prev
prev
prev
nil
nil
nil
prev
nil
O
O
t(x) nil O
?(x)O
?(self)
nil
g(self)
nil
next
next
next
next
20
Normalization 4 Simplify the Grammar

• Regard a(x) and nil as the same.
• Combine same cases and same definitions.

h
at(_)
prev
x
g(x)O
nil
?(self)
next
prev
x
?(x) nil O
Same Cases
nil
next
prev
prev
prev
nil
nil
nil
O
O
t(x) nil O
?(self)
nil
g(self)
next
next
next
21
Normalization 4 Simplify the Grammar

• Regard a(x) and nil as the same.
• Combine same cases and same definitions.

h
at(_)
prev
x
Same Definitions
g(x)O
nil
?(self)
next
prev
prev
x
x
?(x) nil O
O
nil
?(self)
next
next
prev
prev
nil
nil
O
t(x) nil O
?(self)
g(self)
next
next
b(self)
22
Normalization 4 Simplify the Grammar

• Regard a(x) and nil as the same.
• Combine same cases and same definitions.

h
at(_)
Same Cases
prev
prev
x
x
?(x) nil O
O
nil
?(self)
next
next
prev
nil
t(x) nil O
?(self)
next
23
Normalization 4 Simplify the Grammar

• Regard a(x) and nil as the same.
• Combine same cases and same definitions.

h
at(_)
prev
x
?(x) nil
O
?(self)
next
prev
nil
t(x) nil O
?(self)
next
24
Summary

• Execute the loop body abstractly
• LoopBodyA2
• Join the old and new values
• A2 t LoopBodyA2
• Normalize the obtained abstract state
• For each shape graph, identify heap objects and
  express them using nonterminals.
• Unify similar shape graphs.
• Remove the definitions of unused nonterminals.
• Simplify the grammar.

25
Correctness

• The meaning of each abstract state (G,R) is given
  by an assertion trans(G,R) in sep. logic.
• Correctness theorem If C(G,R) (G,R), then
  trans(G,R)Ctrans(G,R) is derivable in sep.
  logic.
• Termination Since the domain Nk is finite, the
  analysis terminates.

26
Conclusion

• Presented an analysis that infers the loop
  invariant of complex pointer programs.
• The key idea is to use a grammar to describe the
  structure of a heap object (i.e., data
  structure).
• Future work
• Develop a systematic reusable framework.
• Handle data structures with more extensive
  sharing.
• dags and trees with linked leaf nodes, etc.
• Prove a property that relates the input and ouput
  states.
• SW recovers link fields to their original values.

27
Inferred Loop Invariant

• Inferred shape-graph set and grammar
• Representation by an assertion

prev
nil
?(x) nil O
h
?(self)
next
prev
x
?(x) nil O
a?(_)
?(self)
next

• letrec
• ?(a,x) (empÆanil) Ç 9b.(a ? nil,b) ?(b,a)
• ?(a,x) (empÆanil) Ç 9b.(a ? x,b) ?(b,a)
• in
• 9a. ha Æ 8x.?(a,x)

28
Abstract Domain D

• D Pf(Graph) x Grammar T
• Shape graph
• Each node can be concrete (a), annotated with
  nil (d), or annotated with a nonterminal (c
  and b).
• Semantics by separation-logic assertions
• 9abcd.(xaÆyb) Æ ((8y.tree(c,y))(a?c,d)(cnilÆe
  mp)dList(b,a))
• Formal definition
• Graph (Var!finSymL) x (SymL!finVal)
• Val nil, lta,bgt, ?(a), ?() a,b2SymL,
  ?2NonTerm

y
Stack
x
Heap
a
ctree(_)
bdList(a)
dnil
29
Grammar

• A grammar R is a set of following rules
• ?(x) nil O
• where V1, V2 2 nil, self, x, ?(_), ?(self),
  ?(nil)
• Examples
• tree(x) nil O dList(x) x O
• Semantics by separation-logic assertions
• tree(c,x) (cnilÆemp) Ç 9lr.(c?l,r)(8y.tree(l,y
  ))(8y.tree(r,y))
• dList(c,p) (cnilÆemp) Ç 9n.(c?p,n)dList(n,c)
• Formal definition
• Grammar NonTerm !finPnf(nil Case x Case)
• Case nil, self, arg, ?(_), ?(arg), ?(self)
  ?2NonTerm

V1
V2
tree(_)
x
tree(_)
dList(self)
30
Normalized Abstract Domain N

• N consists of abstract states (G,R) in D s.t.
• all data structures are expressed by
  nonterminals
• All similar shape graphs and rules are merged.

x
a
cnil
b
dnil
enil
x
y
x
y
?(_)
?(x) nil O ?(x) nil O
?(_)
a
a
nil
nil
cnil
b?(_)
cnil
bb(_)