Shape Analysis by Graph Decomposition

1
Shape Analysisby Graph Decomposition
R. Manevich M. Sagiv Tel Aviv University
G. Ramalingam MSR India
J. Berdine B. Cook MSR Cambridge
2
Motivation

• Challenge precise and efficient shape analyses
• Prove properties of dynamically allocated linked
  data structures
• Observation often many correlations irrelevant
  for proving shape properties
• Our approach develop a flexible abstraction that
  takes advantage of this

3
Example program 2 lists
// _at_assume h1!null h1t1 h1.nnull //
h2!null h2t2 h2.nnull////
_at_loop_invariant Reach(h1,t1) //
Reach(h2,t2) //
DisjointLists(h1,h2)EnqueueEvents() L1 while
(...) List temp new
List(getEvent()) if (nondet())
t1.n temp t1 temp
else t2.n temp t2
temp
4
Abstract states - full heaps VMCAI05
h1
t1
h1
t1
h1
t1
1
1
h2
t2
h2
t2
h2
t2
h1
t1
h1
t1
h1
t1
1
1
h2
t2
h2
t2
h2
t2
1
1
1
h1
t1
h1
t1
h1
t1
1
1
h2
t2
h2
t2
h2
t2
1
1
1
5
Graph decomposition
h1
t1
1
h2
t2
1
6
Graph decomposition
Connected components by undirected reachability
7
Abstract states decomposed heaps
h1
t1
h1
t1
h1
t1
1
1
h2
t2
h2
t2
h2
t2
1
1
Coarser abstraction precise enough to prove
invariantbut generates fewer states
For k listsfull heap abstraction generates
3k abstract statesdecomposed heap
abstraction generates 3k abstract states
8
Overall view
Concrete domainconcrete heaps
Full heaps domainshape graphs
Decomposed heaps domainshape subgraphs
?FH
?GD
h1
t1
h1
t1
h1
t1
1
1
...
h2
t2
h2
t2
h2
t2
1
...
1
h1
t1
h1
t1
h1
t1
1
1
h2
t2
h2
t2
h2
t2
1
1
?FH
?GD
Shape subgraphs trackSOME correlations
Shape graphs trackALL correlations
9
Main results

• New abstraction for shape analysis reduces
  exponential factors by
• Connected component decomposition
• Abstracting away null-value correlations
• Sound and sufficiently precise transformers
• Most precise transformers are FNP-complete
• Polynomial time efficient transformers
• Sufficiently precise
• Implementation and empirical results
• Sufficiently precise on set of benchmarks,includi
  ng Windows device driver models
• State space/time reduced by factor of 33/212

10
Outline

• Full heap abstraction VMCAI05
• Reference abstraction
• Further abstraction by decomposition
• Connected component decomposition
• Abstracting away null-value correlations(details
  in paper)
• Abstract transformers
• Concretization by composition
• Experimental results

11
Full heap abstraction VMCAI05
Concrete domainconcrete heaps
Full heaps domainshape graphs
Decomposed heaps domainshape subgraphs
?FH
?GD
h1
t1
h1
t1
h1
t1
1
1
...
h2
t2
h2
t2
h2
t2
1
...
1
h1
t1
h1
t1
h1
t1
1
1
h2
t2
h2
t2
h2
t2
1
1
?FH
?GD
12
Full heap abstraction VMCAI05

• Abstraction for singly-linked lists
• Basic concepts
• Interruptions (bounded number of)
• Uninterrupted list segments (bounded number of)
• Abstraction keeps interruptions and abstracts
  segment lengths to 1,1
• Result is a shape graph

Concrete heap
x
y
?FH by point-wiseextension
13
Graph decomposition abstraction
Concrete domainconcrete heaps
Full heaps domainshape graphs
Decomposed heaps domainshape subgraphs
?FH
?GD
h1
t1
h1
t1
h1
t1
1
1
...
h2
t2
h2
t2
h2
t2
1
...
1
h1
t1
h1
t1
h1
t1
1
1
h2
t2
h2
t2
h2
t2
1
1
?FH
?GD
14
Graph decomposition abstraction

• Abstraction of shape graphs
• Further abstraction over shape graphs
• Decouples connected components
• Intuitively different components different
  logical data structures
• Result set of shape subgraphs

15
Connected components decomposition
h1
t1
h2
t2
1
h1
t1
1
h2
t2
16
Abstracting null-value correlations

• Actual shape graph representation captures
  null-value correlations(null node not shown in
  other slides)
• Abstraction reduces exponential factor due to
  null-value correlations
• Details in paper

17
Concretization ?GD
Concrete domainconcrete heaps
Full heaps domainshape graphs
Decomposed heaps domainshape subgraphs
?FH
?GD
h1
t1
h1
t1
h1
t1
1
1
...
h2
t2
h2
t2
h2
t2
1
...
1
h1
t1
h1
t1
h1
t1
1
1
h2
t2
h2
t2
h2
t2
1
1
?GD
?FH
18
Abstracting correlations
h1
t1
h2
t2
1
h1
t1
1
h2
t2
19
Abstract transformers

• Need transformers for program statements
• xnew List()
• xnull
• xy
• xy.n
• x.ny
• assume(x!y)
• assume(xy)

20
Abstract transformers outline

• Induced transformers by concretization(from
  subgraphs and shape graphs)
• Problem concretization introduces exponential
  space blow-up
• Most precise transformers by partial
  concretization
• Avoids exponential space blow-up
• Requires oracle to test strong feasibility
• Strong feasibility test NP-complete
• Conservative transformers
• Give up on strong feasibility test
• Avoids exponential time blow-up

21
Most precise transformer CC77
Concrete domainconcrete heaps
Full heaps domainshape graphs
Decomposed heaps domainshape subgraphs
?FH
h1
t1
...
h2
t2
...
?st
h1
t1
h2
t2
?FH
Problem concretization is exponential space in
worst-case
22
Partial concretization

• Compose weakly-feasible subgraphs
• Subgraphs that do not share any variables
• Compose only subgraphs in footprint of statement
• Compose at most any 2 or 3 subgraphs

23
Transformer example
temp
h2
t2
h1
t1
h1
t1
1
?t1.n temp
?t1.n temp
?t1.n temp
?t1.n temp
24
Most precise transformer

• Most precise requires strong feasibility test
• Check that subgraphs can be extended to include
  all variables

M1
M2
M3
M4
M5
x
z
w
x
z
y
w
y
Can we extend to havevariable w?
x
z
y
25
Most precise transformer

• Most precise requires strong feasibility test
• Check that subgraphs can be extended to include
  all variables

M1
M2
M3
M4
M5
x
z
w
x
z
y
w
y
Inconsistency shared variable x
x
z
y
26
Most precise transformer

• Strong feasibility NP-complete
• Therefore most precise transformer FNP-complete

M1
M2
M3
M4
M5
x
z
w
x
z
y
w
y
Inconsistencyshared variable y
Conclusion cant extend with w
x
z
y
M1 and M4 are weakly-feasiblebut not
strongly-feasible in M1,,M5
27
Making the transformers efficient

• Vanilla transformer inefficient in practice
• Incremental transformers
• Reuse results of previous iterations
• Details in paper
• Engineering optimizations
• Avoid unnecessarily composing subgraphs
• Optimized transformers linear time in practice

28
Prototype implementation

• Implemented in Java
• Supports assertions
• assertReach(x,y)
• assertDisjointLists(x,y)
• assertAcyclicList(x)
• assertCyclicList(x)
• assert(xy) assert(x!y)
• Check cleanness properties
• Absence of null derefs
• Absence of memory leaks
• No misuse of dangling pointers

29
Experiments precision

• Precision lost in just 2/21 benchmarks
• getLast
• Unable to prove x points to last cell
• Due to imprecise transformer
• Can be avoided by simple and efficient heuristics
• queue_2_stack
• Intentionally constructed
• Loss of correlations important to prove property
• Same precision as full heap analysis on other
  benchmarks

30
Experiments standard suite

• Programs operating on 1-2 lists
• insert, delete, reverse, merge
• New analysis slightly less efficient
• But running times

31
Experiments multiple lists
number of shape graphsnumber of subgraphs
x
(89,430 / 7,733)
32
Experiments multiple lists
full shape graph analysis time graph
decomposition analysis time
x
(552.6 / 2.6)
33
Properties of the abstraction

• No loss of precision when connected components
  represent completely independent lists
• Reduces state space exponentially
• Loss of precision when mixing abstract
  states?GD(X1 ? X2) ? ?GD(X1) ? ?GD(X2)
• So where is this technique useful?

34
Related work

• Partial isomorphism join Manevich et al. SAS04
• Applied in more generic context but does not
  reduce exponential blow-ups addressed in this
  paper
• Heap analysis by separationYahav et al.
  PLDI04 Hackett et al. POPL05
• Decompose verification problem itself and
  conservatively approximate contexts
• Heap decomposition for interprocedural analysis
  Rinetzky et al. POPL05 Rinetzky et al.
  SAS05 Gotsman et al. SAS06 Gotsman et al.
  PLDI07
• Decompose/compose at procedure boundaries
• Predicate/variable clustering Clark et al.
  CAV00
• Statically-determined decomposition

35
Conclusions

• New abstraction scheme to control precision/cost
  trade-off for shape analyses
• Efficient algorithms for abstract domain
  operations
• Abstraction
• Partial concretization
• Transformers
• Applicable beyond singly-linked lists
• E.g., class of graphs supported by Lev-Ami et al.
  CAV06
• Doubly-linked lists
• Trees

36
Ongoing work

• Extension for concurrent program analysis
• Future work
• Tune abstraction by counterexample-guided
  refinement

37
Questions?
38
Conservative transformer

• Computes superset of subgraph computed by most
  precise transformer
• Algorithm sketch
• Compose components in footprint of statement
• Apply local ?st on footprint and decompose result
• Test consistency instead of strong feasibility
• Pass other components as is
• Time(?st) polynomial in vars in st
• xnull linear
• x.ny quadratic
• assume(xy) cubic

39
Concretization ?GD

• Maps sets of shape subgraphs to sets of full
  shape graphs
• Mathematically ?GD(XG) G ß(G) ? XG
• Algorithmically by composing weakly-feasible
  subgraphs
• Subgraphs that do not share any variables
• Full shape graph includes all program variables