Summarizing Procedures in Concurrent Programs

1
Summarizing Procedures in Concurrent Programs

• Shaz Qadeer
• Sriram K. Rajamani
• Jakob Rehof
• Microsoft Research

2
Motivation

• How do you scale program analyses for sequential
  programs?
• Summarize at procedure boundaries
• Sharir-Pnueli 81, Reps-Horwitz-Sagiv 95
• Used in compiler dataflow analyses
• Used in error detection tools
• SLAM (Ball-Rajamani 00)
• ESP (Das-Lerner-Seigle 02)

3
Summarization is efficient !

• Boolean program with
• g globals
• n procedures, each with at most m locals
• E size of the CFG of the program
• Complexity O( E ? 2 O(gm) )
• Complexity linear in the number of procedures!

4
Summarization gives termination!

• Possibly recursive boolean programs
• Infinite state systems
• Checking terminates with summarization!

5
Question

• Can summarization help analysis of concurrent
  programs?

6
Difficulty

• Assertion checking for multithreaded programs is
  undecidable
• Even if all variables are boolean
• Further, even if only two threads!
• Reduce emptiness of intersection of two CFLs to
  this problem(Ramalingam 00)

7
Our work

• New model checking algorithm using summarization
• useful for concurrent programs
• Summaries provide re-use and efficiency for
  analyzing concurrent programs
• Enable termination of analysis in a large class
  of concurrent programs
• includes programs with recursion, shared
  variables and concurrency

8
Difficulties in summarizing concurrent programs

• What is a summary?
• For sequential programs
• Summary of procedure P Set of all pre-post
  state pairs (s,s) obtained by invoking P
• This doesnt work for concurrent programs
• Does not model concurrent updates by other threads

9
Insight

• In a well synchronized concurrent program
• A threads computation can be viewed as a
  sequence of transactions
• While analyzing a transaction, interleavings with
  other threads need not be considered
• Key idea Summarize transactions!

10

• How do you identify transactions?

Liptons theory of reduction
11
Four atomicities

• R right movers
• lock acquire

• L left movers
• lock release

• B both right left movers
• variable access holding lock

• N non-movers
• access unprotected variable

12
Transaction

• Any sequence of actions whose atomicities are in
  R(N?)L is a transaction

R
R
R
N
R
L
L
S0
S1
S2
S5
S6
S3
S7
S4
Transaction
Precommit
Postcommit
13
Transactions and summaries

• Corollary of Liptons theorem
• No need to schedule other threads in the middle
  of a transaction
• If a procedure body occurs in a transaction, we
  can summarize it!

14
Resource allocator (1)

• bool availableN
• mutex m
• int getResource()
• int i 0
• L0 acquire(m)
• L1 while (i lt N)
• L2 if (availablei)
• L3 availablei false
• L4 release(m)
• L5 return i
• L6 i
• L7 release(m)
• L8 return i

Choose N 2 Summaries ltpc, i, m,
(a0,a1)gt ? ltpc, i, m, (a0,a1)gt
ltL0, 0, 0, (0, 0)gt ? ltL8, 2, 0,
(0,0)gt ltL0, 0, 0, (0, 1)gt ? ltL5, 1,
0, (0,0)gt ltL0, 0, 0, (1, 0)gt ? ltL5,
0, 0, (0,0)gt ltL0, 0, 0, (1, 1)gt ?
ltL5, 0, 0, (0,1)gt
15

• What if transaction boundaries and procedure
  boundaries do not coincide?

Two level model checking algorithm
16
Two level algorithm

• First level maintains stack
• Second level maintains stack-less summaries
• Summaries can start and end anywhere in a
  procedure

17
Resource allocator (2)

• bool availableN
• mutex mN
• int getResource()
• int i 0
• L0 while (i lt N)
• L1 acquire(mi)
• L2 if (availablei)
• L3 availablei false
• L4 release(mi)
• L5 return i
• else
• L6 release(mi)
• L7 i
• L8 return i

Choose N 2 Summaries ltpc,i,(m0,m1),(a0,
a1gt ? ltpc,i,(m0,m1),(a0,a1)gt
ltL0, 0, (0,0), (0,0)gt ? ltL1, 1, (0,0), (0,0)gt
ltL0, 0, (0,0), (0,1)gt ? ltL1, 1, (0,0), (0,1)gt
ltL0, 0, (0,0), (1,0)gt ? ltL5, 0, (0,0), (0,0)gt
ltL0, 0, (0,0), (1,1)gt ? ltL5, 0, (0,0), (0,1)gt
ltL1, 1, (0,0), (0,0)gt ? ltL8, 2, (0,0), (0,0)gt
ltL1, 1, (0,0), (0,1)gt ? ltL5, 1, (0,0), (0,0)gt
ltL1, 1, (0,0), (1,0)gt ? ltL8, 2, (0,0), (1,0)gt
ltL1, 1, (0,0), (1,1)gt ? ltL5, 1, (0,0), (1,0)gt
18

• Two level model checking algorithm in pictures

Lets first review the sequential CFL algorithm
19
main( )
bar( )
bar()
20
main( )
bar( )
bar()
21

• Two level model checking algorithm in pictures

22
main( )
bar( )
bar()
23

• Three kinds of summaries
• MAX
• MAXCALL
• MAXRETURN

main( )
bar( )
MAXCALL
MAX
End of transaction
bar()
MAXRETURN
MAXRETURN
bar
main
main
T1
T2
24
Concurrency recursion
int g 0 mutex m
Summaries for foo ltpc,r,m,ggt ?
ltpc,r,m,ggt ltL0,1,0,0gt ? ltL5,1,0,1gt ltL0,1,0,
1gt ? ltL5,1,0,2gt Summaries for
main ltpc,q,m,ggt ? ltpc,q,m,ggt ltM0,1,0,0gt
? ltM1,1,0,1gt ltM0,1,0,1gt ? ltM1,1,0,2gt ltM1,1,0
,1gt ? ltM4,1,0,1gt ltM1,1,0,2gt ? ltM4,1,0,2gt

• void foo(int r)
• L0 if (r 0)
• L1 foo(r)
• else
• L2 acquire(m)
• L3 g
• L4 release(m)
• L5 return

void main() int q choose(0,1) M0
foo(q) M1 acquire(m) M2 assert(g gt
1) M3 release(m) M4 return
P main() main()
25

• What if the same procedure is called from
  different phases of a transaction?

Instrument the transaction phase into the state
of the program
26
Transactional context
int gm 0, gn 0 mutex m, n void bar()
N0 acquire(m) N1 gm N2 release(m)
void foo2() M0 acquire(n) M1
gn M2 release(n) M3 bar()
M4 return

• void foo1()
• L0 acquire(n)
• L1 gn
• L2 bar()
• L3 release(n)
• L4 return

P foo1() foo2()
27
Recap of technical problems

• How do you identify transactions
• Using the theory of reduction (Lipton 75)
• What if transaction boundaries do not coincide
  with procedure boundaries?
• Two level model checking algorithm
• First level maintains stack
• Second level maintains stack-less summaries
• Procedure can be called from different phases of
  a transaction
• Instrument the transaction phase into the state
  of program

28
Termination

• A function is transactional if no transaction
  ends in the middle of its exectution (includes
  all transitive callees)
• Theorem For concurrent boolean programs, if all
  recursive functions are transactional, then the
  algorithm terminates.

29
Sequential case

• If we feed a sequential program to our algorithm
  it functions exactly like the Reps-Sagiv-Horwitz-P
  OPL95 algorithm
• Our algorithm generalizes the RHS algorithm to
  concurrent programs!

30
Related work

• Summarizing sequential programs
• Sharir-Pnueli 81, Reps-Horwitz-Sagiv 95,
  Ball-Rajamani 00
• ConcurrencyProcedures
• Bouajjani-Esparza-Touili 02
• Esparza-Podeslki 00
• Reduction
• Lipton 75
• Qadeer-Flannagan 03

31
(joint work with Tony Andrews)
32
Automatic abstraction
SLAM
model checker
Data flow analysis implemented using BDDs
Finite state machines
Push down model
FSM
Boolean program
abstraction
C data structures, pointers, procedure calls,
parameter passing, scoping,control flow
Source code
Sequential C program
33
Zing
model checker
Rich control constructs thread creation,
function call, exception, objects, dynamic
allocation Model checking is undecidable!
abstraction
Source code
Device driver (taking concurrency into account),
web services code
34
What is Zing?

• Zing is a framework for software model-checking
• Language, compiler, runtime, tools
• Supports key software concepts
• Enables easier extraction of models from code
• Supports research in exploring large state spaces
• Operates seamlessly with the VS.Net design
  environment

35
Current status

• Summarization
• Theory to appear in POPL 04
• Implementation in progress
• Zing
• Compiler, model checker and conformance checker
  operational
• State-delta and transaction-based reduction
  implemented
• Plans
• Symbolic reasoning
• Automatic abstraction

36

• Bluetooth demo

37
BPEL4WS checking
BPEL Processes
Buyer
Seller
Zing Model
Auction House
Reg Service