Static and Runtime Verification A Monte Carlo Approach

1
Static and Runtime VerificationA Monte Carlo
Approach
Radu Grosu
State University of New York at Stony
Brook grosu_at_cs.sunysb.edu
2
Embedded Software Systems

• Difficult to develop maintain
• Concurrent and distributed (OS, ES, middleware),
• Complicated by DS improving performance (locks,
  RC,...),
• Mostly written in C programming language.
• Have to be high-confidence
• Provide the critical infrastructure for all
  applications,
• Failures are very costly (business, reputation),
• Have to protect against cyber-attacks.

3
What is High-Confidence?
Ability to guarantee that
?
system-software S satisfies LTL property f
4
Automata-Theoretic Approach

• Every LTL formula ? can be translated to a Büchi
  automaton B? such that L(?) L(B?).
• Büchi automaton NFA over ?-words with acceptance
  condition - a final state must be visited ?-
  often.
• State transition graph of S can also be viewed as
  a Büchi automaton.
• Satisfaction reduced to language emptiness
• S ? ? ? L(BS ? B?? ) ?

5
Checking Non-Emptiness
Lassos Computation Tree (CT) of B
recurrence diameter
Explore all lassos in the CT DDFS,SCC time
efficient DFS memory efficient
6
Checking for High-Confidence (in-principle)
All Lassos Non-accepting
BA BS
LTL-P ?
BA BS ? B??
Instrumenter (Product)
Execution Engine
Accepting Lasso L
7
Randomized Algorithms

• Huge impact on CS (distributed) algorithms,
  complexity theory, cryptography, etc.
• Takes of next step algorithm may depend on random
  choice (coin flip).
• Benefits of randomization include simplicity,
  efficiency, and symmetry breaking.

8
Randomized Algorithms

• Monte Carlo may produce incorrect result but
  with bounded error probability.
• Example Elections result prediction
• Las Vegas always gives correct result but
  running time is a random variable.
• Example Randomized Quick Sort

9
Monte Carlo Approach
Lassos Computation tree (CT) of B
recurrence diameter

flip a k-sided coin
Explore N(?,?) independent lassos in the CT Error
margin ? and confidence ratio ?
10
Lassos Probability Space
11
Geometric Random Variable

• Value of geometric RV X with parameter pz
• No. of independent trials (lassos) until success
• Cumulative Distribution Function
• PX ? N 1 (1-pz)N

12
How Many Lassos?

• Requiring 1 (1-pz)N 1- d yields
• N ln (d) / ln (1- pz)
• Lower bound on number of trials N needed to
  achieve success with confidence ratio d.

13
What If pz Unknown?

• Requiring pz ? e yields
• M ln (d) / ln (1- e) ? N ln (d) / ln
  (1- pz)
• and therefore PX ? M ? 1- d
• Lower bound on number of trials M needed to
  achieve success with
• confidence ratio d and error margin e .

14
Statistical Hypothesis Testing

• Null hypothesis H0 pz ? e
• Inequality becomes P X ? M H0 ? 1- d
• If no success after N trials, i.e., X gt M, then
  reject H0

15
Monte Carlo Verification (MV)
input B(S,Q,Q0,d,F), e, d N ln (d) / ln
(1- e) for (i 1 i ? N i) if (RL(B) 1)
return (1, error-trace) return (0, reject H0
with a Pr X gt N H0 lt d) RL(B) performs
a uniform random walk through B
storing states encountered in hash table to
obtain a random sample (lasso).
16
Model Checking ISOLA04, TACAS05

• Implemented DDFS and MV in jMocha model checker
  for synchronous systems specified using Reactive
  Modules.
• Performance and scalability of MV compares very
  favorably to DDFS.

17
DPh Symmetric Unfair Version
(Deadlock freedom)
18
Checking for High-Confidence (in-practice)

• Make scalability a priority
• Open source compiler technology started to
  mature,
• Apply techniques to source code rather than
  models,
• Models can be obtained by abstraction-refinement
  techniques,
• Probabilistic techniques trade-of between
  precision-effort.

19
GCC Compiler

• Early stages a modest C compiler.
• Translation source code translated directly to
  RTL.
• Optimization at low RTL level.
• High level information lost calls, structures,
  fields, etc.
• Now days full blown, multi-language compiler
• generating code for more than 30 architectures.
• Input C, C, Objective-C, Fortran, Java and
  Ada.
• Tree-SSA added GENERIC, GIMPLE and SSA ILs.
• Optimization at GENERIC, GIMPLE, SSA and RTL
  levels.
• Verification Tree-SSA API suitable for
  verification, too.

20
GCC Compilation Process
21
GCC Compilation Process
API Plug-In
22
C Program and its GIMPLE IL
int main int a,b,c int T1,T2,T3,T4
a 5 b a 10 T1 foo(a,b)
T2 a T1 if (a gt T2) goto fi T3
b / a T4 b a c T2 T3
b b 1 fi bar(a,b,c)
int main() int a,b,c a 5 b a 10
c a foo(a,b) if (a gt c) c b/a
ba bar(a,b,c)
Gimplify
23
Associated GIMPLE CFG
24
MC Static Verification of ESS SOFTMC05, NGS06
25
Monte Carlo Algorithm

• Input a set of CFGs.
• Main function A specifically designated CFG.
• Random walks in the Büchi automaton generated
  on-the-fly.
• Initial state of the main routine bookkeeping
  information.
• Next state choose process call GAM on its CFG.
• Processes created by using the fork primitive.
• Optimization GAM returns only upon context
  switch.
• Lassos detected by using a hierarchic hash
  table.
• Local variables removed upon return from a
  procedure.

26
GIMPLE Abstract Machine (GAM)

• Interprets GIMPLE statements according to their
  semantics. Interesting
• Inter-procedural call(), return(). Manipulate
  the frame stack.
• Catches and interprets function calls to
  various modeling and concurrency primitives
• Modeling toss(), assert(). Nondeterminism and
  checks.
• Processes fork(), Manipulate the process
  list.
• Communication send(), recv(). Manipulate shared
  vars. May involve a context switch.

27
Results TCAS
28
MC Runtime Verification of ESS MBT06, NGS06
SS S
Gimplify
GCC
CFG BS
CFG BS ? B??
Instrument
LTL-P ?
Verifier
29
Runtime Verification Challenges

• Inserting instrumentation code
• Verifying states and transitions
• Reducing overheads

30
Inserting Instrumentation Code

• struct inode my_inode
• atomic_t my_atomic
• my_atomic
• my_inode-gti_count

if(instrument) log_event(ATOMIC_INC,
INODE, my_atomic)
atomic_inc(my_atomic)
31
Instrumentation Plug-Ins

• Ref-Counts detects misuse of reference counts
• Instruments inc(rc), dec(rc),
• Checks st-inv (rc?0), tr-inv (rc'-rc1),
  leak-inv (rcgt0 gt rc0),
• Maintains a list of reference counts and their
  container type.
• Malloc detects allocation bugs at runtime
• Instruments malloc() and free() function calls,
• Checks sequences free()free(), free() and
  malloc(),
• Maintains a list of existing allocations.

32
RC Runtime Verification

• Lasso concept weakened (abstracted)
• Execution where RC vary 0 ? ? 0
• State may include FS caches, HW regs, etc
• Lasso sampling used to reduce overhead
• Check for acceptance (error)
• Dynamically adjust sampling rate

33
Sampling Granularity
Sample
34
State and Transition Invariants
Change gt1
Change lt1
Value lt0
35
The Leak Invariant
Timeout
Timeout
36
Proof of Concept

• Checked Linux file system cache objects
• inodes on-disk files
• dentries name-space nodes
• Optionally, log all events
• Simple per-category sampling policy
• Initially sample all objects
• Hypothesize err. rate e gt 10-5 and con. ratio d
  10-5
• Stop sampling if hypothesis is false.

37
Benchmarks

• Directory traversal benchmark
• Create a directory tree (depth 5, degree 6)
• Traverse the tree
• Recursively delete the tree
• Also tested GNU tar compilation
• Testbed
• 1.7GHz Pentium 4 (256Kb cache)
• 1Gbyte RAM
• Linux 2.6.10

38
Results
Logging 10x
3x
1,33x
39
Conclusions

• GSRV is a novel tool suite for randomized
• Static and runtime verification of ESS (growing)
• General purpose tools (plug-ins)
• Code instrumenter constructs the product BA
• Intra/inter-procedural slicer in work
• Static verification tools (plug-ins)
• GAM CFG-GIMPLE abstract machine
• Monte Carlo MC statistical algorithm for LTL-MC
• Runtime verification tools (static libraries)
• Dispatcher catches and dispatches events to RV
• Monte Carlo RV statistical algorithm for LTL-RV