CHESS: Systematic Concurrency Testing - PowerPoint PPT Presentation

About This Presentation
Title:

CHESS: Systematic Concurrency Testing

Description:

CHESS in a nutshell. Replace the OS scheduler with a demonic scheduler ... CHESS can systematically test the boot and shutdown process ... So, is CHESS is unsound? ... – PowerPoint PPT presentation

Number of Views:227
Avg rating:3.0/5.0
Slides: 39
Provided by: MadanMu
Category:

less

Transcript and Presenter's Notes

Title: CHESS: Systematic Concurrency Testing


1
CHESS Systematic Concurrency Testing
  • Tom Ball, Sebastian Burckhardt,
  • Madan Musuvathi, Shaz Qadeer
  • Microsoft Research
  • http//research.microsoft.com/CHESS/

2
Testing concurrent programs is HARD
  • Rare thread interleavings expose bugs
  • Coverage problem
  • Testing misses thread interleavings that expose
    errors
  • Reproducibility problem
  • Concurrency bugs Heisenbugs
  • Not reproducible ? hard to debug
  • Crash dumps dont help

3
Thread interleavings
Thread 1
Thread 2
x x
x2 x2
0
1
0
2
2
0
1
4
3
1
2
4
2
2
4
3
8
5
6
4
Concurrency testing today
  • Concurrency testing stress testing
  • Example testing a concurrent queue
  • Create 100 threads performing queue operations
  • Run for days/weeks
  • Stress increases the interleaving variety, but
  • Not systematic might miss interleavings
  • Not predictable cannot find the same error again
  • Makes any error found hard to debug

5
Why stress is not sufficient
1
6
Concurrency testing what we need
  • Methodology and tools to
  • systematically and predictably
  • test thread interleavings

7
CHESS in a nutshell
ConcurrentProgram
  • Replace the OS scheduler with a demonic scheduler
  • Systematically explore all scheduling choices

Win32 API
Kernel Scheduler
Demonic Scheduler
8
CHESS will run this program 6 times exploring all
the different interleavings
Thread 1
Thread 2
x x
x2 x2
0
1
0
2
2
0
1
4
3
1
2
4
2
2
4
3
8
5
6
9
Dont stress, use CHESS
2
10
CHESS architecture
Program
CHESS runs the scenario in a loop
While(not done) TestScenario()
TestScenario()
CHESS
  • Every run takes a different interleaving
  • Every run is repeatable
  • Intercept synch. threading calls
  • To control and introduce nondeterminism

Win32 API
  • Detect
  • Assertion violations
  • Deadlocks
  • Dataraces
  • Livelocks

Kernel Threads, Scheduler,
Synchronization Objects
11
CHESS methodology generalizes
  • Need wrappers for every concurrency API
  • CHESS has wrappers for Win32, .NET, Singularity
  • Wrappers understand the semantics of the API
  • Expose nondeterminism in the API
  • Looking for volunteers to build wrappers for
    Linux and Java

12
CHESS clients
  • PCP Parallel Computing Platform (for
    multi/many-cores)
  • PLINQ Parallel LINQ
  • CDS Concurrent Data Structures
  • STM Software Transactional Memory
  • TPL Task Parallel Library
  • ConcRT Concurrency RunTime
  • CCR Concurrency Coordination Runtime
  • Dryad
  • Part of COSMOS
  • Singularity/Midori
  • CHESS can systematically test the boot and
    shutdown process

13
Stateless model checking Verisoft 97
  • Systematically enumerate all paths in a
    state-space graph
  • Dont capture program states
  • Capturing states is extremely hard for large
    programs
  • Effective for message-passing programs
  • CHESS applies stateless model checking for
    shared-memory multithreaded programs

14
Outline
  • Preemption bounding PLDI 07
  • Fair stateless model checking PLDI 08
  • Sober CAV 08, EC2 08
  • FeatherLite
  • Concurrency Explorer EC2 08

15
Outline
  • Preemption bounding
  • Makes CHESS effective on deep state spaces
  • Fair stateless model checking
  • Sober
  • FeatherLite
  • Concurrency Explorer

16
State space explosion
Thread 1
Thread n
  • Number of executions
  • O( nnk )
  • Exponential in both n and k
  • Typically n lt 10 k gt 100
  • Limits scalability to large programs

x 1 y k
x 1 y k

k steps each
n threads
Goal Scale CHESS to large programs (large k)
17
Preemption bounding
  • Prioritize executions with small number of
    preemptions
  • Two kinds of context switches
  • Preemptions forced by the scheduler
  • e.g. Time-slice expiration
  • Non-preemptions a thread voluntarily yields
  • e.g. Blocking on an unavailable lock, thread end

Thread 1
Thread 2
x 1 if (p ! 0) x p-gtf
x 1 if (p ! 0)
p 0
preemption
x p-gtf
non-preemption
18
Polynomial state space
  • Terminating program with fixed inputs and
    deterministic threads
  • n threads, k steps each, c preemptions
  • Number of executions lt nkCc . (nc)!

  • O( (n2k)c. n! )
  • Exponential in n
    and c, but not in k

Thread 1
Thread 2
  • Choose c preemption points

x 1 y k
x 1 y k
x 1
x 1
  • Permute nc atomic blocks


y k
y k
19
Preemption bounding
3
20
Find lots of bugs with 2 preemptions
Program Lines of code Bugs
Work Stealing Q 4K 4
CDS 6K 1
CCR 9K 3
ConcRT 16K 4
Dryad 18K 7
APE 19K 4
STM 20K 2
TPL 24K 9
PLINQ 24K 1
Singularity 175K 2
37 (total)
Acknowledgement testers from PCP team
21
So, is CHESS is unsound?
  • Soundness prove that the program is correct for
    a given input test harness
  • Need to exhaustively explore all interleavings
  • For small programs, CHESS is sound
  • Iteratively increase the preemption bound
  • Preemption bounding helps scale to large programs
  • A good knob to trade resources for coverage
  • Better search algorithms ? more coverage faster
  • Partial-order reduction
  • Modular testing of loosely-coupled programs

22
Outline
  • Preemption bounding
  • Makes CHESS effective on deep state spaces
  • Fair stateless model checking
  • Makes CHESS effective on cyclic state spaces
  • Enables CHESS to find liveness violations
    (livelocks)
  • Sober
  • FeatherLite
  • Concurrency Explorer

23
Concurrent programs have cyclic state spaces
  • Spinlocks
  • Non-blocking algorithms
  • Implementations of synchronization primitives
  • Periodic timers

Thread 1
Thread 2
! done L2
! done L1
L1 while( ! done) L2 Sleep()
M1 done 1
done L2
done L1
24
A demonic scheduler unrolls any cycle ad-infinitum
Thread 1
Thread 2
while( ! done) Sleep()
done 1
! done
done
! done
done
! done
done
! done
25
Depth bounding
  • Prune executions beyond a bounded number of steps

! done
done
! done
done
! done
done
! done
Depth bound
26
Problem 1 Ineffective state coverage
  • Bound has to be large enough to reach the deepest
    bug
  • Typically, greater than 100 synchronization
    operations
  • Every unrolling of a cycle redundantly explores
    reachable state space

! done
! done
! done
! done
Depth bound
27
Problem 2 Cannot find livelocks
  • Livelocks lack of progress in a program

Thread 1
Thread 2
temp done while( ! temp) Sleep()
done 1
28
Key idea
  • This test terminates only when the scheduler is
    fair
  • Fairness is assumed by programmers
  • All cycles in correct programs are unfair
  • A fair cycle is a livelock

Thread 1
Thread 2
while( ! done) Sleep()
done 1
! done
! done
done
done
29
We need a fair demonic scheduler
  • Avoid unrolling unfair cycles
  • Effective state coverage
  • Detect fair cycles
  • Find livelocks (violations of fair termination)

Test Harness
ConcurrentProgram
Win32 API
Demonic Scheduler
Fair Demonic Scheduler
30
Fair termination allows CHESS to check for
arbitrary liveness properties
  • Example Good Samaritan assumption
  • Forall threads t GF scheduled(t) ? GF yield(t)
  • A thread when scheduled infinitely often yields
    the processor infinitely often
  • Examples of yield
  • Sleep(), ScheduleThread(), asm rep nop
  • Thread completion

Thread 1
Thread 2
while( ! done) Sleep()
done 1
31
Outline
  • Preemption bounding
  • Makes CHESS effective on deep state spaces
  • Fair stateless model checking
  • Makes CHESS effective on cyclic state spaces
  • Enables CHESS to find liveness violations
    (livelocks)
  • Sober
  • Detect relaxed-memory model errors
  • Do not miss behaviors only possible in a relaxed
    memory model
  • FeatherLite
  • Concurrency Explorer

32
C Example
volatile bool isIdling volatile bool hasWork
//Consumer thread void BlockOnIdle()
lock (condVariable) isIdling true
if (!hasWork)
Monitor.Wait(condVariable) isIdling
false //Producer thread
void NotifyPotentialWork() hasWork
true if (isIdling) lock
(condVariable) Monitor.Pulse(condVar
iable)
33
Example Store Buffer Vulnerability
  • Key pieces of code on previous slide
  • On x86, hardware may perform store late
  • Bug Producer thread does not notice waiting
    Consumer, does not send signal

volatile int ii 0 volatile int hw 0
Consumer
Producer
Store ii, 1
Store ii, 1
Load hw, 0
Store hw, 1
Load ii, 1
0
34
Sober algorithm
  • Programmers assume sequential-consistency (SC)
  • Insert synchronizations fences to counter
    memory-model relaxations
  • Sober checks if a program is memory-model safe
  • i.e., program has only SC executions in a memory
    model
  • Reports any such violation as an error
  • Sober is a dynamic monitor that checks if any SC
    execution can be extended to a non-SC execution
  • Theorem CHESS Sober guarantees memory-model
    safety

35
Outline
  • Preemption bounding
  • Makes CHESS effective on deep state spaces
  • Fair stateless model checking
  • Makes CHESS effective on cyclic state spaces
  • Enables CHESS to find liveness violations
    (livelocks)
  • Sober
  • Detect relaxed-memory model errors
  • Do not miss behaviors only possible in a relaxed
    memory model
  • FeatherLite
  • A light-weight data-race detection engine (lt20
    overhead)
  • Concurrency Explorer

36
Outline
  • Preemption bounding
  • Makes CHESS effective on deep state spaces
  • Fair stateless model checking
  • Makes CHESS effective on cyclic state spaces
  • Enables CHESS to find liveness violations
    (livelocks)
  • Sober
  • Detect relaxed-memory model errors
  • Do not miss behaviors only possible in a relaxed
    memory model
  • FeatherLite
  • A light-weight data-race detection engine (lt20
    overhead)
  • Concurrency Explorer
  • First-class concurrency debugging

37
Conclusion
  • Dont stress, use CHESS
  • CHESS binary and papers available at
    http//research.microsoft.com/CHESS
  • Stateless model checking is very effective
  • Preemption bounding to scale to deep state spaces
  • Fair demonic scheduler to handle nonterminating
    programs
  • Need better testing and debugging methodologies
    for concurrent programs

38
Questions
Write a Comment
User Comments (0)
About PowerShow.com