COTS Challenges for Embedded Systems

1
E81 CSE 532S Advanced Multi-Paradigm Software
Development
A Concurrency and Synchronization Pattern
Language
Chris Gill and Venkita Subramonian Department of
Computer Science and Engineering Washington
University, St. Louis cdgill_at_cse.wustl.edu
2
Main Themes of this Talk

• Review of concurrency and synchronization
• Review of patterns and pattern languages
• Design scenario case study
• A peer-to-peer client-server application
• Increasing liveness with concurrency patterns
• Maintaining safety with synchronization patterns

3
Concurrency

• Forms of concurrency
• Logical (single processor) parallelism
• Physical (multi-processor) parallelism
• Goal Increase Liveness
• Progress is made deadlock is avoided
• Also, avoid long blocking intervals if possible
• Progress is also optimized where possible
• Independent activities logically parallel
• Ideally, physically parallel if you can afford
  additional resources
• Full utilization of resources something is
  always running
• E.g., both I/O bound and compute bound threads in
  a process
• The most eligible task is always running
• E.g., the one with the highest priority among
  those enabled

4
Concurrency, Continued

• Benefits
• Performance
• Still make progress if one thread blocks (e.g.,
  for I/O)
• Preemption
• Higher priority threads preempt lower-priority
  ones
• Drawbacks
• Object state corruption due to race conditions
• Resource contention
• Need isolation of inter-dependent operations
• For concurrency, synchronization patterns do this
• At a cost of reducing concurrency somewhat
• And at a greater risk of deadlock

5
Synchronization

• Forms of synchronization
• Thread (lightweight) vs. process (heavy) locks
• Why is process locking more expensive?
• Binary (mutex) vs. counted (semaphore) locks
• What is being locked
• Scopes scoped locking
• Objects thread-safe interface
• Role-based readers-writer locks
• It depends strategized locking
• Goal Safety
• Threads do not corrupt objects or resources
• More generally, bad inter-leavings are avoided
• Atomic runs to completion without being
  preempted
• Granularity at which operations are atomic
  matters

6
Synchronization, Continued

• Benefits
• Safety
• Prevent threads from interfering in bad ways
• Control over (and reproducibility of) concurrency
• Restrict the set of possible inter-leavings of
  threads
• Drawbacks
• Reduces concurrency (extreme form deadlock)
• Adds overhead, increases complexity of code
• Need to apply locking carefully, with a purpose
• Only lock where necessary
• Based on design forces of the specific
  application
• Allow locks to be changed, or even removed

7
Concurrency Synchronization

• Concurrency hazard race conditions
• Two or more threads access an object/resource
• The interleaving of their statements matters
• Some inter-leavings have bad consequences
• Synchronization hazard deadlock
• One or more threads access an object/resource
• Access to the resource is serialized
• Chain of accesses leads to mutual blocking
• Clearly, there is a design tension
• Must be resolved for each distinct application
• Applying patterns helps if guided by design forces

8
Review What is a Pattern Language?

• A narrative that composes patterns
• Not just a catalog or listing of the patterns
• Reconciles design forces between patterns
• Provides an outline for design steps
• A generator for a complete design
• Patterns may leave consequences
• Other patterns can resolve them
• Generative designs resolve all forces
• Internal tensions dont pull design apart

9
Towards a Pattern Language

• Key insights
• Each pattern solves a particular set of issues
• When applied, it may leave others open
• Depends on the scale of the context
• E.g., ACT does not itself address synchronization
• Although it may help in combination with say the
  TSS pattern
• It may also raise new issues
• For example, synchronization raises deadlock
  issues
• None of this matters without a context
• Patterns are only useful when applied to a design
• A design stems from the context (problemforces)
  of something were trying to build
• So, lets imagine were building something

10
Motivating Scenario

• Peer-to-peer client server applications
• Each process sends and receives requests
• Communicates with many other endsystems at once
• Distributed data and analysis components
• Spread more or less evenly throughout network
• Each has a particular function or kind of
  information
• Endsystems can look up locations of specific
  components
• Directory, naming, and trading services are
  available
• Heterogeneous requests
• Long-running independent file retrieval requests
• Send me the news data feed from 04/21/03
• Send me the stock market ticker for 04/21/03
• Short-duration interdependent analysis requests
• Compute the change ( or -) for each stock
  sector
• Then find the largest change in a stock sector
• Then find 5 news events with highest correlation
  to that change

11
Motivating Scenario, Continued

• Given
• A high-quality LAN-scale analysis server
• designed to manage tens of components
• and interact with tens of other similar servers
• all connected via one companys private network
• Assume
• Written in C, leverages C11/STL features
  extensively
• Design uses Wrapper Façade, other basic patterns
• Reactive implementation (initially
  single-threaded)
• Well discuss other alternatives in the next part
  of the course
• E.g., proactive approaches using interceptor to
  mark/notify progress
• Goal
• Scale this to a business-to-business VPN
  environment
• with hundreds instead of tens of nodes locally
• each with hundreds instead of tens of
  collaborators

12
Review of Concurrency Patterns

• Active Object
• Methods hand off request to internal worker
  thread
• Monitor Object
• Threads allowed to access object state
  one-at-a-time
• Half-Sync/Half-Async
• Process asynchronously arriving work in
  synchronous thread
• Leader/Followers
• HS/HA optimization leader delegates until its
  work arrives
• Thread Specific Storage
• Separate storage for each thread, to avoid
  contention
• These complement the synchronization patterns

13
Review of Synchronization Patterns

• Scoped Locking
• Ensures a lock is acquired/released in a scope
• Strategized Locking
• Customize locks for safety, liveness,
  optimization
• Thread-Safe Interface
• Reduce internal locking overhead
• Avoid self-deadlock
• These complement the concurrency patterns

14
How Do We Get Started?

• Simply apply patterns in order listed?
• Sounds ad hoc, no reason to think this would work
• A better idea is to let the context guide us
• Compare current design forces to a pattern
  context
• Lets start with the most obvious issue
• Long-running tasks block other tasks
• Long-running tasks need to be made concurrent
• Leads to selection of our first pattern
• Concurrent handlers for long-running requests
• I.e., apply the Active Object pattern

15
Active Object Context

• Patterns use in this design
• Worker thread in handler
• Input thread deposits requests for it
• Queue decouples request/execution between threads
• LRRHhandle_event
• An enqueue method for AO that puts a command
  object into the queue
• Allows AOs to be registered with an
  asynchronously triggered dispatcher, e.g., an
  input (or reactor) thread

Long Running Request Handlers (LRRH)
enqueue requests
worker thread
input thread
16
Active Object Consequences

• Clear benefits for LRRH
• Method invocation execution are decoupled
• Helps prevent deadlock
• Reduces blocking
• Drawbacks
• Queuing overhead
• Context-switch overhead
• Implementation complexity
• Costs well amortized so far
• But only for LRRH
• So, we cant apply as is to short-running
  requests
• Next design refinement
• Apply Half-Sync/Half-Async

Long Running Request Handlers (LRRH)
enqueue requests
worker thread
input thread
17
Half-Sync/Half-Async Context
Mixed Duration Request Handlers (MDRH)

• Still want long-running activities
• Less overhead when related data are processed
  synchronously
• Also want short-lived enqueue and dequeue
  operations
• Requests arrive asynchronously
• Keep reactor and handlers responsive
• How to get both?
• Make queueing layer smarter
• Chain requests by their dependencies
• E.g., HS/HA (via pipes filters)
• How this is done is crucial
• Blocking input (reactor) thread is bad
• So, worker thread chains them before it works on
  a chain!

worker chains related requests
chain requests
enqueue requests
worker thread
input thread
18
HS/HA Consequences, part I

• Problem race condition
• Reactor thread enqueues while active object
  thread dequeues
• BTW, this has been an issue
• Since we applied Active Object
• But we pushed first on concurrency
• Could have taken the other branch in our design
  process first, instead
• Emphasizes iterative/rapid design!
• Need a mutex
• Serialize access to the queue data structure
  itself
• But then need to avoid deadlock
• What if an exception is thrown? (E.g., a call to
  new fails)
• Solution apply scoped locking

Mixed Duration Request Handlers (MDRH)
worker chains related requests
enqueue requests
worker thread
input thread
19
Scoped Locking Context

• Define/identify scopes around critical sections
• I.e., the active object enqueue and dequeue
  methods
• Provide a local guard object on the stack
• as the first element within the scope of a
  critical section
• Guard ensures a thread automatically acquires
  the lock
• Guard constructor locks upon entering a critical
  section
• Guard ensures thread automatically releases the
  lock
• Guard destructor unlocks upon exiting the local
  scope
• Guarded locks will thus always be released
• even if exceptions are thrown from within the
  critical section

20
Scoped Locking Consequences

• Benefit increased robustness with exceptions
• Liability deadlock potential when used
  recursively
• Self deadlock could occur if the lock is not
  recursive
• E.g., if worker thread generates requests for
  itself
• Pattern language solution
• Apply another pattern!
• I.e., strategized locking

21
Strategized Locking Context Consequences

• Parameterizes synchronization mechanisms
• That protect critical sections from concurrent
  access
• Different handlers may need different kinds of
  synchronization
• Null locks (i.e., if wed ever add passive
  handlers)
• Recursive and non-recursive forms of mutex locks
• Readers/writer locks
• Decouple handlers and locks
• Can change one without changing the other
• Enhancements, bug fixes should be straight
  forward
• Flexibility vs. complexity, time vs. space

22
HSHA Consequences, part II

• Pop back up to HS/HA
• Now that weve addressed queue locking issues
• A new concurrency issue
• Chains that are ready, may wait
• While thread works on others
• Could increase of MDRHs
• Also increase context switches
• A better idea
• Notice the chaining of related requests?
• Looks like sorting the mail
• I.e., apply Leader/Followers

Mixed Duration Request Handlers (MDRH)
worker chains related requests
enqueue requests
worker thread
input thread
23
Leader/Followers Context

• Leader when elected picks an incomplete chain
• Appends requests from reactor to growing chains
• When a chain is complete
• If its for the leader
• Leader takes request chain
• A new leader is elected
• Old leader becomes follower
• Old leader works on the chain
• If its for someone else
• Leader hands off to others
• Continues to build chains

Mixed Duration Request Handlers (MDRH)
follower threads
hand off chains
enqueue requests
leader thread
input thread
24
Leader/Followers Consequences

• Key benefits
• Greater cache affinity thread locality of
  reference
• Context switch only at some events
• Liabilities
• Some increase in implementation complexity
• I.e., to elect leaders, identify whose chain is
  whose
• Well just live with this for the purposes here
• Some additional overhead
• But we already separated reactor from handlers
• So impact on lowest asynchronous level is reduced
• Well go a couple steps further in reducing this
• Overall, benefits of applying LF here outweigh
  costs

25
TSS Context Consequences

• Must lock thread hand-off
• But access w/in a chain is single-threaded and
  thus ok
• Can save results locally
• Put request result in TSS
• Retrieve as input for next request in that chain
• Benefits
• Avoids locking overhead
• Increases concurrency
• Liabilities
• Slightly more complex to store and fetch results
• Portability of TSS (may need to emulate in code)

Mixed Duration Request Handlers (MDRH)
follower threads
hand off chains
enqueue requests
leader thread
input thread
26
A Few Remaining Issues

• Shared services across threads (naming, lookup)
• Threads need safe access
• Design trade-off
• Between simplicity and potential concurrency
  improvement
• Alternative patterns
• Monitor Object
• E.g., for new services
• Thread Safe Interface
• E.g., for legacy services

Mixed Duration Request Handlers (MDRH)
follower threads
hand off chains
enqueue requests
leader thread
input thread
27
Monitor Object Context Consequences
follower threads

• Threads use common services
• E.g., Naming, Directory
• Need thread-safe service access
• Single copy of each service
• Inefficient to duplicate services
• Threads access them sporadically
• Could just make services active
• More complex if even possible
• Solution Monitor Object
• Synchronize access to the service
• Consequences
• If worried about performance, can yield on
  condition variables
• Assume thats not an issue here (reasonable for
  Naming, etc.)

common service
28
Thread-Safe Interface Context and Consequences

• Since synchronous service performance is fine
• Go for simplicity, low-overhead
• Avoids self-deadlock from intra-component method
  calls
• Avoid complexity of condition based thread yield
• Minimizes locking overhead
• Separates Concerns
• Interface methods deal with synchronization
• Implementation methods deal with functionality
• Only option with some single-threaded legacy
  services
• Cant modify, so wrap instead

follower threads
thread-safe service interface
service implementation
29
Pattern Language Review

• We first applied Active Object
• Made long-running handlers concurrent
• Then we applied Half-Sync/Half-Async
• For chains of short-running requests
• Next were scoped strategized locking
• To make thread hand-off safe, efficient
• Then we applied Leader/Followers
• To reduce thread context switches

30
Pattern Language Review, Continued

• We applied TSS next
• Efficient access to separate information for each
  thread can help reduce contention
• Finally we applied Monitor Object and/or
  Thread-Safe Interface
• For synchronized access to common (possibly
  legacy) services

31
Concluding Remarks

• Design mastery is often not based on novelty
• Though occasionally it can help
• It is mostly a matter of careful observation
• of the current context as the design unfolds
• and how consequences of each decision shape later
  ones
• Study patterns and pattern languages
• Benefit from prior experience with patterns
• Including published experience of other design
  masters
• Practice working through design examples like
  this
• Build intuition about how patterns fit together
• Mastery is a journey not a destination
• Learn from every design, yours and others