A Log(n) Multi-Mode Locking Protocol for Distributed Systems

1
A Log(n) Multi-Mode Locking Protocol for
Distributed Systems

• Nirmit Desai, Frank Mueller
• mueller_at_cs.ncsu.edu
• Department of Computer Science
• North Carolina State University

2
Background

• Problem
• Large parallel and distributed systems
• Scalability of synchronization
• Motivation
• Increasing need to share resources
• Inadequate scalability of current protocols
• Approach
• Leveraging hierarchical locking in distributed
  synchronization
• Contribution
• Highly scalable protocol
• Supports hierarchical locking

3
Large Distributed Environments
Cluster Environments
Distributed Environments
4
Distributed Synchronization(Naimi et al.)
Request from A
Request from C

• Single locking mode - Read and Write are not
  distinguished
• Single level of locking granularity (either
  entire table or a tuple)

5
Hierarchical Locking(a.k.a. Multi-granularity
Locking)

• How is a lock associated with data items ?

- High overhead - High concurreny
- Low overhead - No concurrency
- Low overhead - High concurrency
6
Multi-Mode Locking Protocol

• 1. Intention Read (IR) /Intention Write (IW) on
  all upper level resources
• 2. Read (R) locks and Write (W) locks on the
  resource itself

A Reads tuple 2 t0Acquire IR on the
table t1Acquire R on tuple 2 t2Read tuple
2 t3Release R t4Release IR
B Writes tuple 3 t0Acquire IW on the
table t1Acquire W on tuple 3 t2Write tuple
3 t3Release W t4Release IW
7
Multi-Mode Locking Protocol (cont.)

• Modes incompatible? ? Conflict ? Block

A reads entire table t0Acquire R on the
table t1Read table t2Release R on the table

B Writes a tuple t0Request IW on the table t1
wait for lock t2Acquires IW t3Acquire W on
tuple t4...
8
Compatibility Matrix

• X indicates a conflict
• Upgrade Read followed by write to same data
• assures consistent data
• Lock Mode Strength Less compatible ? Stronger
  lock mode
• IR lt R lt U IW lt W

9
Our Protocol - Conventions

• Held Mode Mh Mode of held lock
• Owned Mode Mo Strongest held mode in tree
  (rootedowner)

• Pending Mode Mp await this mode
• Node state (Mo, Mh, Mp)
• Request arrives ? test compatibility ?
  grant/queue/forward based on result
• Actions rules transition tables
• Requester becomes a child of granter

10
A Granting/Forwarding Example
Invariant Node cannot grant request for
stronger mode than it owns ? has to forward it
to parent

• If Mr compatible with Mo ? Mr compatible w/ all
  modes in subtree

11
A Release Example

• Invariant Node knows owned mode of its children

B(0,0,0)

• Notice If a child releases a mode but owned mode
  is unchanged, no need to notify our parent

12
A Queuing Example

• Invariants Queue if request is conflicting.
  Freeze modes if it is a) incompatible with queued
  request and b) grantable otherwise

• Starvation of queued request is avoided, FIFO is
  ensured

13
Measurements

• Metrics of interest Scalability of
• Message overhead per lock request
• Request latency time
• Effect of changing concurrency level on
  scalability
• higher ratio ? lower concurency
• Compare with Naimis protocol
• Coarse Lock the entire table
• Fine Lock each entry
• Multi-airline reservation system
• IBM SP Power3

14
Comparison with Naimis protocols

• Message overhead per Request

Response time per Request

• Naimi coarse has different functionality

15
Effect of concurrency level
C

• Concurrency level increases as of nodes increase

16
Applications

• State management of replicated data
• Distributed databases (Oracle, MS-SQL)
• Distributed transaction processing
• Middleware concurrency services (CORBA)
• Coordinated, distributed Server Farms
• Fault tolerance in clusters
• State replication for redundant computing
• Distributed agreement
• State coherence

17
Related Work

• Naimi et. al. (JPDC 96)
• Distributed mutual exclusion, single level of
  granularity
• Ramamritham et. Al. (SIGMOD 90)
• Concurrency protocols, centralized transaction
  coordinator
• Other approaches complexity gt O(log (n))
• Hierarchical locking only in context of
  distributed synchronization problems in past
• Scalability had not been the focus

18
Conclusions

• Novel peer-to-peer distributed concurrency
  protocol
• Compact formulation rules and tables ?
  simplicity intriguing
• Highly scalable O(log n) msgs, asymptote 3-5
  msgs
• High degree of concurrency common
• Scalability unaffected by concurrency level
• Response time linear increasing w/ concurrency
  level
• Best scalable protocol for hierarchical locking
  known
• Wide applications in parallel and distributed
  world

19
Breakup of Messages
C

• Request propagation is major component

20
Request Message

• more request msgs
• propagation paths longer

21
Token Message

• more contention
• lower chance to grant token

22
Grant Message

• Concurrency level Granting
• Nodes Tree height Granting

• more contention
• more child grants

23
Release Message

• higher concurrency
• more trees release msgs

24
Freeze Message

• higher concurrency
• more freezes (be fair)

25
Rule for Granting at Non-root
X cannot grant
26
Rule for Queuing/Forwarding
Forward Queue
27
Rule for Freezing Modes
Modes to freeze