Skiplist-based Concurrent Priority Queues

1
Skiplist-based Concurrent Priority Queues

• Itay Lotan
• Stanford University
• Nir Shavit
• Sun Microsystems Laboratories

IPDPS 2000
2
Priority Queues For Large-Scale MultiProcessor
Machines

• Priority Queue A data structure that allows n
  asynchronous processes to insert an item, or
  delete the item with highest priority
• Large Scale Machine Hundreds of Processors
• Usually different architecture than small scale
  machines

3
Fixed Set Priority Queue

• A priority queue with a fixed and predetermined
  set of priorities
• Used by operating systems to distribute CPU time.
• There is a scalable solution - Funnel Tree
  Shavit, Zemach

4
General Priority Queue

• Supports an unlimited range of priorities
• Between any two priorities may be another
• Useful for
• Heuristic searches
• Graph searches
• Best solution Heap algorithm of Hunt et al
• Works better than binary search trees

5
Heaps Hunt et al
6
Scalability Problem of Heaps
Contention hot spots
7
New Approach - SkipQueues

• Based on Skip List of Pugh
• All locking is distributed and local
• Balancing of structure is probabilistic
• Deletions evenly distributed, minimal contention
• No need to pre-allocate memory
• All operations in expected logarithmic time

8
Coming up Next

1. The inside story on concurrent Skip Lists Pugh
2. How to make a SkipQueue
3. Experimental Results

9
Skip List Structure Pugh
10
Inserting an Element

• Insert (key_t k)
• Randomly choose a level l
• Find at each level of the list up to l the
  largest key smaller than k
• For each level from 1 to l
• Acquire a lock on the item found before
• Insert the new key after it
• Release lock

11
Skip List - Insertion
New Item
12
Deleting an Element

• Delete (key_t k)
• Find k at all levels at which it appears.
• For each level from l to 1
• Acquire locks on k and the item before it.
• Remove k
• Release locks

13
Skip List - Deletion
14
SkipQueues

• Problem How to allow Delete_Min operations
  without creating a bottle neck around the first
  element?

15
Our Solution

• Observation - The lowest level of the Skip List
  contains all the elements in ascending order
• Processes can advance down this list and
  logically delete the first available key
• Each process can then delete the key that it
  previously deleted logically

16
Delete_Min Operation

• Traverse the bottom level of the Skip List
  structure
• For each traversed Item
• Try to SWAP its Deleted flag
• If it was not already deleted, return the key of
  the item
• Else, go on to next item
• If at the end of the list, return EMPTY
• No locking at all

17
From Skip List to SkipQueue
18
Performance Benchmarks

• We compared the performance of 3 structures
• The Heap structure of Hunt et al
• FunnelList a simple linked list, access to
  which is governed by a combining-funnel Shavit,
  Zemach structure
• SkipQueue structure as described before

19
Benchmark Methodology

• We used the Proteus multiprocessor simulator by
  Brewer et al
• We simulated a 256 processors machine similar to
  the MIT Alewife
• Processors alternate between performing small
  amount of work and accessing the queue.

20
Benchmark Methodology Contd

• Processors randomly choose whether to insert or
  delete
• Priorities of inserted items are chosen uniformly
  at random
• We measured the average latency of Insert and
  Delete_Min operations

21
Small Structure - Deletions
22
Small Structure - Insertions
23
Large Structure - Deletions
24
Large Structure - Insertions
25
Large Structure Benchmark With 70 Deletions -
Deletions
26
Large Structure Benchmark With 70 Deletions -
Insertions
27
Conclusions

• SkipQueues scale significantly better than Heaps
• SkipQueues are highly distributed no hot spots
  or bottle necks
• Deletes are 3 times faster and Inserts are 10
  times faster when concurrency reaches 256
  processors
• Future Directions
• Implementation without locks
• Experimenting with other data structures

28
Delete_Min Contd

• In order to assure a stronger ordering property
  we added the following
• Each item is timestamped when its insertion is
  completed
• Each Delete_Min operation notes the time at which
  it begins
• The Delete_Min operation will only try to
  logically delete items that were inserted
  before it started.

29
Serialization Order

• Each successful Delete_Min is ordered at the time
  its logical deletion was completed
• Each unsuccessful Delete_Min that returned EMPTY
  is ordered at the time of its return instruction
• Each uncompleted Delete_Min is ordered at the
  time of its invocation

30
SkipQueue Specification

• For every Delete_Min operation
• Let I be the set of keys inserted by Insert
  operations serialized before it
• Let D be the set of keys removed by Delete_Min
  operations serialized before it
• The returned value is the smallest in the set I
  D, or EMPTY if I D is empty.