CSE 326: Data Structures Priority Queues (Heaps)

1
CSE 326 Data Structures Priority Queues (Heaps)

• Lecture 9 Monday, Jan 27, 2003

2
Not Quite Queues

• Consider applications
• ordering CPU jobs
• searching for the exit in a maze
• emergency room admission processing
• Problems?
• short jobs should go first
• most promising nodes should be searched first
• most urgent cases should go first

3
Priority Queue ADT

• Priority Queue operations
• Priority Queue property for two elements in the
  queue, x and y, if x has a lower priority value
  than y, x will be deleted before y

create ? heap insert heap ? value ?
heap findMin heap ? value deleteMin heap ?
heap is_empty heap ? boolean
F(7) E(5) D(100) A(4) B(6)
deleteMin
insert
C(3)
G(9)
4
Applications of the Priority Q

• Hold jobs for a printer in order of length
• Store packets on network routers in order of
  urgency
• Simulate events
• Anything greedy

5
Discrete Event Simulation

• An event is a pair (x,t) where x describes the
  event and t is time it should occur
• A discrete event simulator (DES) maintains a set
  S of events which it intends to simulate in time
  order

repeat Find and remove (x0,t0) from S such
that t0 is minimum Do whatever x0 says to do
in the process new events (x1,t1)(xk,tk)
may be generated Insert the new events into S

6
Emergency Room Simulation

• Patient arrive at time t with injury of
  criticality C
• If no patients waiting and a free doctor, assign
  them to doctor and create a future departure
  event else put patient in the Criticality
  priority queue
• Patient departs at time t
• If someone in Criticality queue, pull out most
  critical and assign to doctor create a future
  departure event

arrive(t,c)
patient generator
time queue
criticality (triage) queue
depart(t)
assignpatient to doctor
arrive(t,c)
depart(t)
7
Naïve Priority Queue Data Structures

• Unsorted list
• insert
• findMin
• deleteMin
• Sorted list
• insert
• findMin
• deleteMin

8
BST Tree Priority Queue Data Structure

• Regular BST
• insert
• findMin
• deleteMin
• AVL Tree
• insert
• findMin
• deleteMin

8
11
5
12
10
6
2
13
4
14
7
9
Can we do better?
9
Binary Heap Priority Q Data Structure

• Heap-order property
• parents key is less than childrens keys
• result minimum is always at the top
• Structure property
• complete tree with fringe nodes packed to the
  left
• result depth is always O(log n) next open
  location always known

How do we find the minimum?
10
Nifty Storage Trick

• Calculations
• child
• parent
• root
• next free

1
2
3
4
7
5
6
8
9
12
10
11
1
2
3
4
5
6
7
8
9
10
11
12
0
2
4
5
7
6
10
8
11
9
12
14
20
12
11
Nifty Storage Trick

• Calculations
• child left 2node
• right2node1
• parent floor(node/2)
• root 1
• next free length1

1
2
3
4
7
5
6
8
9
12
10
11
1
2
3
4
5
6
7
8
9
10
11
12
0
2
4
5
7
6
10
8
11
9
12
14
20
12
12
findMin
pqueue.findMin()
2
Time O(1)
13
DeleteMin
pqueue.deleteMin()
2
20
5
4
8
10
6
7
20
14
12
9
11
14
Percolate Down
20
4
5
4
5
20
8
10
6
7
8
10
6
7
14
12
9
11
14
12
9
11
4
4
5
6
5
6
8
10
20
7
8
10
12
7
14
12
9
11
14
20
9
11
15
DeleteMin Code

• Comparable deleteMin()
• x A1
• A1Asize--
• percolateDown(1)
• return x

percolateDown(int hole) tmpAhole while
(2hole lt size) left 2hole right
left 1 if (right lt size
Aright lt Aleft) target right
else target left if (Atarget lt
tmp) Ahole Atarget hole
target else break Ahole
tmp
Trick to avoid repeatedly copying the value at
A1
Move down
Time O(log n)(why ?)
16
Insert
pqueue.insert(3)
2
5
4
8
10
6
7
20
14
12
9
11
3
17
Percolate Up
2
2
5
4
5
4
8
10
6
7
8
3
6
7
20
14
12
9
11
3
20
14
12
9
11
10
2
3
4
8
5
6
7
20
14
12
9
11
10
18
Insert Code

• void insert(Comparable x)
• // Efficiency hack we wont actually put x
• // into the heap until weve located the
  position
• // it goes in. This avoids having to copy it
• // repeatedly during the percolate up.
• int hole size
• // Percolate up
• for( holegt1 x lt Ahole/2 hole hole/2)
• Ahole Ahole/2
• Ahole x

Time O(log n)(why ?)
19
Performance of Binary Heap
Binary heap worst case Binary heap avg case AVL tree worst case BST tree avg case
Insert O(log n) O(1) percolates 1.6 levels O(log n) O(log n)
Delete Min O(log n) O(log n) O(log n) O(log n)

• In practice binary heaps much simpler to code,
  lower constant factor overhead

20
Changing Priorities

• In many applications the priority of an object in
  a priority queue may change over time
• if a job has been sitting in the printer queue
  for a long time increase its priority
• unix renice
• Must have some (separate) way of find the
  position in the queue of the object to change
  (e.g. a hash table)

21
Other Priority Queue Operations

• decreaseKey
• Given the position of an object in the queue,
  increase its priority (lower its key). Fix heap
  property by
• increaseKey
• given the position of an an object in the queue,
  decrease its priority (increase its key). Fix
  heap property by
• remove
• given the position of an an object in the queue,
  remove it. Do increaseKey to infinity then

22
BuildHeap

• Task Given a set of n keys, build a heap all at
  once
• Approach 1 Repeatedly perform Insert(key)
• Complexity

23
BuildHeapFloyds Method
5
11
3
10
6
9
4
8
1
7
2
12
pretend its a heap and fix the heap-order
property!
12
buildHeap() for (isize/2 igt0 i--)
percolateDown(i)
11
5
9
6
10
3
2
7
1
8
4
24
Build(this)Heap
12
12
11
5
11
5
9
2
10
3
9
2
1
3
6
7
1
8
4
6
7
10
8
4
12
12
2
5
2
1
9
6
1
3
9
6
5
3
11
7
10
8
4
11
7
10
8
4
25
Finally
1
2
3
9
6
5
4
11
7
10
8
12
26
Complexity of Build Heap

• Note size of a perfect binary tree doubles (1)
  with each additional layer
• At most n/4 percolate down 1 levelat most n/8
  percolate down 2 levelsat most n/16 percolate
  down 3 levels

O(n)
27
Thinking about Heaps

• Observations
• finding a child/parent index is a multiply/divide
  by two
• operations jump widely through the heap
• each operation looks at only two new nodes
• inserts are at least as common as deleteMins
• Realities
• division and multiplication by powers of two are
  fast
• looking at one new piece of data terrible in a
  cache line
• with huge data sets, disk accesses dominate

28
Solution d-Heaps
1

• Each node has d children
• Still representable by array
• Good choices for d
• optimize performance based on of
  inserts/removes
• choose a power of two for efficiency
• fit one set of children in a cache line
• fit one set of children on a memory page/disk
  block

2
3
7
9
6
5
4
8
10
12
11
4
3
7
2
8
5
12
11
10
6
9
1
12
29
New Operation Merge

• Merge(H1,H2) Merge two heaps H1 and H2 of size
  O(N).
• E.g. Combine queues from two different sources to
  run on one CPU.
• Can do O(N) Insert operations O(N log N) time
• Better Copy H2 at the end of H1 (assuming array
  implementation) and use Floyds Method for
  BuildHeap.
• Running Time O(N)
• Can we do even better? (i.e. Merge in O(log N)
  time?)

30
Binomial Queues
insert heap ? value ? heap findMin heap ?
value deleteMin heap ? heap merge heap ? heap
? heap

• All in O(log n) time
• Recursive Definition of Binomial Tree Bk of
  height k
• B0 single root node
• Bk Attach Bk-1 to root of another Bk-1
• Idea a binomial heap H is a forest of binomial
  trees H B0 B1 B2 ... Bkwhere each Bi may
  be present, or may be empty

31
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
32
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
33
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
34
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
35
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
36
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
37
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
Recall a binomial heap may have any subset of
these trees
38
Why Binomial?

• Why are these trees called binomial?
• Hint how many nodes at depth d?

B0 B1 B2 B3
39
Why Binomial?

• Why are these trees called binomial?
• Hint how many nodes at depth d?
• Number of nodes at different depths d for Bk
• 1, 1 1, 1 2 1, 1 3 3 1,
• Binomial coefficients of (a b)k
  k!/((k-d)!d!)

B0 B1 B2 B3
40
Binomial Queue Properties

• Suppose you are given a binomial queue of N nodes
• There is a unique set of binomial trees for N
  nodes
• What is the maximum number of trees that can be
  in an N-node queue?
• 1 node ? 1 tree B0 2 nodes ? 1 tree B1 3 nodes
  ? 2 trees B0 and B1 7 nodes ? 3 trees B0, B1 and
  B2
• Trees B0, B1, , Bk can store up to 20 21
  2k 2k1 1 nodes N.
• Maximum is when all trees are used. So, solve
  for (k1).
• Number of trees is ? log(N1) O(log N)

41
Definition of Binomial Queues
Binomial Queue forest of heap-ordered
binomial trees
B0 B2 B0 B1 B3
1
-1
21
5
3
3
7
2
1
9
6
11
5
8
7
Binomial queue H1 5 elements 101 base 2 ? B2 B0
Binomial queue H2 11 elements 1011 base 2 ? B3
B1 B0
6
42
findMin()

• In each Bi, the minimum key is at the root
• So scan sequentially B1, B2, ..., Bk, compute the
  smallest of their keys
• Time O(log n) (why ?)

B0 B1 B2 B3
43
Binomial Queues Merge

• Main Idea Merge two binomial queues by merging
  individual binomial trees
• Since Bk1 is just two Bks attached together,
  merging trees is easy
• Steps for creating new queue by merging
• Start with Bk for smallest k in either queue.
• If only one Bk, add Bk to new queue and go to
  next k.
• Merge two Bks to get new Bk1 by making larger
  root the child of smaller root. Go to step 2 with
  k k 1.

44
Example Binomial Queue Merge
H1 H2
1
-1
5
3
21
3
9
7
2
1
6
11
5
7
8
6
45
Example Binomial Queue Merge
H1 H2
1
-1
5
3
3
9
7
2
1
6
21
11
5
7
8
6
46
Example Binomial Queue Merge
H1 H2
1
-1
5
3
9
7
2
1
6
3
11
5
7
8
21
6
47
Example Binomial Queue Merge
H1 H2
1
-1
3
5
7
2
1
3
11
5
9
6
8
21
6
7
48
Example Binomial Queue Merge
H1 H2
-1
1
3
2
1
5
7
3
11
5
8
9
6
6
21
7
49
Example Binomial Queue Merge
H1 H2
-1
1
3
2
1
5
7
3
11
5
8
9
6
6
21
7
50
Binomial Queues Merge and Insert

• What is the run time for Merge of two O(N)
  queues?
• How would you insert a new item into the queue?

51
Binomial Queues Merge and Insert

• What is the run time for Merge of two O(N)
  queues?
• O(number of trees) O(log N)
• How would you insert a new item into the queue?
• Create a single node queue B0 with new item and
  merge with existing queue
• Again, O(log N) time
• Example Insert 1, 2, 3, ,7 into an empty
  binomial queue

52
Insert 1,2,,7
1
53
Insert 1,2,,7
1
2
54
Insert 1,2,,7
3
1
2
55
Insert 1,2,,7
3
1
2
4
56
Insert 1,2,,7
1
2
3
4
57
Insert 1,2,,7
1
5
2
3
4
58
Insert 1,2,,7
1
5
2
3
6
4
59
Insert 1,2,,7
1
5
7
2
3
6
4
60
Binomial Queues DeleteMin

• Steps
• Find tree Bk with the smallest root
• Remove Bk from the queue
• Delete root of Bk (return this value) You now
  have a new queue made up of the forest B0, B1, ,
  Bk-1
• Merge this queue with remainder of the original
  (from step 2)
• Run time analysis Step 1 is O(log N), step 2 and
  3 are O(1), and step 4 is O(log N). Total time
  O(log N)
• Example Insert 1, 2, , 7 into empty queue and
  DeleteMin

61
Insert 1,2,,7
1
5
7
2
3
6
4
62
DeleteMin
5
7
2
3
6
4
63
Merge
5
2
3
6
4
7
64
Merge
5
2
3
6
4
7
65
Merge
5
2
6
3
7
4
66
Merge
5
2
6
3
7
DONE!
4
67
Implementation of Binomial Queues

• Need to be able to scan through all trees, and
  given two binomial queues find trees that are
  same size
• Use array of pointers to root nodes, sorted by
  size
• Since is only of length log(N), dont have to
  worry about cost of copying this array
• At each node, keep track of the size of the (sub)
  tree rooted at that node
• Want to merge by just setting pointers
• Need pointer-based implementation of heaps
• DeleteMin requires fast access to all subtrees of
  root
• Use First-Child/Next-Sibling representation of
  trees

68
Efficient BuildHeap for Binomial Queues

• Insert one at a time - O(n log n)
• Better algorithm
• Start with each element as a singleton tree
• Merge trees of size 1
• Merge trees of size 2
• Merge trees of size 4
• Complexity