Data Structures

1
Data Structures Algorithms Priority Queues
HeapSort
2
HeapSort Priority Queues

• Often need to insert and remove items dynamically
• Priority queues for scheduling
• Event-based simulations
• Numerical computations
• Desired
• Work in-place
• Fast sorting (N lg N time)
• Support fast insertion
• Support fast removal of largest item

3
Priority Queues

• Defn. 9.1 A priority queue is a data structure
  of items with keys that supports two basic
  operations
• insert a new item, and
• remove the item with the largest key.
• Can use a priority queue to sort by
• inserting all items,
• then removing all items.

4
Priority Queue ADT
Basic interface
template ltclass Itemgt class PQ private //
implementation-dependent code public PQ(int) in
t empty() const void insert(Item) Item
getmax()
5
Priority Queues

• May want to support many other operations
• Construct a priority queue from N items
• Insert a new item
• Remove the maximum item
• Change the priority (key) of an item
• Remove an arbitrary item
• Join two priority queues into one

6
Priority Queue

• Basic implementations
• 1 quick insert/slow remove
• Add item to unordered list/array
• Remove max by searching list, swap with last item
  (no holes)

• 2 slow insert/fast remove
• Add item to ordered list/array
• Remove max by removing last item

7
Priority Queue
Worst case costs of PQ operations
Implementation Insert Remove Max Remove Find Max Change Priority Join
Ordered array N 1 N 1 N N
Ordered list N 1 1 1 N N
Unordered array 1 N 1 N 1 N
Unordered list 1 N 1 N 1 1
Heap lg N lg N lg N 1 lg N N
Binomial queue lg N lg N lg N lg N lg N lg N
Best in theory 1 lg N lg N 1 1 1
8
Heap
Defn 9.2 A tree is heap-ordered if the key in
each node is less than or equal to the key in the
node's parent (if any). Property 9.1 No node in
a heap-ordered tree has a key larger than the
root
9
Heap
Very convenient to use a complete binary tree
(tree filled level order left to right). Complete
binary tree easy to represent in array with root
in a1, its children at a2 and a3, ai's
at a2i and a2i1 Defn 9.3 A heap is a set
of nodes arranged in a complete heap-ordered
tree, represented as an array.
10
Heap
T
M
R
H
I
O
L
G
A
leftChild(i) 2I rightChild(i) 2i1
T
R
M
L
O
I
H
A
G
1
2
3
4
5
6
7
8
9
11
Heapify
Making an array interpreted as a complete binary
tree become heap-ordered. If start with heap,
and make a modification that could violate the
heap property, then heapify to fix heap.
12
Heapify
If priority of a node is increased, or new node
is added at bottom (end of array), must travel
up until node is in right place. If priority of
a node is decreased, or replace root with another
node, then must travel down the heap until
suitable place is found.
13
Heap
T
M
R
H
I
O
L
G
A
T
R
M
L
O
I
H
A
G
1
2
3
4
5
6
7
8
9
14
Heap
T
M
R
H
I
O
L
G
A
T
R
M
L
O
I
H
A
G
1
2
3
4
5
6
7
8
9
15
Heap - broken
T
M
R
H
I
O
L
G
A
S
T
R
M
L
O
I
H
A
G
S
1
2
3
4
5
6
7
8
9
10
16
Heap - broken
T
M
R
H
I
O
L
G
A
S
T
R
M
L
O
I
H
A
G
S
1
2
3
4
5
6
7
8
9
10
17
Heap - broken
T
M
R
H
I
S
L
G
A
O
T
R
M
L
S
I
H
A
G
O
1
2
3
4
5
6
7
8
9
10
18
Heap - broken
T
M
R
H
I
S
L
G
A
O
T
R
M
L
S
I
H
A
G
O
1
2
3
4
5
6
7
8
9
10
19
Heap - broken
T
M
S
H
I
R
L
G
A
O
T
S
M
L
R
I
H
A
G
O
1
2
3
4
5
6
7
8
9
10
20
Heap - fixed
T
M
S
H
I
R
L
G
A
O
T
S
M
L
R
I
H
A
G
O
1
2
3
4
5
6
7
8
9
10
21
Bottom-up Heapify
template ltclass Itemgt void bubbleUp(Item a, int
k) while (k gt 1 ak/2 lt ak)
exch(ak, ak/2) k / 2
22
Heap remove replace root
T
M
S
H
I
R
L
G
A
O
T
S
M
L
R
I
H
A
G
O
1
2
3
4
5
6
7
8
9
10
23
Heap remove replace root
T
M
S
H
I
R
L
G
A
O
T
S
M
L
R
I
H
A
G
O
1
2
3
4
5
6
7
8
9
10
24
Heap remove replace root
M
S
H
I
R
L
G
A
O
S
M
L
R
I
H
A
G
O
1
2
3
4
5
6
7
8
9
10
25
Heap remove replace root
M
S
H
I
R
L
G
A
O
S
M
L
R
I
H
A
G
O
1
2
3
4
5
6
7
8
9
10
26
Heap broken
O
M
S
H
I
R
L
G
A
O
S
M
L
R
I
H
A
G
1
2
3
4
5
6
7
8
9
27
Heap broken
O
M
S
H
I
R
L
G
A
O
S
M
L
R
I
H
A
G
1
2
3
4
5
6
7
8
9
28
Heap broken
S
M
O
H
I
R
L
G
A
S
O
M
L
R
I
H
A
G
1
2
3
4
5
6
7
8
9
29
Heap broken
S
M
O
H
I
R
L
G
A
S
O
M
L
R
I
H
A
G
1
2
3
4
5
6
7
8
9
30
Heap broken
S
M
R
H
I
O
L
G
A
S
R
M
L
O
I
H
A
G
1
2
3
4
5
6
7
8
9
31
Heap fixed
S
M
R
H
I
O
L
G
A
S
R
M
L
O
I
H
A
G
1
2
3
4
5
6
7
8
9
32
Top-down Heapify
template ltclass Itemgt void siftDown(Item a, int
k, int N) while (2k lt N)
int j 2 // left child // pick
larger child if (j lt N aj lt aj1)
j // if out of place, ... if
(!(ak lt aj)) break // swap with
larger child exch(ak, aj) k
j
33
Priority Queue
So, implement insert by appending new item to end
of array (bottom of tree), then bubbling
up. Implement remove max by removing root item
(first item in array) and moving last item to
root, then sifting down. Each of these operations
take at most O(lg N) time (maximum path length in
complete binary tree).
34
Priority Queue
The change priority, remove arbitrary node, and
replace the maximum operations all take O(lg N)
time in a heap. Sort by inserting, then removing
items from largest to smallest. Note that
joining two heaps is not mentioned here a bit
harder to do.
35
Priority Queue Sort
template ltclass Itemgt void PQsort(Item a, int
l, int r) int k PQltItemgt pq(r-l1) // make
priority queue for (k l k lt r k)
pq.insert(ak) // insert all items for (k
r k gt 0 --k) // in decreasing order
ak pq.getmax() // copy back to a
36
HeapSort
If already have unordered array, can put into
heap order faster than N lg N time. Rather than
do N insertions, just sift down starting at next
to bottom level. Each of these operations starts
with two sub-heaps and a root that may be out of
place, and makes a bigger sub-heap. This is
bottom-up heap construction.
37
HeapSort
Property 9.4 Bottom-up heap construction takes
linear time. Prf For a tree of size N 2n 1,
we have 2n-2 sub-heaps of size 3, which may need
one promotion 2n-3 sub-heaps of size 7 which may
need two promotions, etc. The maximum number of
promotions is Sum k 2n-k-1 2n n 1 lt N.
Proof is similar when N1 is not 2n
38
HeapSort
template ltclass Itemgt void heapSort(Item a, int
l, int r) int k, N r-l1 Item pq
al-1 // make part of a PQ for (k N/2 k gt
0 --k) siftDown(pq, k, N) // bottom-up
heapify while (N gt 1) // in decreasing order
exch(pq1, pqN) // largest to
end siftDown(pq, 1, --N) // fix heap
39
HeapSort
Property 9.5 HeapSort uses fewer than 2N lg N
comparisons to sort N elements. Prf Build heap
bottom up in linear time. Then swap largest item
with last item, decrement heap size, and siftDown
the new root in log time. A bound of 3 N lg N is
thus easy to get, but a more careful counting
shows that it is really 2 N lg N.
40
HeapSelect
Property 9.6 Heap-based selection finds the kth
largest element in time proportional to N when k
is small or close to N, and in time N lg N
otherwise. Build a heap (linear time), then
remove k largest items (2k lg N operations), or a
minimum-oriented heap and remove N-k smallest
items.
41
Which Sort to Use?
Heapsort? Quicksort? Mergesort? Often an issue
of choosing between stability and using extra
space. Heapsort vs. quicksort reduces to choice
between average-case speed and worst-case speed.
Mergesort and quicksort exhibit good locality
(plays nice with virtual memory).
42
Next
Binomial queues (fast join for PQ) Radix Sort