Heaps

1
Heaps

• Chapter 6 in CLRS

2
Motivation

• Dijkstras algorithm for single source shortest
  path
• Prims algorithm for minimum spanning trees

3
Motivation

• Want to find the shortest route from New York to
  San Francisco
• Model the road-map with a graph

4
A Graph G(V,E)
V is a set of vertices
E is a set of edges (pairs of vertices)
5
Model driving distances by weights on the edges
V is a set of vertices
E is a set of edges (pairs of vertices)
6
Source and destination
V is a set of vertices
E is a set of edges (pairs of vertices)
7
Dijkstras algorithm

• Assume all weights are non-negative
• Finds the shortest path from some fixed vertex s
  to every other vertex

8
Example
s4
8
15
s3
2
10
s
s2
3
4
5
1
s1
9
Example
s4
8
Maintain an upper bound d(v) on the shortest path
to v
8
8
15
s3
2
10
s
s2
3
4
8
0
5
1
s1
8
10
s4
8
Maintain an upper bound d(v) on the shortest path
to v
8
8
15
s3
2
A node is either scanned (in S) or labeled (in Q)
10
s
s2
3
4
8
5
0
1
s1
Initially S ? and Q V
8
11
Initially S ? and Q V
s4
8
Pick a vertex v in Q with minimum d(v) and add it
to S
8
8
15
s3
2
10
s
s2
3
4
8
5
0
1
s1
8
12
Initially S ? and Q V
s4
8
Pick a vertex v in Q with minimum d(v) and add it
to S
8
8
15
s3
2
w
10
s
s2
3
4
15
8
v
5
0
1
s1
8
For every edge (v,w) where w in Q relax(v,w)
13
Relax(v,w) If d(v) w(v,w) lt d(w) then d(w) ?
d(v) w(v,w) p(w) ? v
s4
8
8
8
15
s3
2
w
10
s
s2
3
4
15
8
v
5
0
1
s1
8
For every edge (v,w) where w in Q relax(v,w)
14
S s
s4
8
8
8
15
s3
2
10
Relax(s,s4)
s
s2
3
4
8
5
0
1
s1
8
15
S s
s4
15
8
8
15
s3
2
10
Relax(s,s4)
s
s2
3
4
8
5
0
1
s1
8
16
S s
s4
15
8
8
15
s3
2
10
Relax(s,s4)
s
s2
3
4
Relax(s,s3)
8
5
0
1
s1
8
17
S s
s4
15
10
8
15
s3
2
10
Relax(s,s4)
s
s2
3
4
Relax(s,s3)
8
5
0
1
s1
8
18
S s
s4
15
10
8
15
s3
2
10
Relax(s,s4)
s
s2
3
4
Relax(s,s3)
8
5
0
1
s1
Relax(s,s1)
8
19
S s
s4
15
10
8
15
s3
2
10
Relax(s,s4)
s
s2
3
4
Relax(s,s3)
8
5
0
1
s1
Relax(s,s1)
5
20
S s
s4
15
Pick a vertex v in Q with minimum d(v) and add it
to S
10
8
15
s3
2
10
s
s2
3
4
8
5
0
1
s1
5
21
S s,s1
s4
15
Pick a vertex v in Q with minimum d(v) and add it
to S
10
8
15
s3
2
10
s
s2
3
4
8
5
0
1
s1
5
22
S s,s1
s4
15
Pick a vertex v in Q with minimum d(v) and add it
to S
10
8
15
s3
2
10
Relax(s1,s3)
s
s2
3
4
8
5
0
1
s1
5
23
S s,s1
s4
15
Pick a vertex v in Q with minimum d(v) and add it
to S
9
8
15
s3
2
10
Relax(s1,s3)
s
s2
3
4
8
5
0
1
s1
5
24
S s,s1
s4
15
Pick a vertex v in Q with minimum d(v) and add it
to S
9
8
15
s3
2
10
Relax(s1,s3)
s
s2
3
4
Relax(s1,s2)
8
5
0
1
s1
5
25
S s,s1
s4
15
Pick a vertex v in Q with minimum d(v) and add it
to S
9
8
15
s3
2
10
Relax(s1,s3)
s
s2
3
4
Relax(s1,s2)
6
5
0
1
s1
5
26
S s,s1
s4
15
Pick a vertex v in Q with minimum d(v) and add it
to S
9
8
15
s3
2
10
s
s2
3
4
6
5
0
1
s1
5
27
S s,s1,s2
s4
15
Pick a vertex v in Q with minimum d(v) and add it
to S
9
8
15
s3
2
10
s
s2
3
4
6
5
0
1
s1
5
28
S s,s1,s2
s4
15
Pick a vertex v in Q with minimum d(v) and add it
to S
9
8
15
s3
2
10
Relax(s2,s3)
s
s2
3
4
6
5
0
1
s1
5
29
S s,s1,s2
s4
15
Pick a vertex v in Q with minimum d(v) and add it
to S
8
8
15
s3
2
10
Relax(s2,s3)
s
s2
3
4
6
5
0
1
s1
5
30
S s,s1,s2
s4
15
Pick a vertex v in Q with minimum d(v) and add it
to S
8
8
15
s3
2
10
s
s2
3
4
6
5
0
1
s1
5
31
S s,s1,s2,s3
s4
15
Pick a vertex v in Q with minimum d(v) and add it
to S
8
8
15
s3
2
10
s
s2
3
4
6
5
0
1
s1
5
32
S s,s1,s2,s3,s4
s4
15
Pick a vertex v in Q with minimum d(v) and add it
to S
8
8
15
s3
2
10
s
s2
3
4
6
5
0
1
s1
5
33
S s,s1,s2,s3,s4
s4
15
When Q ? then the d() values are the distances
from s
8
8
15
s3
2
10
The p function gives the shortest path tree
s
s2
3
4
6
5
0
1
s1
5
34
S s,s1,s2,s3,s4
s4
15
When Q ? then the d() values are the distances
from s
8
8
15
s3
2
10
The p function gives the shortest path tree
s
s2
3
4
6
5
0
1
s1
5
35
Implementation of Dijkstras algorithm

• We need to find efficiently the vertex with
  minimum d() in Q
• We need to update d() values of vertices in Q

36
Required ADT

• Maintain items with keys subject to
• Insert(x,Q)
• min(Q)
• Deletemin(Q)
• Decrease-key(x,Q,?)

37
Required ADT

• Insert(x,Q)
• min(Q)
• Deletemin(Q)
• Decrease-key(x,Q,?) Can simulate by
• Delete(x,Q), insert(x-?,Q)

38
How many times we do these operations ?

• Insert(x,Q)
• min(Q)
• Deletemin(Q)
• Decrease-key(x,Q,?) Can simulate by
• Delete(x,Q), insert(x-?,Q)

n V
n
n
m E
39
Do we know an algorithm for this ADT ?

• Insert(x,Q)
• min(Q)
• Deletemin(Q)
• Decrease-key(x,Q,?) Can simulate by
• Delete(x,Q), insert(x-?,Q)

Yes! Use binary search trees, we had insert and
delete and also min in the dictionary data type
40
Heap

• Heap is
• A complete binary tree
• The items at the children of v are greater than
  the one at v

7
23
17
19
26
29
24
33
65
79
40
Heap-ordered tree
41
Array Representation

• Representing a heap with an array
• Get the elements from top to bottom, from left to
  right

7
23
17
19
26
29
24
33
65
79
40
7
23
17
24
29
26
19
65
33
40
79
Q
42
Array Representation

• Left(i) 2i1
• Right(i) 2i2
• Parent(i) ?(i-1)/2?

7
23
17
19
26
29
24
33
65
79
40
7
23
17
24
29
26
19
65
33
40
79
Q
43
Find the minimum

• Return Q0

7
23
17
19
26
29
24
33
65
79
40
7
23
17
24
29
26
19
65
33
40
79
Q
44
Delete the minimum
7
23
17
19
26
29
24
33
65
79
40
7
23
17
24
29
26
19
65
33
40
79
Q
45
Delete the minimum
23
17
19
26
29
24
33
65
79
40
23
17
24
29
26
19
65
33
40
79
Q
46
Delete the minimum
79
23
17
19
26
29
24
33
65
40
79
23
17
24
29
26
19
65
33
40
Q
47
Delete the minimum
17
23
79
19
26
29
24
33
65
40
17
23
79
24
29
26
19
65
33
40
Q
48
Delete the minimum
17
23
19
79
26
29
24
33
65
40
17
23
19
24
29
26
79
65
33
40
Q
49
Delete the minimum
Q0 ? Qsize(Q)-1
size(Q) ? size(Q)-1
Heapify-down(Q,0)
17
23
19
79
26
29
24
33
65
40
17
23
19
24
29
26
79
65
33
40
Q
50
Heapify-down(Q,i)

• Heapify-down(Q, i)
• l ? left(i)
• r ? right(i)
• smallest ? i
• if l lt size(Q) and Ql lt Qsmallest
• then smallest ? l
• if r lt size(Q) and Qr lt Qsmallest
• then smallest ? r
• if smallest gt i then Qi ? Qsmallest
• Heapify-down(Q,
  smallest)

51
Inserting an item
Insert(15,Q)
17
23
19
79
26
29
24
33
65
40
17
23
19
24
29
26
79
65
33
40
Q
52
Inserting an item
Insert(15,Q)
Qsize(Q) ? 15
Heapify-up(Q,size(Q))
size(Q) ? size(Q) 1
17
23
19
79
26
29
24
33
65
40
15
17
23
19
24
29
26
79
65
33
40
15
Q
53
Inserting an item
17
23
19
79
26
15
24
33
65
40
29
17
23
19
24
15
26
79
65
33
40
29
Q
54
Inserting an item
17
15
19
79
26
23
24
33
65
40
29
17
15
19
24
23
26
79
65
33
40
29
Q
55
Inserting an item
15
17
19
79
26
23
24
33
65
40
29
15
17
19
24
23
26
79
65
33
40
29
Q
56
Heapify-up

• Heapify-up(Q, i)
• while i gt 0 and Qi lt Qparent(i)
• do Qi ? Qparent(i)
• i ? parent(i)

15
17
19
79
26
23
24
33
65
40
29
15
17
19
24
23
26
79
65
33
40
29
Q
57
Other operations

• Decrease-key(x,Q,?)
• Delete(x,Q)
• Can implement them easily using heapify

58
Heapsort (Williams, Floyd, 1964)

• Put the elements in an array
• Make the array into a heap
• Do a deletemin and put the deleted element at the
  last position of the array

59
Put the elements in the heap
79
65
26
29
15
19
24
33
23
7
40
79
65
26
24
19
15
29
23
33
40
7
Q
60
Make the elements into a heap
79
65
26
29
15
19
24
33
23
7
40
79
65
26
24
19
15
29
23
33
40
7
Q
61
Make the elements into a heap
Heapify-down(Q,4)
79
65
26
29
15
19
24
33
23
7
40
79
65
26
24
19
15
29
23
33
40
7
Q
62
Heapify-down(Q,4)
79
65
26
29
15
7
24
33
23
19
40
79
65
26
24
7
15
29
23
33
40
19
Q
63
Heapify-down(Q,3)
79
65
26
29
15
7
24
33
23
19
40
79
65
26
24
7
15
29
23
33
40
19
Q
64
Heapify-down(Q,3)
79
65
26
29
15
7
23
33
24
19
40
79
65
26
23
7
15
29
24
33
40
19
Q
65
Heapify-down(Q,2)
79
65
26
29
15
7
23
33
24
19
40
79
65
26
23
7
15
29
24
33
40
19
Q
66
Heapify-down(Q,2)
79
65
15
29
26
7
23
33
24
19
40
79
65
15
23
7
26
29
24
33
40
19
Q
67
Heapify-down(Q,1)
79
65
15
29
26
7
23
33
24
19
40
79
65
15
23
7
26
29
24
33
40
19
Q
68
Heapify-down(Q,1)
79
7
15
29
26
65
23
33
24
19
40
79
7
15
23
65
26
29
24
33
40
19
Q
69
Heapify-down(Q,1)
79
7
15
29
26
19
23
33
24
65
40
79
7
15
23
19
26
29
24
33
40
65
Q
70
Heapify-down(Q,0)
79
7
15
29
26
19
23
33
24
65
40
79
7
15
23
19
26
29
24
33
40
65
Q
71
Heapify-down(Q,0)
7
79
15
29
26
19
23
33
24
65
40
7
79
15
23
19
26
29
24
33
40
65
Q
72
Heapify-down(Q,0)
7
19
15
29
26
79
23
33
24
65
40
7
19
15
23
79
26
29
24
33
40
65
Q
73
Heapify-down(Q,0)
7
19
15
29
26
40
23
33
24
65
79
7
19
15
23
40
26
29
24
33
79
65
Q
74
How much time does it take to build the heap this
way ?
7
23
15
26
19
29
24
33
79
65
40
We have at most n/2 nodes heapified at height 1
We have at most n/4 nodes heapified at height 2
We have at most n/8 nodes heapified at height 3
75
Summary

• We can build the heap in linear time
• We still have to deletemin the elements one by
  one in order to sort that will take O(nlog(n))