2IL05 Data Structures 2IL06 Introduction to Algorithms

1
2IL05 Data Structures 2IL06 Introduction to
Algorithms

• Spring 2009Lecture 3 Heaps

2
Solving recurrences
one more time
3
Solving recurrences

• Easiest Master theoremcaveat not always
  applicable
• Alternatively Guess the solution and use the
  substitution method to prove that your guess it
  is correct.
• How to guess
• expand the recursion
• draw a recursion tree

4
Example

• Example(A)
• ? A is an array of length n
• n ? lengthA
• if n1
• then return A1
• else begin
• Copy A1 n/2 to auxiliary
  array B1... n/2
• Copy A1 n/2 to auxiliary
  array C1 n/2
• b ? Example(B) c ? Example(C)
• for i ? 1 to n
• do for j ? 1 to i
• do Ai ?
  Aj
• return 43
• end

5
Example

• Let T(n) be the worst case running time of
  Example on an array of length n.
• Lines 1,2,3,4,11, and 12 take T(1) time.
• Lines 5 and 6 take T(n) time.
• Line 7 takes T(1) 2 T( n/2 ) time.
• Lines 8 until 10 take
• time.
• If n1 lines 1,2,3 are executed,
• else lines 1,2, and 4 until 12 are executed.
• ? T(n)
• ? use master theorem

• Example(A)
• ? A is an array of length n
• n ? lengthA
• if n1
• then return A1
• else begin
• Copy A1 n/2 to auxiliary
  array B1... n/2
• Copy A1 n/2 to auxiliary
  array C1 n/2
• b ? Example(B) c ? Example(C)
• for i ? 1 to n
• do for j ? 1 to i
• do Ai ?
  Aj
• return 43
• end

6
Quiz

• Recurrence
• T(n) 4 T(n/2) T(n3)
• T(n) 4 T(n/2) T(n)
• T(n) T(n/2) 1
• T(n) T(n/3) T(2n/3) n
• T(n) 9 T(n/3) T(n2)
• T(n) vn T(vn) n

• Master theorem?
• yes
• yes
• yes
• no
• yes
• no

T(n) T(n3) T(n) T(n2) T(n) T(log
n) T(n) T(n log n) T(n) T(n2 log n) T(n)
T(n log log n)
7
Tips

• Analysis of recursive algorithmsfind the
  recursion and solve with master theorem if
  possible
• Analysis of loops summations
• Some standard recurrences and sums
• T(n) 2T(n/2) T(n) ?
• ½ n(n1) T(n2)
• T(n3)

T(n) T(n log n)
8
Heaps
9
Event-driven simulation

• Stores a set of events, processes first event
  (highest priority)
• Supporting data structure
• insert event
• find (and extract) event with highest priority
• change the priority of an event

10
Priority queue

• Max-priority queueabstract data type (ADT) that
  stores a set S of elements, each with an
  associated key (integer value).
• OperationsInsert(S, x) inserts element x into
  S, that is, S ? S ? xMaximum(S) returns
  the element of S with the largest
  keyExtract-Max(S) removes and returns the
  element of S with the largest
  keyIncrease-Key(S, x, k) give keyx the value
  k
• condition k is larger than the
  current value of keyx
• Min-priority queue

11
Implementing a priority queue
T(1)
T(1)
T(n)
T(n)
T(1)
T(n)
T(n)
T(n)

• Today

T(1)
T(log n)
T(log n)
T(log n)
12
Max-heap

• Heapnearly complete binary tree, filled on all
  levels except possibly the lowest.(lowest level
  is filled from left to right)
• Max-heap property for every node i other than
  the root keyParent(i) keyi

13
Properties of a max-heap

• LemmaThe largest element in a max-heap is stored
  at the root.
• Proof

14
Properties of a max-heap

• LemmaThe largest element in a max-heap is stored
  at the root.
• Proof x root
• y arbitrary node
• z1, z2, , zk nodes on path between x and
  y
• Invariant ? keyx keyz1 keyzk
  keyy
• ? the largest element is stored at x

x
z1
z2
y
15
Implementing a heap with an array
35
19
30
11
12
8
30
2
17
5
array A1 length(A)
35
30
19
30
8
12
11
17
2
5
1
2
3
4
heap-sizeA

• lengthA length of array A
• heap-sizeA number of elements in the heap

16
Implementing a heap with an array
level 0
level 1
level 2, position 3
array A1 length(A)
1
2
3
4
heap-sizeA

• kth node on level j is stored at position
• left child of node at position i Left(i)
• right child of node at position i Right(i)
• parent of node at position i Parent(i)

A2j-1k
2i
2i 1
i / 2
17
Priority queue

• Max-priority queueabstract data type (ADT) that
  stores a set S of elements, each with an
  associated key (integer value).
• OperationsInsert(S, x) inserts element x into
  S, that is, S ? S ? xMaximum(S) returns
  the element of S with the largest
  keyExtract-Max(S) removes and returns the
  element of S with the largest
  keyIncrease-Key(S, x, k) give keyx the value
  k
• condition k is larger than the
  current value of keyx

18
Implementing a max-priority queue

• Set S is stored as a heap in an array A.
• Operations Maximum, Extract-Max, Insert,
  Increase-Key.

19
Implementing a max-priority queue

• Set S is stored as a heap in an array A.
• Operations Maximum, Extract-Max, Insert,
  Increase-Key.
• Maximum(A)
• if heap-sizeA lt 1
• then error
• else return A1
• running time O(1)

20
Implementing a max-priority queue

• Set S is stored as a heap in an array A.
• Operations Maximum, Extract-Max, Insert,
  Increase-Key.

21
Implementing a max-priority queue

• Set S is stored as a heap in an array A.
• Operations Maximum, Extract-Max, Insert,
  Increase-Key.

35
30
19
30
17
11
12
8
30
2
17
5
22
Implementing a max-priority queue

• Set S is stored as a heap in an array A.
• Operations Maximum, Extract-Max, Insert,
  Increase-Key.
• Heap-Extract-Max(A)
• if heap-sizeA lt 1
• then error
• max ? A1
• A1 ? A heap-sizeA
• heap-sizeA ? heap-sizeA1
• Max-Heapify(A,1)
• return max

35
5
19
30
30
11
12
8
2
17
5
restore heap property
23
Max-Heapify

• Max-Heapify(A, i)
• ? ensures that the heap whose root is stored at
  position i has the max-heap property
• ? assumes that the binary trees rooted at Left(i)
  and Right(i) are max-heaps

24
Max-Heapify

• Max-Heapify(A, i)
• ? ensures that the heap whose root is stored at
  position i has the max-heap property
• ? assumes that the binary trees rooted at Left(i)
  and Right(i) are max-heaps
• Max-Heapify(A,1)
• exchange A1 with largest child
• Max-Heapify(A,2)
• exchange A2 with largest child
• Max-Heapify(A,5)
• A5 larger than its children ? done.

40
10
10
35
19
40
10
30
11
12
35
2
17
5
25
Max-Heapify

• Max-Heapify(A, i)
• ? ensures that the heap whose root is stored at
  position i has themax-heap property
• ? assumes that the binary trees rooted at
  Left(i) and Right(i) are max-heaps
• if Left(i) heap-sizeA and ALeft(i) gt Ai
• then largest ? Left(i)
• else largest ? i
• if Right(i) heap-sizeA and ARight(i) gt
  Alargest
• then largest ? Right(i)
• if largest ? i
• then exchange Ai and Alargest
• Max-Heapify(A, largest)
• running time?

O(height of the subtree rooted at i) O(log n)
26
Implementing a max-priority queue

• Set S is stored as a heap in an array A.
• Operations Maximum, Extract-Max, Insert,
  Increase-Key.
• Insert (A, key)
• heap-sizeA ? heap-sizeA 1
• Aheap-sizeA ? - infinity
• Increase-Key(A, heap-sizeA, key)

27
Implementing a max-priority queue

• Set S is stored as a heap in an array A.
• Operations Maximum, Extract-Max, Insert,
  Increase-Key.

28
Implementing a max-priority queue

• Set S is stored as a heap in an array A.
• Operations Maximum, Extract-Max, Insert,
  Increase-Key.
• Increase-Key(A, i, key)
• if key lt Ai
• then error
• Ai ? key
• while igt1 and Aparent(i) lt Ai
• do exchange Aparent(i) and Ai
• i ? parent(i)
• running time O(log n)

35
19
30
30
11
12
8
42
2
17
5
29
Building a heap

• Build-Max-Heap(A)
• ? Input array A1..n of numbers
• ? Output array A1..n with the same numbers,
  but rearranged, such that the max-heap
  property holds
• heap-size ? lengthA
• for i ? lengthA downto 1
• do Max-Heapify(A,i)
• P(k) nodes k, k1, , n are each the root of a
  max-heap
• Loop InvariantP(k1) holds before line 3 is
  executed with ik, P(k) holds afterwards

30
Building a heap

• Build-Max-Heap(A)
• heap-size ? lengthA
• for i ? lengthA downto 1
• do Max-Heapify(A,i)
• ?i O(1 height of i)
• ?0 h log n ( nodes of height h) O(1h)
• ?0 h log n (n / 2h1) O(1h)
• O(n) ?0 h log n h / 2h1
• O(n)

O(height of node i)
height of node i edges longest simple
downward path from i to a leaf.
31
The sorting problem

• Input a sequence of n numbers a1, a2, , an
• Output a permutation of the input such that ai1
  ain
• Important properties of sorting algorithms
• running time how fast is the algorithm in the
  worst case
• in place only a constant number of input
  elements are ever stored outside the
  input array

32
Sorting with a heap Heapsort

• HeapSort(A)
• Build-Max-Heap(A)
• for i ? lengthA downto 2
• do exchange A1 and Ai
• heap-sizeA ? heap-sizeA 1
• Max-Heapify(A,1)
• P(k) Ak1 .. n is sorted and contains the n-k
  largest elements, A1..k is a max-heap on the
  remaining elements
• Loop InvariantP(k) holds before lines 3-5 are
  executed with ik, P(k-1) holds afterwards

33
Sorting algorithms
T(n2)
yes
T(n log n)
no
yes
T(n log n)
34
Tutorials this week

• Small tutorials on Tuesday 34.
• Wednesday 78 big tutorial.
• Small tutorial Friday 78.