Priority Queues, Heaps

1
Priority Queues, Heaps Leftist Trees

• CSE, POSTECH

2
Priority Queues

• A priority queue is a collection of zero or more
  elements ? each element has a priority or value
• Unlike the FIFO queues, the order of deletion
  from a priority queue (e.g., who gets served
  next) is determined by the element priority
• Elements are deleted by increasing or decreasing
  order of priority rather than by the order in
  which they arrived in the queue

3
Priority Queues

• Operations performed on priority queues
• 1) Find an element, 2) insert a new element, 3)
  delete an element, etc.
• Two kinds of (Min, Max) priority queues exist
• In a Min priority queue, find/delete operation
  finds/deletes the element with minimum priority
• In a Max priority queue, find/delete operation
  finds/deletes the element with maximum priority
• Two or more elements can have the same priority

4
Priority Queues

• See ADT 12.1 Program 12.1 for max priority
  queue specification
• What would be different for min priority queue
  specification?
• Read Examples 12.1, 12.2
• What are other examples in our daily lives that
  utilize the priority queue concept?

5
Implementation of Priority Queues

• Implemented using heaps and leftist trees
• Heap is a complete binary tree that is
  efficiently stored using the array-based
  representation
• Leftist tree is a linked data structure suitable
  for the implementation of a priority queue

6
Max (Min) Tree

• A max tree (min tree) is a tree in which the
  value in each node is greater (less) than or
  equal to those in its children (if any)
• See Figure 12.1, 12.2 for examples
• Nodes of a max or min tree may have more than two
  children (i.e., may not be binary tree)

7
Max Tree Example
8
Min Tree Example
9
Heaps - Definitions

• A max heap (min heap) is a max (min) tree that is
  also a complete binary tree
• Figure 12.1 (a) (c) are max heap
• Figure 12.2 (a) (c) are min heap
• Why arent Figure 12.1 (b) 12.2 (b) max/min
  heap?
• How can you change the Figure 12.1 (b) 12.2 (b)
  so that they are max/min heap?

10
Max Heap with 9 Nodes
11
Min Heap with 9 Nodes
12
Array Representation of Heap

• A heap is efficiently represented as an array.

13
Heap Operations

• When n is the number of elements (heap size),
• Insertion ? O(log2n)
• Deletion ? O(log2n)
• Initialization ? O(n)

14
Insertion into a Max Heap

• New element is 5
• Are we finished?

15
Insertion into a Max Heap

• New element is 20
• Are we finished?

16
Insertion into a Max Heap

• Exchange the positions with 7
• Are we finished?

17
Insertion into a Max Heap

• Exchange the positions with 8
• Are we finished?

18
Insertion into a Max Heap

• Exchange the positions with 9
• Are we finished?

19
Complexity of Insertion

• See also Figure 12.3 for another insertion
  example
• At each level, we do ?(1) work
• Thus the time complexity is O(height) O(log2n),
  where n is the heap size

20
Deletion from a Max Heap

• Max element is in the root
• What happens when we
• delete an element?

21
Deletion from a Max Heap

• After the max element is removed.
• Are we finished?

22
Deletion from a Max Heap

• Heap with 10 nodes.
• Reinsert 8 into the heap.

23
Deletion from a Max Heap

• Reinsert 8 into the heap.
• Are we finished?

24
Deletion from a Max Heap

• Exchange the position with 15
• Are we finished?

25
Deletion from a Max Heap

• Exchange the position with 9
• Are we finished?

26
Complexity of Deletion

• See also Figure 12.4 for another deletion example
• The time complexity of deletion is the same as
  insertion
• At each level, we do ?(1) work
• Thus the time complexity is O(height) O(log2n),
  where n is the heap size

27
Max Heap Initialization

• Heap initialization means to construct a heap by
  adjusting the tree if necessary
• Example input array -,1,2,3,4,5,6,7,8,9,10,11
  

28
Max Heap Initialization
- Start at rightmost array position that has a
child. - Index is floor(n/2).
29
Max Heap Initialization
30
Max Heap Initialization
31
Max Heap Initialization
32
Max Heap Initialization
33
Max Heap Initialization

• Are we finished?
• Done!

34
Complexity of Initialization

• See Figure 12.5 for another initialization
  example
• Height of heap h.
• Number of nodes at level j is lt 2j-1.
• Time for each node at level j is O(h-j1).
• Time for all nodes at level j is lt 2j-1(h-j1)
  t(j).
• Total time is t(1) t(2) t(h) O(2h)
  O(n).

35
The Class MaxHeap

• See Program 12.2 for Insertion into a MaxHeap
• See Program 12.3 for Deletion from a MaxHeap
• See Program 12.4 for Initializing a nonempty
  MaxHeap

36
PUSH OPERATION

• templateltclass Tgt
• void maxHeapltTgtpush(const T theElement)
• // Add theElement to heap.
• // increase array length if necessary
• if (heapSize arrayLength - 1)
• // double array length
• changeLength1D(heap, arrayLength, 2
  arrayLength)
• arrayLength 2
• // find place for theElement
• // currentNode starts at new leaf and moves up
  tree
• int currentNode heapSize
• while (currentNode ! 1 heapcurrentNode /
  2 lt theElement)
• // cannot put theElement in
  heapcurrentNode
• heapcurrentNode heapcurrentNode / 2
  // move element down
• currentNode / 2
  // move to parent

37
POP OPERATION

• templateltclass Tgt
• void maxHeapltTgtpop()
• // Remove max element.
• // if heap is empty return null
• if (heapSize 0) // heap empty
• throw queueEmpty()
• // Delete max element
• heap1.T()
• // Remove last element and reheapify
• T lastElement heapheapSize--
• // find place for lastElement starting at root
• int currentNode 1,
• child 2 // child of currentNode
• while (child lt heapSize)
• // heapchild should be larger child of
  currentNode

38
INITIALIZE

• templateltclass Tgt
• void maxHeapltTgtinitialize(T theHeap, int
  theSize)
• // Initialize max heap to element array
  theHeap1theSize.
• delete heap
• heap theHeap
• heapSize theSize
• // heapify
• for (int root heapSize / 2 root gt 1
  root--)
• T rootElement heaproot
• // find place to put rootElement
• int child 2 root // parent of child is
  target
• // location for
  rootElement
• while (child lt heapSize)
• // heapchild should be larger sibling
• if (child lt heapSize heapchild lt
  heapchild 1)

39
Exercise 12.7

• Do Exercise 12.7
• theHeap -, 10, 2, 7, 6, 5, 9, 12, 35, 22, 15,
  1, 3, 4

40
Exercise 12.7 (a)

• 12.7 (a) complete binary tree

41
Exercise 12.7 (b)

• 12.7 (b) The heapified tree

42
Exercise 12.7 (c)

• 12.7 (c) The heap after 15 is inserted is

43
Exercise 12.7 (c)

• 12.7 (c) The heap after 20 is inserted is

44
Exercise 12.7 (c)

• 12.7 (c) The heap after 45 is inserted is

45
Exercise 12.7 (d)

• 12.7 (d) The heap after the first remove max
  operation is

46
Exercise 12.7 (d)

• 12.7 (d) The heap after the second remove max
  operation is

47
Exercise 12.7 (d)

• 12.7 (d) The heap after the third remove max
  operation is

48
Leftist Trees

• Despite heap structure being both space and time
  efficient, it is NOT suitable for all
  applications of priority queues
• Leftist tree structures are useful for
  applications
• to meld (i.e., combine) pairs of priority queues
• using multiple queues of varying size
• Leftist tree is a linked data structure suitable
  for the implementation of a priority queue
• A tree which tends to lean to the left.

49
Leftist Trees

• External node a special node that replaces each
  empty subtree
• Internal node a node with non-empty subtrees
• Extended binary tree a binary tree with
  external nodes added (see Figure 12.6)

50
Extended Binary Tree
51
Height-Biased Leftist Tree (HBLT)

• Let s(x) be the length (height) of a shortest
  path from node x to an external node in its
  subtree
• If x is an external node, s(x) 0
• If x is an internal node, s(x) min s(L), s(R)
  1, where L and R are left and right children of
  x
• A binary tree is a height-biased leftist tree
  (HBLT) iff at every internal node, the s value of
  the left child is greater than or equal to the s
  value of the right child
• Is Figure 12.6(a) an HBLT?
• If not, how can we change it to become an HBLT?

52
Max/Min HBLT

• A max HBLT is an HBLT that is also a max tree
• Are the trees of Figure 12.1 are also max HBLTs?
• YES!
• A min HBLT is an HBLT that is also a min tree
• Are the trees of Figure 12.2 are also min HBLTs?
• YES!

53
Weight-Biased Leftist Tree (WBLT)

• Let the weight, w(x), of node x to be the number
  of internal nodes in the subtree with root x
• If x is an external node, w(x) 0
• If x is an internal node, its weight is one more
  than the sum of the weights of its children
• A binary tree is a weight-biased leftist tree
  (WBLT) iff at every internal node, the w value of
  the left child is greater than or equal to the w
  value of the right child

54
Weight-Biased Leftist Tree (WBLT)

• A max (min) WBLT is a max (min) tree that is also
  a WBLT
• Is Figure 12.6(a) an WBLT?
• If not, how can we change it to become an WBLT?

55
Operations on a Max HBLT

• Read Section 12.5.2 for Insertion into a Max HBLT
• Read Section 12.5.3 for Deletion from a Max HBLT
• Read Section 12.5.4 for Melding Two Max HBLTs
• See Figure 12.7 and read Example 12.3 for Melding
  max HBLTs
• Read Section 12.5.5 for Initialization of a Max
  HBLT
• See Figure 12.8 for Initializing a max HBLT

56
Melding max HBLTs
57
The Class maxHBLT

• See Program 12.5 for Melding of two leftist trees
• See Program 12.6 for meld, push and pop methods
• See Program 12.7 for Initializing a max HBLT
• Do Exercise 12.19

58
Exercise 12.19
(a) The first six calls to meld create the
following six max leftist trees.
The next three calls to meld combine pairs of
these trees to create the following three trees
What would be next?
59
Applications of Heaps

• Sort (heap sort)
• Machine scheduling
• Huffman codes

60
Heap Sort

• use element key as priority
• Algorithm
• put elements to be sorted into a priority queue
  (i.e., initialize a heap)
• extract (delete) elements from the priority queue
• if a min priority queue is used, elements are
  extracted in non-decreasing order of priority
• if a max priority queue is used, elements are
  extracted in non-increasing order of priority

61
Heap Sort Example

• After putting into a max priority queue

62
Sorting Example

• After first remove max operation

63
Sorting Example

• After second remove max operation

64
Sorting Example

• After third remove max operation

65
Sorting Example

• After fourth remove max operation

66
Sorting Example

• After fifth remove max operation

67
Complexity Analysis of Heap Sort

• See Program 12.8 for Heap Sort
• See Figure 12.9 for another Heap Sort example
• Heap sort n elements.
• Initialization operation takes O(n) time
• Each deletion operation takes O(log n) time
• Thus, the total time is O(n log n) - Why?
• ? The heap has to be reinitialized (melded) after
  each delete operation
• compare with O(n2) for sort methods of Chapter 2

68
Machine Scheduling Problem

• m identical machines
• n jobs to be performed
• The machine scheduling problem is to assign jobs
  to machines so that the time at which the last
  job completes is minimum

69
Machine Scheduling Example

• 3 machines and 7 jobs
• job times are 6,2,3,5,10,7,14
• What are some possible schedules?
• A possible schedule
• What algorithm did we use for the above
  scheduling?
• What are other scheduling algorithms

70
Machine Scheduling Example

• What is the finish time (length) of the schedule?
• ? 21
• Objective Find schedules with minimum finish
  time
• Minimum finish time scheduling is NP-hard.

71
NP-hard Problems

• The class of problems for which no one has
  developed a polynomial time algorithm.
• No algorithm whose complexity is O(nk ml) is
  known for any NP-hard problem (for any constants
  k and l)
• NP stands for Nondeterministic Polynomial
• NP-hard problems are often solved by heuristics
  (or approximation algorithms), which do not
  guarantee optimal solutions
• Longest Processing Time (LPT) rule is a good
  heuristic for minimum finish time scheduling.

72
LPT Schedule Example

• Longest Processing Time (LPT) first
• Jobs are scheduled in the descending order14,
  10, 7, 6, 5, 3, 2
• Each job is scheduled on the machineon which it
  finishes earliest

finish time is 16!
73
LPT Schedule Example

• What is the minimum finish time with thee
  machines for jobs (2, 14, 4, 16, 6, 5, 3)?
• See Figure 12.10

74
LPT using a Min Heap

• Min Heap has the finish times of the m machines.
• Initial finish times are all 0.
• To schedule a job, remove the machine with
  minimum finish time from the heap.
• Update the finish time of the selected machine
  and put the machine back into the min heap.
• See Program 12.9 for LPT scheduler

75
Complexity Analysis of LPT

• When n ? m (i.e., more machines than jobs), LPT
  takes ?(1) time
• When n ? m, (i.e., more jobs than machines), the
  heap sort takes O(n log n) time
• Heap initialization takes O(m) time
• DeleteMin operation takes O(log m) time
• Insert operation takes O(log m) time
• n DeleteMin and n Insert takes O(n log m) time
• Thus, the total time is O(n log n n log m)
  O(n log n) time (as n gt m)

76
Huffman Codes

• For text compression, the LZW method relies on
  the recurrence of substrings in a text
• Huffman codes is another text compression method,
  which relies on the relative frequency (i.e., the
  number of occurrences of a symbol) with which
  different symbols appear in a text
• Uses extended binary trees
• Variable-length codes that satisfy the property,
  where no code is a prefix of another
• Huffman tree is a binary tree with minimum
  weighted external path length for a given set of
  frequencies (weights)

77
Huffman Codes

• READ Section 12.6.3
• READ all of Chapter 12