HEAPS

1
HEAPS PRIORITY QUEUES

• Array and Tree implementations

2
Priority queue

• A stack is first in, last out
• A queue is first in, first out
• A priority queue is least-first-out
• The smallest element is the first one removed

3

• The definition of smallest is up to the
  programmer (for example,
• you might define it by implementing Comparator or
  Comparable)

4
Priority queue

• If there are several smallest elements,
• the implementer must decide which to remove first
• Remove any smallest element (dont care which)
• Remove the first one added

5

• A priority queue is an
• ADT where items are removed based on their
  priority
• (highest-to-lowest), regardless of the order they
  ARE inserted in the structure

6
A priority queue ADT

• PriorityQueue() Methods
• void add(Comparable o) inserts o into the
  priority queue
• Comparable removeLeast() removes and returns the
  least element
• Comparable getLeast() returns (but does not
  remove) the least element
• boolean isEmpty() returns true iff empty
• int size() returns the number of elements
• void clear() discards all elements

7
Evaluating implementations

• When we choose a data structure,
• it is important to look at usage patterns
• If we load an array once and do thousands of
  searches on it,
• we want to make searching fast
• so we would probably sort the array

8

• When we choose a data structure,
• it is important to look at usage patterns
• If we load a huge array and expect to do only a
  few searches,
• we probably dont want to spend time sorting the
  array

9
Evaluating implementations

• For almost all uses of a queue (including a
  priority queue),
• we eventually remove everything that we add

10
TIMING

• Hence,
• when we analyze a priority queue,
• neither add nor remove is more important
• we need to look at the timing for (add remove)

11
Array implementations

• A priority queue could be implemented as an
  unsorted array (with a count of elements)
• Adding an element would take O(1) time (why?)
• Removing an element would take O(n) time (why?)
• Hence, (adding and removing) an element takes
  O(n) time
• This is an inefficient representation a BigO(N)

12
Array implementations

• A priority queue could be implemented as a
  sorted array (again, with a counter of
  elements)
• Adding an element would take O(n) time (why?)
• Removing an element would take O(1) time (why?)
• So adding and removing) an element takes
• O(n) time
• Again, this is very inefficient

13
Linked list implementations

• A priority queue could be implemented as an
• unsorted linked list
• Adding an element would take O(1) time (why?)
• Removing an element would take (N) time (why?)
• Again, inefficient!

14
SORTED LINKED LIST

• A priority queue could be implemented as a
• sorted linked list
• Adding an element would take O(n) time (why?)
• Removing an element would take O(1) time (why?)
• Again, an inefficient algorithm

15
Binary tree implementations

• A priority queue could be represented as a
• balanced binary search tree
• Insertion and removal could destroy the balance
• We would need an algorithm to rebalance the
  binary tree
• Good rebalancing algorithms require O(log n)
  time, but are complicated

16
Heap Implementaton

• The concepts of heaps is used to refer to memory
  in a computer,
• E.g. the part of the memory that provides dynamic
  memory allocation ( get it as you need it)
• Heap is also used as a name for a special kind
  of binary tree

17
Heap is a complete binary tree

• A heap is a complete binary tree
• with values stored in its nodes such
  that
• no child has a value bigger than the
  value of its parent
• (though it can be equal)

18

• Therefore we would like to find a better
  algorithm. We want to
• Insert and Remove with a Big O of one

19
HEAPS Priority Queues

• A binary tree represented as a heap facilitates
  the operation
• to find the maximum element
• it is always in the root of the tree
• Because finding the node with the maximum value
  in a heap is a
  BigO(I)
• A HEAP is the most efficient structure for
  implementing
  priority queues

20
Heaps

• A heap is a certain kind of complete binary tree.

Root
When a complete binary tree is built, its first
node must be the root.
21
Heaps

• Complete binary tree.

Right child
Left child
The Second and third Nodes are always the left
and right child of the root.
22
Heaps

• Complete binary tree.

The next nodes always fill the next level from
left-to-right.
23
Heaps

• Complete binary tree.

24
Heaps
45

• A heap is a certain kind of complete binary tree.

23
35
4
22
21
27
19
Each node in a heap contains a key that can be
compared to other nodes' keys.
25
To implement a heap as a priority queue

• A node has the heap property if it is
  Equal to or greater than
  as its children
• OR
• if it is smaller than or equal to its children
  (since smaller numbers represent higher
  priorities)

26
Heaps

• A heap is a certain kind of complete binary tree.
  
• This example is a max heap.

45
23
35
4
22
21
27
19
The "heap property" requires that each node's key
is gt to the keys of its children
27
Priority Queues

• A heap can easily implement a priority queue
• but what happens with we remove the element of
  highest priority?
• How do we re-arrange the tree?

28
Removing from a heap

• There are two things we need to consider when
  rearranging
• We want to end up with a heap which means that
• The tree has to be complete
• ALL HEAPS ARE COMPLETE TREES!

29
To remove an element at top of tree

• Remove the element at location 0
• Move the element in the lastIndex to location 0,
  decrement
  lastIndex
• (lastIndex is at the end of the heap)
• Reheap the new root node (the one now at location
  0)
• This is called down-heap bubbling or
  percolating down
• Down-heap bubbling requires O(log n) time
• to remove an element

30
Removing the Top of a Heap(max heap)

• Move the last node onto the root.

45
23
42
4
22
21
35
19
27
31
Removing the Top of a Heap(max heap)
27

• Move the value in the last node into the root.

23
42
4
22
21
35
19
32
Removing the Top of a Heap
27

• Move the last node onto the root.
• Push the out-of-place node downward,
• swapping with its larger child
• until the new node reaches an acceptable location.

23
42
4
22
21
35
19
33
Removing the Top of a Heap
42

• Move the last node onto the root.
• Push the out-of-place node downward, swapping
  with its larger child until the new node reaches
  an acceptable location.

23
27
4
22
21
35
19
34
Removing the Top of a Heap

• Move the last node onto the root.
• Push the out-of-place node downward, swapping
  with its larger child until the new node reaches
  an acceptable location.
• Note that we discard half of the tree with each
  move downward

42
23
35
4
22
21
27
19
35
reheapification downward.
42

• We stop when
• The children all have keys lt to the out-of-place
  node, or
• The node reaches the leaf.
• The process of pushing the node downward is
  called reheapification
  downward.

23
35
4
22
21
27
19
36
Priority Queues

• Another Example
• How to delete a node from a Priority Queue
• Since it is queue - first in first out,
• we remove the first one
• The Root

37
An Example of a MAX HeapThe root has largest
value
38
Deleting a node from the heap

• We use a heap to implement a priority queue
• the "next" element ( the root ) in the queue was
  removed (pop operation).
• We end up with a root with no value

Root has no value
39
Deleting a node from the heap
The obvious question is which node can we use to
replace this one?
16
12
7
9
13
5
2
3
1
40
Deleting a node from the heap
16
12
7
9
13
5
If we want this tree to stay complete the
rightmost element in the bottommost level is the
obvious choice
2
3
1
41
Deleting a node from the heap
We now have a problem We have a complete tree
but this is not a heap!   WHY???   This can be
solved by applying systematic swap of nodes.
2
16
12
7
9
13
5
3
1
42
Deleting a node from the heap
2
16
12
7
9
13
5
We systematically swap the root with the
larger of the children nodes until no more swaps
can be done
3
1
43
Deleting a node from the heap
16
2
12
7
9
13
5
3
1
44
Deleting a node from the heap
16
13
12
7
9
2
5
3
1
45
Deleting a node from the heap
16
13
12
7
9
2
5
3
1
The tree has restored its heap property!
46
Insert operation

• Another example - the heap below
• Recall that while delete causes pairwise swaps
  from the root to the bottom,
• insert causes pairwise swaps from the bottom to
  the top.
• Suppose that we want to insert an element with
  value 20

22
17
12
11
3
13
5
1
4
6
47
insert (20)
22
17
12
11
3
13
5
1
4
20
6
48
insert (20)
22
17
12
11
20
13
5
1
4
3
6
49
insert (20)
22
20
17
12
11
13
5
1
4
3
6
50
insert (20)
22
17
20
11
12
13
5
1
4
3
6
51
Implementing a Heap

• We will store the data from the nodes in a
  partially-filled array.

42
23
35
21
27
An array of data
52
Implementing a Heap

• Data from the root goes in the first
  location of the
  array.

42
23
35
21
27
42
An array of data
53
Implementing a Heap

• Data from the next row goes in the next two array
  locations.

42
23
35
21
27
42
35
23
An array of data
54
Implementing a Heap

• Data from the next row goes in the next two array
  locations.

42
23
35
21
27
42
35
23
27
21
An array of data
55
Implementing a Heap

• Data from the next row goes in the next two array
  locations.

42
23
35
21
27
42
35
23
27
21
An array of data
56
Points about the Implementation

• The links between the tree's nodes are not
  actually stored as pointers, or in any other way.
• The only way we "know" that "the array is a tree"
  is from the way we manipulate the data.

42
23
35
21
27
42
35
23
27
21
An array of data
57
Array representation of a heap
lastIndex location 5

• Left child of node i is 2i 1, right child is
  2i 2
• Unless the value is larger than lastIndex
  -therefore no such child
• Parent of node i is (i 1)/2 unless i 0

58
Adding to heap - - Implementation

• algorithm for the array implementation
• Increase lastIndex and put the new value where
  lastIndex now references --
• that is add to the end of the array
• Reheap the newly added node
• This is called up-heap bubbling or percolating up
• Up-heap bubbling requires O(log n) time

59
Points about the Implementation
42

• If you know the index of a node, then it is easy
  to figure out the
• indexes of that node's parent and children.

23
35
21
27
42
35
23
27
21
0 1 2 3 4
60
Points about the Implementation
42

• If we add 45 to the tree and to the array,
  45s parent is at what index? i ?
• (i-1)/2 2

23
35
21
27
45
45
42
35
23
27
21
0 1 2 3 4 5
61
Points about the Implementation

• If we add 45 to the tree and the array, 45s
  parent is at what index?
• (i-1)/2 which is 2.
• We do a comparison of 45 with its parent and
• swap if it is greater than the parent - it is
  greater than it

42
23
35
21
27
45
45
42
35
23
27
21
0 1 2 3 4
5
62
Points about the Implementation

• If we add 45 to the tree and to the tree and the
  array,
• 45s parent is at what index?
• (i-1)/2 which is 2.
• We do a comparison and swap.
• It is still out of place.

42
45
35
21
27
23
42
35
45
27
21
23
0 1 2 3 4
5
63
Points about the Implementation
45

• 45 is still out of place,
• so we do another comparison
• and swap with 45s parent which is 42.
• So what is the terminating condition)(s)?

42
35
21
27
23
23
45
35
42
27
21
0 1 2 3 4
5
64
Summary

• A heap is a complete binary tree, where
• the entry at each node is greater than or equal
  to the entries in its children.
• OR
• the entry at each node is less than or equal to
  the entries in its children.

65

• Summary Adding/Deleting
• To add an entry to a heap,
• place the new entry at the next available spot,
• and perform a reheapification upward.
• To remove the biggest entry,
• move the value in the last node onto the root,
  and
• perform a reheapification downward.

66
Reheapifying a Tree

• Fine,
• we know how to delete one element and
• restore the heap,
• but there are several other operations involving
  heaps

67

• If we find a systematic way of heapifying a tree,
  
• we could perform any operation
• because we can execute the procedure heapify

68
Reheapifying a Tree

• we can heapify a tree using a quite simple idea.
• Given a binary tree T (not necessarily a heap)
  with root R and left subtree Tleft and Tright,
• if these two subtrees are heaps,
• we can make T a heap using the process described
  in the previous slide (of applying systematic
  swaps)
• We start the process of heapifying a tree from
  the lower levels

69
Algorithm for Reheapifying Tree

• The algorithm aims at heapifying a tree by
  always comparing three nodes at a time.
• To start from the lower levels
• we choose to look at the internal nodes in
• reverse level order
• External nodes have no children so they are
  (individually) heaps

70
HEAPIFY

• void heapify (Heap h)
• NodeType n
• for (n the internal nodes of h in reverse
  level-order)
• Reheapify the heap h starting at node n

71
Let change this tree into a heap - A MAX
HEAP
6
17
4
12
1
13
5
3
22
11
72
An example
6
17
4
12
1
13
5
3
22
11
Internal Nodes in Reverse Level-Order are 1,
12, 17, 4, 6 , start with the leftmost leaf
compare 3 To 1
73
6
17
4
12
3
13
5
1
22
11
3 and 1 have
swapped Internal Nodes in Reverse Level-Order 3,
12, 17, 4, 6
74
An example
6
17
4
3
13
5
12
1
11
22

• Compare 12 with its children and swap 12 and
  22
• Internal Nodes in Reverse Level-Order 3, 22, 17,
  4, 6

75
An example
6
17
4
22
3
13
5
1
12
11
Compare 17 with its children and no changes
necessary 3, 22, 17, 4, 6 ---
NEXT COMPARE 4 AND 22
76
6
17
22
4
3
13
5
1
12
11
Compare 22 with 4 and swap 4 IS NOW OUT
OF PLACE
77
An example
6
17
22
12
3
13
5
1
4
11
Compare 12 and 4 and swap, NEXT compare 22 to 6
3, 12, 17, 22, 6
78
22
17
6
12
3
13
5
1
4
11
Heapify 6 downwards until it finds its proper
location 3, 12, 17,
6
79
22
17
12
6
3
13
5
1
4
11
Reheap down 6 Compare 6 and 11
80
An example
6 is in place and the tree is
reheaped
22
17
12
11
3
13
5
1
4
6
81
An example
22
17
12
11
3
13
5
1
4
6
82
heapsort

• Heapsort is another algorithm for sorting lists
• The idea behind heap sort is that
• to sort a list
• we can insert the elements of a list into a heap
  using heapify
• Then remove all the elements using remove.

83

• The idea behind heap sort is that
• to sort a list we can insert the elements of a
  list into a heap using heapify
• Then remove all the elements using remove.
• It is guaranteed that the largest element is
  removed first,
• So by systematically removing the elements
• we get the list ordered ( in reverse order)