HeapSort

1
HeapSort

• 25 March 2003

2
HeapSort

• Heaps or priority queues provide a means of
  sorting
• Construct a heap,
• Add each item to it (maintaining the heap
  property!),
• When all items have been added, remove them one
  by one (restoring the heap property as each one
  is removed).
• Addition and deletion are both O(logn)
  operations. We need to perform n additions and
  deletions, leading to an O(nlogn) algorithm.

3
Binary Trees

• A binary tree is completely full if it is of
  height, h, and has 2h1-1 nodes.
• A binary tree of height, h, is complete iff
• it is empty or
• its left subtree is complete of height h-1 and
  its right subtree is completely full of height
  h-2 or
• its left subtree is completely full of height h-1
  and its right subtree is complete of height h-1.
• A complete tree is filled from the left
• all the leaves are on the same level or
• two adjacent ones and
• All nodes at the lowest level are as far to the
  left as possible

4
Heaps (Priority Queues)

• A binary tree has the heap property iff
• it is empty or
• the key in the root is larger than that in either
  child and (recursively) both subtrees have the
  heap property.
• The value of the heap structure is that we can
  both extract the highest item and insert a new
  one in O(logn) time.

5
Remove Root
6
Bring Item to Root
To maintain the heap property, use the fact that
a complete tree is filled from the left. So that
the position which must become empty is the one
occupied by the M. Put it in the vacant root
position.
7
Check Heap Property
M at the root violates the condition that the
root must be greater than each of its children.
So interchange the M with the larger of its
children.
8
Recheck Heap Property
The left subtree has now lost the heap property.
So again interchange the M with the larger of its
children. This tree is now a heap again, so
we're finished.
9
Complexity

• We need to make at most h interchanges of a root
  of a subtree with one of its children to fully
  restore the heap property. Thus deletion from a
  heap is O(h) or O(logn).

10
Add Item to Heap
To add an item to a heap, we follow the reverse
procedure. Place it in the next leaf position
and move it up. Again, we require at most O(h)
or O(logn) exchanges.
11
Storing Heaps in an Array

• If we number the nodes from 1 at the root and
  place
• the left child of node k at position 2k
• the right child of node k at position 2k1
• Then the fill from the left nature of the
  complete tree ensures that the heap can be stored
  in consecutive locations in an array.

12
2k and 2k1 Are Children of k
2k
1 2 3 4 5 6 7 8 9
2k1
N 9
13
HeapSort Animation

• http//www.cs.brockport.edu/cs/java/apps/sorters/h
  eapsort.html

14
Empirical Studies
Empirical studies show that generally QuickSort
is considerably faster than HeapSort. The
following counts of compare and exchange
operations were made for three different sorting
algorithms running on the same data
15
QuickSort is Method of Choice

• Thus, when an occasional blowout to O(n2) is
  tolerable, we can expect that, on average,
  QuickSort provides considerably better
  performance - especially if one of the modified
  pivot choice procedures is used.
• Most commercial applications would use QuickSort
  for its better average performance they can
  tolerate an occasional long run in return for
  shorter runs most of the time.

16
Nothing is Guaranteed

• QuickSort should never be used in applications
  which require a guarantee of response time,
  unless it is treated as an O(n2) algorithm in
  calculating the worst-case response time.
• If you have to assume O(n2) time, then - if n is
  small, you're better off using insertion sort -
  which has simpler code and therefore smaller
  constant factors.

17
HeapSort is an Alternative

• If n is large, use heap sort, for it is
  guaranteed O(nlog n) time. Life-critical (medical
  monitoring, life support) and mission-critical
  (monitoring and control in plants handling
  dangerous materials, control for aircraft,
  defense, etc.) software will generally have
  response time as part of the system
  specifications. In such systems, it is not
  acceptable to design based on average
  performance, you must allow for the worst case,
  and thus treat QuickSort as O(n2).

18
Priority Queue

• A priority queue is a collection of zero or more
  elements. Each element has a priority or value.
• Find an element
• Insert a new element
• Remove an element
• min priority queue
• max priority queue

19
Example of a Priority Queue

• Operating system is handing out CPU time slices
  to multiple users of the system
• Certain tasks (especially those involving I/O)
  have to be serviced before the data is lost
• Order requests for service by priority
• Use a priority queue to decide which request
  receives service next

20
Speeding up QuickSort

• The recursive calls in QuickSort are generally
  expensive on most architectures - the overhead of
  any procedure call is significant and reasonable
  improvements can be obtained with equivalent
  iterative algorithms.
• Two things can be done to eke a little more
  performance out of your processor when sorting.
  (See next slide)

21
Fine Tuning

1. QuickSort- in it's usual recursive form - has a
   reasonably high constant factor relative to a
   simpler sort such as insertion sort. Thus, when
   the partitions become small (n lt 10), a switch
   to insertion sort for the small partition will
   usually show a measurable speed-up. (The point at
   which it becomes effective to switch to the
   insertion sort is extremely sensitive to
   architectural features and needs to be determined
   for any target processor although a value of 10
   is a reasonable guess!)
2. Write the whole algorithm in an iterative form.

22
Sorting Summary
Bubble O(n2) No additional space Loop check for early exit
Heap O(nlgn) No additional space
Insertion O(n2) No additional space Loop check for early exit
Merge O(nlgn) Duplicate array Stack for recursion Can be used with linked lists
Quick O(nlgn) on average, O(n2) worst Stack for recursion
Selection O(n2) No additional space O(n) exchanges
Shell Between O(n(lgn)2) and O(n1.5) No additional space