Heap Sort: Snakes and Ladders

1
Heap Sort Snakes and Ladders

• Jeff Parker
• Merrimack College

2
Outline

• Heapsort is one of the simplest good sorts
• Simple enough to introduce in CS 1
• Complex enough to merit close study
• I often introduce it to beginners in survey
  course as clever algorithm
• I use Heapsort to introduce some advanced ideas
• I find the following phenomena a useful hook for
  the discussion

3
Visualization

• You can observe a lot just by watching Yogi
  Bera
• Film Sorting Our Sorting - Ronald Baecker
• Now on YouTube
• http//homepages.dcc.ufmg.br/dorgival/applets/Sor
  tingPoints/SortingPoints.html
• http//www.cs.ubc.ca/harrison/Java/sorting-demo.h
  tml

4
Scatter Plot Visualization

• Sorting a permutation of the integers 1, 2, 3,
  N
• At each step of sort, plot the contents of the
  array (x, y) (i, ai)
• When we are done, we will have the line y x
• Along the way, we see how the sort organizes
  things
•  

5
Artifact

• When we Heapsort an already sorted array, we see
  many pairs of lines
• These lines continue throughout the stages of the
  sort
• There seems to be a box half as big as the heap
  in the lower left
• http//homepages.dcc.ufmg.br/dorgival/applets/Sor
  tingPoints/SortingPoints.html

6
Review of Heapsort

• Heapsort is a form of selection Sort
• Standard Selection Sort takes O(N2) steps
• But after finding the best player at Wimbledon,
  it is easier to find the second best
• Note that the second best may not have made it to
  the final round
• She may have met the best player in her first
  game.

7
Definition of a Heap

• Heaps are a special kind of tree with two
  properties
• A shape property and an order property
• A binary tree T is a heap if it is left-complete
  tree with the heap partial order property
• Heap Partial Order Property
• Each parent is greater than either son
• Heaps have weak order property, but a strong
  shape property

8
How do we store heaps?

• How can we store a heap without using additional
  space for pointers?
• The left-complete shape allows us to use implicit
  "pointers"

The sons of position 1 are in 2 and 3 The sons of
position 2 are in 4 and 5 The sons of position 3
are in 6 and 7 The parent of position 6 is
position 3 Our arrays are not 1-based, so...
9
FixDown

• void fixDown(int a, int root, int limit)
• int child 2 root
• while ( child lt limit ) // Does root have
  child?
• if ( (child 1 lt limit)
• (achild lt achild 1) ) // Largest child
• child child 1
• if (aroot lt achild)
• swap(a, root, child)
• root child
• child 2 child
• else break

10
Heap Sort

• void heapsort(item a)
• int left, right, N a.length
• // Construct Heap from unsorted array
• for (left N/2 left gt 0 left-- )
• fixDown(a, left, N)
• // Select largest element. Swap into last open
  slot
• for (right N - 1 right gt 1 right--)
• swap(a, 1, right) // Put largest in place
• // Heap condition now false at root restore
• fixDown(a, 1, right)

11
Talking Points

• Heapsort is not greedy - would be greedy to take
  advantage of sorted array
• It sorts in place nothing leaves the big box
• Heapsort is an O(N lnN) sort
• As with any fast sorting algorithm, it exchanges
  distant items
• We exchange the items ai and a2i
• In a heap, we
• Know exactly where largest element is.
• Know approximately where second largest is
• Know the area that holds the third largest
• Know less about the fourth largest

12
Greed is not good

• Can select one outcome out of 8 in three compares
  if we are not greedy
• If we try to be too greedy, it will take 4 steps
  most of the time
• Heapsort is not greedy ignores fact that array
  is sorted

13
Sorts in place

• Heapsort is often described working on a binary
  tree
• Does not appear to take place within the confines
  of the array
• The visualization shows that all swaps are made
  within the big box.

14
Each pass takes ln(N) steps

• After creating a heap, we make N passes over the
  data
• Each pass takes puts largest remaing item in
  place, then fixDown(1)
• Each invocation of fixDown() takes at most ln(N)
  steps
• (I can only double ln(N) times)
• I am skipping between lines y x/2k
• There are only ln(N) pairs of lines

15
Build Heap Analysis

• Assume the worst case we need to fixDown each
  item to leaf
• How many moves is this?
• We show paths of equivalent length total length
  is lt N

16
Inversions

• Consider each pairs of entries in an array
• If the pair is not in order, it is said to be
  inverted
• The number of inversions is a measure of how out
  of order an array is
• The array below has 4 inversions two are
  illustrated
• What is the most inverted an array can be?
  4 3 2 1
• This is quadratic
• Since exchanging adjacent items can only remove
  one inversion,
• Any efficient sort cannot limit itself to
  exchanging adjacent items

3
7
9
6
5
17
Compare at a distance

• Any efficient sort cannot limit itself to
  exchanging adjacent items
• Heapsort exchanges distant items
• We exchange the items ai and a2i

18
Weak Order condition

• Know exactly where largest element is top layer
• Know approximately where second largest is
  second layer
• Know the area that holds the third largest
  second or third
• Know less about the fourth largest second,
  third, or fourth

19
Artifact

• Let's spend some time looking at why the artifact
  arises
• Although we look at a specific example, the
  behavior is visible in arrays of different size,
  and in arrays that are close to being sorted

20
In action

• A heap of size N has ceil(N/2) leaves
• Thus the last half of the array is held in leaves
• Every item in the first half is smaller than
  items below it
• Don't need to fixDown the leaves

21
Second Layer

• First step is to fixDown the layer above the
  leaves
• A heap of size N has ceil(N/2) leaves
• Thus the last half of the array is held in leaves
• Every item in the first half is smaller than
  items below it
• Thus every item in the first half will move down

22
Mechanism

• We are comparing ai with a2i and a2i 1
• We swap ai with a2i1
• Send i to slot 2i1 and 2i1 to slot i
• We are removing points from y x to build y 2x
  and y x/2
• Calls to fixDown the next layer will create y
  4x and y x/4

23
Power of 2?

• This behavior is seen in arrays of other sizes.
• Shows up best in large arrays
• We can perform a similar analysis with items 1
  step away from leaf, 2 steps away, etc.

24
Third Layer

• When we fixDown the third level from the bottom
• We have fewer points, but they remain in a line
• The large items are harvested from the row below
• They are replaced by smaller items
• As we process each new row
• The old inhabitants find their way to the bottom,
  forming a new line

25
Make Heap

• In the final image, we have 7 lines (two
  consisting of a single point)
• Note that one element (7) has returned to it's
  original place
• It is the only item in the inner box

pairs of dual lines
26
Sorting

• As we sort, we are decreasing the size of the
  heap, and moving the midpoint towards the front
• The fundamental operation remains swaping ai
  with a2i or a2i1
• Remove items from lines y 2kx and move them to
  y 2k1x or y2k-1x
• Like periodic sandbars take material from level
  n for level n1 and n-1

27
Summary

• Heapsort is a sort that repays close study
• I use it to explore a number of issues important
  in sorting
• Sorting a sorted array has artifacts that engage
  students

28
References

• J. W. J. Williams, "Algorithm 232 Heapsort",
  CACM 7, (1964), 347-348
• R. W. Floyd, "Algorithm 245 Treesort 3", CACM 7
  (1964), 701.
• Floyd introduced the linear-time algorithm for
  constructing a heap
• B. I. Shaw, Animations of Sort Algorithms Baruch
  College, CUNY, 2/98.
• http//homepages.dcc.ufmg.br/dorgival/applets/Sor
  tingPoints/SortingPoints.html
• J. Harrison and J. Boritz, Sorting Algorithms,
  2001
• http//www.cs.ubc.ca/harrison/Java/sorting-demo.h
  tml
• based on applet by James Gosling