Chapter 8 Sorting in Linear Time

1
Chapter 8 Sorting in Linear Time
2
Introduction

• Comparison sort the sorted order an algorithm
  determine is based only on comparisons between
  the input elements
• Any comparison must take ?(n lg n) comparisons in
  the worst case
• Merge sort and heapsort are asymptotically
  optimal
• Sorting in linear time
• Of course, use operations other than comparisons
  to determine the sorted order
• Counting sort, radix sort, and bucket sort

3
Lower Bounds for Comparison Sort
4
The Decision-Tree Model

• Decision tree
• A full binary tree that represents the
  comparisons between elements that are performed
  by a particular sorting algorithm operating on an
  input of a given size
• ij ? compare ai and aj
• Each leaf is annotated by a permutation lt?(1),
  ?(2), , ?(n) gt
• n! possible permutations in total
• The execution of the sorting algorithm
  corresponds to tracing a path from the root to a
  leaf
• A necessary condition for a comparison sort to be
  correct is that each of the n! permutations on n
  elements must appear as one of the leaves of the
  decision tree, and that each of these leaves must
  be reachable from the root by a path (reachable
  leaves) corresponding to an actual execution of
  the comparison sort

5
The Decision-Tree Model
5
7
3
5
7
3
5
3
7
6
A Lower Bound for the Worst Case

• The length of the longest path from the root of a
  decision tree to any of its reachable leaves
  represents the worst-case number of comparisons
  that the corresponding sorting algorithm performs
• Height of a decision tree
• Theorem 8.1. Any comparison sort algorithm
  requires ?(n lg n) comparisons in the worst case
• Consider a decision tree of height h with l
  reachable leaves
• n! ? l ? 2h
• h ? lg(n!) (since the lg function is
  monotonically increasing) ?(n lg n) (by
  equation (3.18))

7
Counting Sort
8
Counting Sort Overview

• Assumption n input elements are integers in the
  range of 0 to k (integer).
• ?(nk) ? When kO(n), counting sort ?(n)
• Basic Idea
• For each input element x, determine the number of
  elements less than or equal to x
• For each integer i (0 ? i ? k), count how many
  elements whose values are i
• Then we know how many elements are less than or
  equal to i
• Algorithm storage
• A1..n input elements
• B1..n sorted elements
• C0..k hold the number of elements less than or
  equal to i

9
Counting Sort Illustration
(Range from 0 to 5)
10
Counting Sort Illustration (Cont.)
2
5
3
0
2
3
0
3
A
11
Counting Sort Algorithm
?(nk)
?(k)
?(n)
?(k)
?(n)
12
Counting Sort Is Stable

• A sorting algorithm is stable if
• Numbers with the same value appear in the output
  array in the same order as they do in the input
  array
• Ties between two numbers are broken by the rule
  that which ever number appears first in the input
  array appears first in the output array
• Line 9 of counting sort for j?lengthA down to
  1 is essential for counting sort to be stable
• What if for j ? 1 to lengthA

13
Radix Sort
14
Radix Sort Illustration

• Sort on the least significant digit first
• It is essential that the digit sorts in this
  algorithm be stable (why?)

15
Radix Sort Algorithm
d digits in total

• Lemma 8.3 Given n d-digit numbers in which each
  digit can take on up to k possiblevalues,
  RADIX-SORT correctly sorts these numbers in
  ?(d(nk)) time
• Correctness Exercise 8.3-3
• Use counting sort to sort each digit ?(nk)
• When d is constant and kO(n) ? ?(n)

16
Bucket Sort
17
Bucket Sort Overview

• Bucket sort runs in linear time when the input is
  drawn from a uniform distribution over the
  interval 0, 1)
• Basic idea
• Divide 0,1) into n equal-sized subintervals
  (buckets)
• Distribute the n numbers into the buckets
• Sort the numbers in each bucket
• Go through the buckets in order, list the
  elements in each

18
Bucket Sort Illustration
A
B
1 2 3 4 5 6 7 8 9 10
0 1 2 3 4 5 6 7 8 9
.78
/
.17
.39
.26
.72
/
.94
/
.21
.12
.23
/
.68
19
Bucket Sort Illustration (Cont.)
20
Bucket Sort Algorithm
As long as the input has the property that the
sum of the squares of the bucket size is linear
in the total number of elements, bucket sort will
run in linear time
21
Self Study

• Lemma 8.4 (pp. 172-173)
• The total time taken by the n calls to insertion
  sort in line 5 is n O(2 1/n) (pp. 175-177)