Fastest Sorts of Sorts

1
Fastest Sorts of Sorts

• Eugene M. Taranta II

2
Sorting Symposium Subjects

• Sorting
• Why
• Things to Consider
• Heap and Merge
• Linear Time Algorithms
• Counting Sort
• Radix Sort
• Radix Exchange Sort (personal favorite)

3
Why Study Sorting

• Many Applications Rely on Sorted Data
• for Searching
• for Pattern matching
• for Data Analysis
• ... and on and on .

• Increases Your Value
• as a Software Engineer
• as a Problem Solver

4
Things To Consider

• In-Place Sorting
• The Sorting Algorithm does not require a
  significant amount of addition space

• Stable Sorting
• Objects considered to be identical are kept in
  order

5
Heap and Merge Sort

• Both run in O( n log n) time
• Heap Sort utilizes a tree
• Merge Sort utilizes Divide and Conqueror
• Storage Considerations
• - Heap Sort can run inplace
• - Traditional Merge Sort requires 2n space
• Algorithms exist that run Merge Sort inplace

• Boring to talk about
• Covered Merge Sort in Comp. Sci. I, II, and III
• Covered Heap Sort in Comp. Sci. II and III

6
Sorting in Linear Time

• Comparison Based Sorts
• Best you can do ?( n log n )

7
Sorting in Linear Time

• Comparison Based Sorts
• Best you can do ?( n log n )

• Solution Stop Comparing!
• Try Counting
• Try Distributing

8
Counting Sort

• Objects are represented by keys
• Each key must be within a range 1 to k

• There exist k counters
• Counters initialized to zero
• Each object examined increments the appropriate
  counter

• Determine indexing
• Using the counters, we can determine where,
  within an array, each object should be placed.

9
Counting Sort Algorithm

• // Input A, array of objects to sort
• // k, maximum key
• // Output B, sorted array
• CountingSort( A, B, k )
• for i ? 1 to k
• ci ? 0
• for i ? 1 A.length
• cAi.key ? cAi.Key 1
• for i ? 2 to A.length
• ci ? ci ci-1
• for i ? A.length downto 1
• B c Ai.key ? Ai
• cAi.Key ? cAi.Key - 1

Loop 1 Initialize counters
Loop 2 Count keys
Loop 3 Determine last index for
each key by totaling previous and
current counts
Loop 4 Copy Objects into new
array. Decrement indices as
objects are placed.
10
Counting Sort Analyzed

• Not inplace
• Requires 2n k storage

• Runs in ?(nk)
• Two passes on n
• Two passes on k
• When k is constant or k ltlt n, we can assume ?(n)

• Stable

11
Radix Sort

• Radix is the base of a number system
• Create size of radix bins

• Examine each digit in number
• Start with the least significant digit
• Based on digit, distribute into correct bin
• After each pass stack numbers back up

12
Radix Sort Demo
56 36 30 28 13

• 13
• 30
• 28
• 56
• 36

28 36 56 13 30
28 36 56 13 30
13 30 28 56 36

36 30
13 56 28

36
13 28 30 56
Radix Bins 0 1 2 3 4
5 6 7 8 9
13
Radix Sort Algorithm

• // Input A, array of objects to sort
• // r, radix
• // d, digits to examine
• // Output A, sorted
• RadixSort( A, r, d )
• CreateBins( r )
• for i ? 1 to d
• for j ? 1 to A.length
• binsAj.digit(i).add(A)
• StackBins( A, bins )

Create a bin for each possible digit
Outer Loop Examine each digit
Inner Loop Add objects to bins
according to digit
Copy Bins back into Array
14
Radix Sort Analyzed

• Not inplace
• Requires 2n bin management storage

• Runs in ?(dn)
• Requires a pass for each d examined
• Requires two passes for each digit
• Place into bin
• Copy back to Array
• If d is constant or d ltlt n, we can assume ?(n)

• Stable

15
Radix Exchange Sort

• Sorts in place !

But how ?!?

• Uses radix base two

• Starts at most significant digit

• Works like Quick Sort
• Pushes large (bit on) and small (bit off) items
  to opposite ends of the array
• Runs recursively

16
Radix Exchange Sort Demo
111
101 111
? ?
111 101
? ?
? ?
? ?
? ?
101 001 010 000 111
? ?
? ?
101 111 010 000 001
??
101
??
010
010 000 001
? ?
? ?
??
000 001
? ?
001 000
17
Radix Exchange Sort Algorithm

• // Input A, array of objects to sort
• // b, bit number
• // l, length of array
• // Output A, sorted array
• RadixExchangeSort( A, B, k )
• if b 0 and l gt 1
• return
• j ? l
• while jgt1 Aj.bit(b) 1
• j ? j 1
• for i ? 1 to j
• if Aj.bit(b) 1
• swap( Ai, Aj )
• do
• j ? j - 1
• while jgti Aj.bit(b) 1
• RadixExchangeSort( A, b-1, i )

Check for exit condition
Loop 1 Move high bit index counter
down
Loop 2 Examine each object If high
bit is set swap with low bit element
Loop 3 Move high bit index counter
down
Call sort on low numbers Call sort on high numbers
18
Radix Exchange Sort Analyzed

• Inplace !
• Plus some book keeping space

• Runs in ?(bn)
• Requires a pass for each bit examined
• If b is constant or b ltlt n, we can assume ?(n)

• Not Stable

19
Sorting Symposium Summary

• Comparison based sorts are limited to n log n time

• Non-comparison based sorts can run in linear time

• Counting Sort Stable but not in-place

• Radix Sort Stable but not in-place

• Radix Exchange Sort In-place but not sable

20
(No Transcript)
21
(No Transcript)
22
References

• Michael Ben-Or. Lower Bounds For Algebraic
  Computation Trees, Proceedings of the Fifteenth
  Annual ACM Symposium On Theory of Computing,
  pages 80-86, 1983.
• Bing-Chao and Michael A. Langston. Practical
  In-Place Merging, Communications of the ACM Vol
  31,3. pages 348-352. March 1988.
• Thomas H. Cormen, Charles E. Leiserson and Ronald
  L. Riverest. Introduction to Algorithms. MIT
  Press, 1990
• C.C. Cotlieb. Sorting On Computers,
  Communications of the ACM Vol 6,5. pages 194-201.
  May 1963.
• Paul Hilderbrant and Harold Isbitz.
  Radix-Exchange-An Internal Sorting Method for
  Digital Computers, Journal of the ACM Vol 10,2.
  Pages 142-150.
• Richard Johnsonbaugh and Marcus Schaefer.
  Algorithms, Person Prentice Hall, 2004.
• William A. Martin. Sorting, ACM Computing
  Surveys. Vol 3,4. pages 147-174. Dec. 1973.