Sorting algorithms

1
Sorting algorithms their analysis
2
Outline

• Several sorting algorithms
• Bubble Sort
• Selection Sort
• Insertion Sort
• Shell Sort
• Merge Sort
• Quick Sort
• For each algorithm
• Basic Idea
• Example
• Implementation
• Algorithm Analysis

3
Sorting

• Sorting ordering.
• Sorted ordered based on a particular way.
• Generally, collections of data are presented in a
  sorted manner.
• Examples of Sorting
• Words in a dictionary are sorted (and case
  distinctions are ignored).
• Files in a directory are often listed in sorted
  order.
• The index of a book is sorted (and case
  distinctions are ignored).
• Many banks provide statements that list checks in
  increasing order (by check number).
• In a newspaper, the calendar of events in a
  schedule is generally sorted by date.
• Musical compact disks in a record store are
  generally sorted by recording artist.
• Why?
• Imagine finding the phone number of your friend
  in your mobile phone, but the phone book is not
  sorted.

4
Bubble Sort Idea

• Idea bubble in water.
• Bubble in water moves upward. Why?
• How?
• When a bubble moves upward, the water from above
  will move downward to fill in the space left by
  the bubble.
• As a sorting algo
• During each iteration, compare adjacent items
  the bubble floats, the water sinks! (i.e.
  swap items)

5
Bubble sort an iteration
6
Bubble Sort Example
1
2
3
4

• Notice that at least one element will be in the
  correct position each iteration.

7
Bubble Sort Example
5
6
7
8
Stop here why?
8
Bubble Sort Implementation

• void sort(int a)
• for (int i a.length igt0 i--)
• boolean swapped false
• for (int j 0 jlti j)
• ...
• if (aj gt aj1)
• int T aj
• aj aj1
• aj1 T
• swapped true
• ...
• if (!swapped)
• return

9
Bubble Sort Analysis

• Running time
• Worst case O(N2)
• Best case O(N) -- when? why?
• Variant
• bi-directional bubble sort
• original bubble sort only works in one direction
• bi-directional bubble sort works back and forth.

10
Selection Sort Idea

• Initial condition
• Unsorted list data
• Sorted list empty
• Select the best (e.g. smallest) item from the
  unsorted group, then add the best item at the
  end of the sorted group.
• Repeat the process until the unsorted group
  becomes empty.

11
Selection Sort Example
40
2
1
43
3
4
0
-1
58
3
65
42
40
2
1
43
3
4
0
-1
42
65
58
3
42
40
2
1
3
3
4
0
-1
65
58
43
12
Selection Sort Example
42
40
2
1
3
3
4
0
65
58
43
-1
42
-1
2
1
3
3
4
0
65
58
43
40
42
-1
2
1
3
3
4
65
58
43
40
0
42
-1
2
1
0
3
4
65
58
43
40
3
13
Selection Sort Example
42
-1
2
1
3
4
65
58
43
40
3
0
1
42
-1
0
3
4
65
58
43
40
3
2
1
42
-1
0
3
4
65
58
43
40
3
2
1
42
0
3
4
65
58
43
40
3
2
-1
1
42
0
3
4
65
58
43
40
3
2
-1
14
Selection Sort Implementation
void sort(int a) throws Exception for
(int i 0 i lt a.length i) int min
i for (int j i 1 j lt a.length
j) if (aj lt amin)
min j int T amin amin
ai ai T
Select best item from unsorted list.
Add it to the end of the sorted list.
15
Selection Sort Analysis

• Running time
• Worst case O(N2)
• Best case O(N2)
• Based on big-oh analysis, is selection sort
  better than bubble sort?
• Does the actual running time reflect the analysis?

16
Insertion Sort Idea

• Initial condition
• Unsorted list data
• Sorted list empty
• Take the first (any) item from the unsorted
  group, then insert it at its correct position in
  the sorted group.
• Repeat the process until the unsorted group
  becomes empty.

17
Insertion Sort Idea

• Imagine you are sorting a hand of cards.
• 8 5 9 2 6 3
• 5 8 9 2 6 3
• 5 8 9 2 6 3
• 2 5 8 9 6 3
• 2 5 6 8 9 3
• 2 3 5 6 8 9

18
Insertion Sort Example
40
2
40
1
43
3
65
0
-1
58
3
42
4
1
2
40
43
3
65
0
-1
58
3
42
4
19
Insertion Sort Example
1
2
40
43
3
65
0
-1
58
3
42
4
1
2
3
40
43
65
0
-1
58
3
42
4
1
2
3
40
43
65
0
-1
58
3
42
4
20
Insertion Sort Example
1
2
3
40
43
65
0
-1
58
3
42
4
1
2
3
40
43
65
0
-1
58
3
42
4
1
2
3
40
43
65
0
58
3
42
4
1
2
3
40
43
65
0
-1
21
Insertion Sort Example
1
2
3
40
43
65
0
58
3
42
4
1
2
3
40
43
65
0
-1
1
2
3
40
43
65
0
58
42
4
1
2
3
3
43
65
0
-1
58
40
43
65
1
2
3
40
43
65
0
42
4
1
2
3
3
43
65
0
-1
58
40
43
65
22
Insertion Sort Implementation
public static void insertionSort (int a)
for (int ii 1 ii lt a.length ii)
int jj ii while (( jj gt 0)
(ajj lt ajj - 1)) int
temp ajj ajj ajj - 1
ajj - 1 temp jj--

Get first item in unsorted list.
Insert it in the sorted list.

• Note that ajj is always the same ? we can make
  things a bit more efficient!

23
Insertion Sort Efficient Implementation

• A slightly more efficient Insertion sort

public static void insertionSort2 (int a)
for (int ii 1 ii lt a.length ii)
int temp aii int jj ii
while (( jj gt 0) (temp lt ajj - 1))
ajj ajj - 1 jj--
ajj temp
24
Insertion Sort Analysis

• Running time analysis
• Worst case O(N2)
• Best case O(N)
• Is insertion sort faster than selection sort?
• Notice the similarity and the difference between
  insertion sort and selection sort.

25
A Lower Bound

• Bubble Sort, Selection Sort, Insertion Sort all
  have worst case of O(N2).
• It turns out, for any algorithm that exchanges
  adjacent items, this is the best worst case
  O(N2)
• In other words, this is a lower bound!
• See proof in Section 8.3 of Weiss

26
Shell Sort Idea
Donald Shell (1959) Exchange items that are far
apart!
Original
5-sort Sort items with distance 5 elements
27
Shell Sort Example
Original
40
2
1
43
3
65
0
-1
58
3
42
4
After 5-sort
40
0
-1
43
3
42
2
1
58
3
65
4
After 3-sort
2
0
-1
3
1
4
40
3
42
43
65
58
After 1-sort
1
2
3
40
43
65
0
42
1
2
3
3
43
65
0
-1
58
4
43
65
42
58
40
43
65
28
Shell Sort Gap Values

• Gap the distance between items being sorted.
• As we progress, the gap decreases. Shell Sort is
  also called Diminishing Gap Sort.
• Shell proposed starting gap of N/2, halving at
  each step.
• There are many ways of choosing the next gap.

29
Shell Sort Analysis
O(N3/2)? O(N5/4)? O(N7/6)?

• So we have 3 nested loops, but Shell Sort is
  still better than Insertion Sort! Why?

30
Mergesort Basic Idea

• Divide and Conquer approach
• Idea
• Merging two sorted array takes O(n) time
• Split an array into two takes O(1) time

31
Mergesort Merge Implementation

• Implement operation to merge two sorted arrays
  into one sorted array
• public static void merge(int A, int B, int
  C)

Assume A and B are sorted and C A B
32
Mergesort Algorithm

• Base case if the number of items to sort is 0 or
  1, return.
• Recursive case recursively sort the first and
  second half separately.
• Merging step merge the two sorted halves into a
  sorted group.

33
Mergesort Example
split
34
Mergesort Example
split
merge
35
Mergesort Example
merge
36
Mergesort Implementation

• MergeSort implementation (and driver method)
• void mergeSort(int array)
• mergeSort(array, 0, a.length-1)
• void mergeSort(int a, int left, int right)
• if(left lt right)
• int centre (left right)/2
• mergeSort(a, left, centre)
• mergeSort(a, center1, right)
• merge(a, left, center1, right)

How to merge the two subarrays of A without any
temporary space?
37
Mergesort Merge Implementation

• Implement operation to merge two sorted
  subarrays
• public static void merge(int A, int l, int c,
  int r)

38
Mergesort Analysis

• Running Time O(n log n)
• Why?

39
Quicksort Basic Idea

• Yet another divide and conquer approach
• quickSort(S) algorithm
• If the number of items in S is 0 or 1, return.
• Pick any element v ? S. This element is called
  the pivot.
• Partition S v into two disjoint groups
• L x ? S v x ? v and
• R x ? S v x ? v
• Return the result of quickSort(L), followed by v,
  followed by quickSort(R).

40
Quicksort Select Pivot
-1
58
4
2
42
3
43
40
0
1
3
65
41
Quicksort Partition
65
2
42
0
3
40
4
43
-1
58
3
1
42
Quicksort Recursive Sort Merge
43
65
58
42
2
1
3
0
-1
3
4
43
Quicksort Partition Algorithm 1
44
Quicksort Partition Algorithm 1
45
Quicksort Partition Algorithm 1
46
Quicksort Partition Algorithm 2
move pivot out of the way
while gt pivot right--
while lt pivot left
47
Quicksort Partition Algorithm 2
CROSSING
move pivot back
Quicksort recursively
Quicksort recursively
48
Quicksort Implementation

• static void QuickSort(int a, int low, int high)
  
• if(high lt low) return // base case
• pivot choosePivot(a) // select best
  pivot
• int ilow, jhigh-1
• swap(a,pivot,aj) // move pivot out of
  the way
• while(i lt j)
• // find large element starting from left
• while(ilthigh ailtpivot) i
• // find small element starting from right
• while(jgtlow ajgtpivot) j--
• // if the indexes have not crossed, swap
• if(igtj) swap(a, i, j)
• swap(a,i,high-1) // restore pivot

49
Quicksort Analysis

• Partitioning takes
• O(n)
• Merging takes
• O(1)
• So, for each recursive call, the algorithm takes
  O(n)
• How many recursive calls does a quick sort need?

50
Quicksort Choosing The Pivot

• Ideal pivot
• Median element
• Common pivot
• First element
• Element at the middle
• Median of three

51
Epilogue Generic Sort

• So far we have methods to sort integers. What
  about Strings? Employees? Cookies?
• A new method for each class? No!
• In order to be sorted, objects should be
  comparable (less than, equal, greater than).
• Solution
• use an interface that has a method to compare two
  objects.
• Remember A class that implements an interface
  inherits the interface (method definitions)
  interface inheritance, not implementation
  inheritance.

52
The Comparable Interface

• In Java, generic aspect of comparable is
  defined in an interface in package java.lang
• Method compareTo returns
• lt0 the object (this) is smaller than the
  parameter ob
• 0 the object (this) is equal to the parameter
  ob
• gt0 the object (this) is greater than the
  parameter ob

public interface Comparable public int
compareTo (Object ob)