Algorithm Analysis

1
Algorithm Analysis
2
Algorithm

• An algorithm is a set of instructions to be
  followed to solve a problem.
• There can be more than one solution (more than
  one algorithm) to solve a given problem.
• An algorithm can be implemented using different
  programming languages on different platforms.
• An algorithm must be correct. It should correctly
  solve the problem.
• e.g. For sorting, this means even if (1) the
  input is already sorted, or (2) it contains
  repeated elements.
• Once we have a correct algorithm for a problem,
  we have to determine the efficiency of that
  algorithm.

3
Algorithmic Performance

• There are two aspects of algorithmic performance
• Time
• Instructions take time.
• How fast does the algorithm perform?
• What affects its runtime? 
• Space
• Data structures take space
• What kind of data structures can be used?
• How does choice of data structure affect the
  runtime?
• We will focus on time
• How to estimate the time required for an
  algorithm
• How to reduce the time required

4
Analysis of Algorithms

• Analysis of Algorithms is the area of computer
  science that provides tools to analyze the
  efficiency of different methods of solutions.
• How do we compare the time efficiency of two
  algorithms that solve the same problem?
• Naïve Approach implement these algorithms in a
  programming language (C), and run them to
  compare their time requirements. Comparing the
  programs (instead of algorithms) has
  difficulties.
• How are the algorithms coded?
• Comparing running times means comparing the
  implementations.
• We should not compare implementations, because
  they are sensitive to programming style that may
  cloud the issue of which algorithm is inherently
  more efficient.
• What computer should we use?
• We should compare the efficiency of the
  algorithms independently of a particular
  computer.
• What data should the program use?
• Any analysis must be independent of specific data.

5
Analysis of Algorithms

• When we analyze algorithms, we should employ
  mathematical techniques that analyze algorithms
  independently of specific implementations,
  computers, or data.
• To analyze algorithms
• First, we start to count the number of
  significant operations in a particular solution
  to assess its efficiency.
• Then, we will express the efficiency of
  algorithms using growth functions.

6
The Execution Time of Algorithms

• Each operation in an algorithm (or a program) has
  a cost.
• ? Each operation takes a certain of time.
• count count 1 ? take a certain amount of
  time, but it is constant
• A sequence of operations
• count count 1 Cost c1
• sum sum count Cost c2
• ? Total Cost c1 c2

7
The Execution Time of Algorithms (cont.)

• Example Simple If-Statement
• Cost Times
• if (n lt 0) c1 1
• absval -n c2 1
• else
• absval n c3 1
• Total Cost lt c1 max(c2,c3)

8
The Execution Time of Algorithms (cont.)

• Example Simple Loop
• Cost Times
• i 1 c1 1
• sum 0 c2 1
• while (i lt n) c3 n1
• i i 1 c4 n
• sum sum i c5 n
• Total Cost c1 c2 (n1)c3 nc4 nc5
• ? The time required for this algorithm is
  proportional to n

9
The Execution Time of Algorithms (cont.)

• Example Nested Loop
• Cost Times
• i1 c1 1
• sum 0 c2 1
• while (i lt n) c3 n1
• j1 c4 n
• while (j lt n) c5 n(n1)
• sum sum i c6 nn
• j j 1 c7 nn
• i i 1 c8 n
• Total Cost c1 c2 (n1)c3 nc4
  n(n1)c5nnc6nnc7nc8
• ? The time required for this algorithm is
  proportional to n2

10
General Rules for Estimation

• Loops The running time of a loop is at most the
  running time of the statements inside of that
  loop times the number of iterations.
• Nested Loops Running time of a nested loop
  containing a statement in the inner most loop is
  the running time of statement multiplied by the
  product of the sized of all loops.
• Consecutive Statements Just add the running
  times of those consecutive statements.
• If/Else Never more than the running time of the
  test plus the larger of running times of S1 and
  S2.

11
Algorithm Growth Rates

• We measure an algorithms time requirement as a
  function of the problem size.
• Problem size depends on the application e.g.
  number of elements in a list for a sorting
  algorithm, the number disks for towers of hanoi.
• So, for instance, we say that (if the problem
  size is n)
• Algorithm A requires 5n2 time units to solve a
  problem of size n.
• Algorithm B requires 7n time units to solve a
  problem of size n.
• The most important thing to learn is how quickly
  the algorithms time requirement grows as a
  function of the problem size.
• Algorithm A requires time proportional to n2.
• Algorithm B requires time proportional to n.
• An algorithms proportional time requirement is
  known as growth rate.
• We can compare the efficiency of two algorithms
  by comparing their growth rates.

12
Algorithm Growth Rates (cont.)
Time requirements as a function of the problem
size n
13
Common Growth Rates
14
Figure 6.1 Running times for small inputs
15
Figure 6.2 Running times for moderate inputs
16
Order-of-Magnitude Analysis and Big 0 Notation

• If Algorithm A requires time proportional to
  f(n), Algorithm A is said to be order f(n), and
  it is denoted as O(f(n)).
• The function f(n) is called the algorithms
  growth-rate function.
• Since the capital O is used in the notation,
  this notation is called the Big O notation.
• If Algorithm A requires time proportional to n2,
  it is O(n2).
• If Algorithm A requires time proportional to n,
  it is O(n).

17
Definition of the Order of an Algorithm

• Definition
• Algorithm A is order f(n) denoted as
  O(f(n))
• if constants k and n0 exist such that A
  requires
• no more than kf(n) time units to solve a
  problem
• of size n ? n0.
• The requirement of n ? n0 in the definition of
  O(f(n)) formalizes the notion of sufficiently
  large problems.
• In general, many values of k and n can satisfy
  this definition.

18
Order of an Algorithm

• If an algorithm requires n23n10 seconds to
  solve a problem size n. If constants k and n0
  exist such that
• kn2 gt n23n10 for all n ? n0 .
• the algorithm is order n2 (In fact, k is 3 and
  n0 is 2)
• 3n2 gt n23n10 for all n ? 2 .
• Thus, the algorithm requires no more than kn2
  time units for n ? n0 ,
• So it is O(n2)

19
Order of an Algorithm (cont.)
20
A Comparison of Growth-Rate Functions
21
A Comparison of Growth-Rate Functions (cont.)
22
Growth-Rate Functions

• O(1) Time requirement is constant,
  and it is independent of the problems size.
• O(log2n) Time requirement for a logarithmic
  algorithm increases increases slowly
• as the problem size increases.
• O(n) Time requirement for a linear algorithm
  increases directly with the size
• of the problem.
• O(nlog2n) Time requirement for a nlog2n
  algorithm increases more rapidly than
• a linear algorithm.
• O(n2) Time requirement for a quadratic
  algorithm increases rapidly with the
• size of the problem.
• O(n3) Time requirement for a cubic
  algorithm increases more rapidly with the
• size of the problem than the time
  requirement for a quadratic algorithm.
• O(2n) As the size of the problem increases,
  the time requirement for an
• exponential algorithm increases too
  rapidly to be practical.

23
Growth-Rate Functions

• If an algorithm takes 1 second to run with the
  problem size 8, what is the time requirement
  (approximately) for that algorithm with the
  problem size 16?
• If its order is
• O(1) ? T(n) 1 second
• O(log2n) ? T(n) (1log216) / log28 4/3
  seconds
• O(n) ? T(n) (116) / 8 2 seconds
• O(nlog2n) ? T(n) (116log216) / 8log28
  8/3 seconds
• O(n2) ? T(n) (1162) / 82 4 seconds
• O(n3) ? T(n) (1163) / 83 8 seconds
• O(2n) ? T(n) (1216) / 28 28 seconds 256
  seconds

24
Properties of Growth-Rate Functions

• We can ignore low-order terms in an algorithms
  growth-rate function.
• If an algorithm is O(n34n23n), it is also
  O(n3).
• We only use the higher-order term as algorithms
  growth-rate function.
• We can ignore a multiplicative constant in the
  higher-order term of an algorithms growth-rate
  function.
• If an algorithm is O(5n3), it is also O(n3).
• O(f(n)) O(g(n)) O(f(n)g(n))
• We can combine growth-rate functions.
• If an algorithm is O(n3) O(4n), it is also O(n3
  4n2) ? So, it is O(n3).
• Similar rules hold for multiplication.

25
Some Mathematical Facts

• Some mathematical equalities are

26
Growth-Rate Functions Example1

• Cost Times
• i 1 c1 1
• sum 0 c2 1
• while (i lt n) c3 n1
• i i 1 c4 n
• sum sum i c5 n
• T(n) c1 c2 (n1)c3 nc4 nc5
• (c3c4c5)n (c1c2c3)
• an b
• ? So, the growth-rate function for this
  algorithm is O(n)

27
Growth-Rate Functions Example2

• Cost Times
• i1 c1 1
• sum 0 c2 1
• while (i lt n) c3 n1
• j1 c4 n
• while (j lt n) c5 n(n1)
• sum sum i c6 nn
• j j 1 c7 nn
• i i 1 c8 n
• T(n) c1 c2 (n1)c3 nc4
  n(n1)c5nnc6nnc7nc8
• (c5c6c7)n2 (c3c4c5c8)n (c1c2c3)
• an2 bn c
• ? So, the growth-rate function for this
  algorithm is O(n2)

28
Growth-Rate Functions Example3

• Cost Times
• for (i1 iltn i) c1 n1
• for (j1 jlti j) c2
• for (k1 kltj k) c3
• xx1 c4
• T(n) c1(n1) c2( )
  c3 ( ) c4(
  )
• an3 bn2 cn d
• ? So, the growth-rate function for this
  algorithm is O(n3)

29
Growth-Rate Functions Recursive Algorithms

• void hanoi(int n, char source, char dest, char
  spare) Cost
• if (n gt 0) c1
• hanoi(n-1, source, spare, dest) c2
• cout ltlt "Move top disk from pole " ltlt source
  c3
• ltlt " to pole " ltlt dest ltlt endl
• hanoi(n-1, spare, dest, source) c4
• The time-complexity function T(n) of a recursive
  algorithm is defined in terms of itself, and this
  is known as recurrence equation for T(n).
• To find the growth-rate function for that
  recursive algorithm, we have to solve that
  recurrence relation.

30
Growth-Rate Functions Hanoi Towers

• What is the cost of hanoi(n,A,B,C)?
• when n0
• T(0) c1
• when ngt0
• T(n) c1 c2 T(n-1) c3 c4 T(n-1)
• 2T(n-1) (c1c2c3c4)
• 2T(n-1) c ? recurrence equation for the
  growth-rate function of
  hanoi-towers algorithm
• Now, we have to solve this recurrence equation to
  find the growth-rate function of hanoi-towers
  algorithm

31
Growth-Rate Functions Hanoi Towers (cont.)

• There are many methods to solve recurrence
  equations, but we will use a simple method known
  as repeated substitutions.
• T(n) 2T(n-1) c
• 2 (2T(n-2)c) c
• 2 (2 (2T(n-3)c) c) c
• 23 T(n-3) (222120)c (assuming ngt2)
• when substitution repeated i-1th times
• 2i T(n-i) (2i-1 ... 2120)c
• when in
• 2n T(0) (2n-1 ... 2120)c
• 2n c1 ( )c
• 2n c1 ( 2n-1 )c 2n(c1c) c ?
  So, the growth rate function is O(2n)

32
What to Analyze

• An algorithm can require different times to solve
  different problems of the same size.
• Eg. Searching an item in a list of n elements
  using sequential search. ? Cost 1,2,...,n
• Worst-Case Analysis The maximum amount of time
  that an algorithm require to solve a problem of
  size n.
• This gives an upper bound for the time complexity
  of an algorithm.
• Normally, we try to find worst-case behavior of
  an algorithm.
• Best-Case Analysis The minimum amount of time
  that an algorithm require to solve a problem of
  size n.
• The best case behavior of an algorithm is NOT so
  useful.
• Average-Case Analysis The average amount of time
  that an algorithm require to solve a problem of
  size n.
• Sometimes, it is difficult to find the
  average-case behavior of an algorithm.
• We have to look at all possible data
  organizations of a given size n, and their
  distribution probabilities of these
  organizations.
• Worst-case analysis is more common than
  average-case analysis.

33
What is Important?

• An array-based list retrieve operation is O(1), a
  linked-list-based list retrieve operation is
  O(n).
• But insert and delete operations are much easier
  on a linked-list-based list implementation.
• ? When selecting the implementation of an
  Abstract Data Type (ADT), we have to consider how
  frequently particular ADT operations occur in a
  given application.
• If the problem size is always small, we can
  probably ignore the algorithms efficiency.
• In this case, we should choose the simplest
  algorithm.

34
What is Important? (cont.)

• We have to weigh the trade-offs between an
  algorithms time requirement and its memory
  requirements.
• We have to compare algorithms for both style and
  efficiency.
• The analysis should focus on gross differences in
  efficiency and not reward coding tricks that save
  small amount of time.
• That is, there is no need for coding tricks if
  the gain is not too much.
• Easily understandable program is also important.
• Order-of-magnitude analysis focuses on large
  problems.

35
Sequential Search

• int sequentialSearch(const int a, int item, int
  n)
• for (int i 0 i lt n ai! item i)
• if (i n)
• return 1
• return i
• Unsuccessful Search ? O(n)
• Successful Search
• Best-Case item is in the first location of the
  array ?O(1)
• Worst-Case item is in the last location of the
  array ?O(n)
• Average-Case The number of key comparisons 1,
  2, ..., n
• ? O(n)

36
Binary Search

• int binarySearch(int a, int size, int x)
• int low 0
• int high size 1
• int mid // mid will be the index of
• // target when its found.
• while (low lt high)
• mid (low high)/2
• if (amid lt x)
• low mid 1
• else if (amid gt x)
• high mid 1
• else
• return mid
• return 1

37
Binary Search Analysis

• For an unsuccessful search
• The number of iterations in the loop is ?log2n?
  1
• ? O(log2n)
• For a successful search
• Best-Case The number of iterations is 1. ?
  O(1)
• Worst-Case The number of iterations is ?log2n?
  1 ? O(log2n)
• Average-Case The avg. of iterations lt log2n
  ? O(log2n)
• 0 1 2 3 4 5 6 7 ? an array with size 8
• 3 2 3 1 3 2 3 4 ? of iterations
• The average of iterations 21/8 lt log28

38
How much better is O(log2n)?

• n O(log2n)
• 16 4
• 64 6
• 256 8
• 1024 (1KB) 10
• 16,384 14
• 131,072 17
• 262,144 18
• 524,288 19
• 1,048,576 (1MB) 20
• 1,073,741,824 (1GB) 30

39
Sorting

• Sorting is a process that organizes a collection
  of data into either ascending or descending
  order.
• An internal sort requires that the collection of
  data fit entirely in the computers main memory.
• We can use an external sort when the collection
  of data cannot fit in the computers main memory
  all at once but must reside in secondary storage
  such as on a disk.
• We will analyze only internal sorting algorithms.
• Any significant amount of computer output is
  generally arranged in some sorted order so that
  it can be interpreted.
• Sorting also has indirect uses. An initial sort
  of the data can significantly enhance the
  performance of an algorithm.
• Majority of programming projects use a sort
  somewhere, and in many cases, the sorting cost
  determines the running time.
• A comparison-based sorting algorithm makes
  ordering decisions only on the basis of
  comparisons.

40
Sorting Algorithms

• There are many sorting algorithms, such as
• Selection Sort
• Insertion Sort
• Bubble Sort
• Merge Sort
• Quick Sort
• The first three are the foundations for faster
  and more efficient algorithms.

41
Selection Sort

• The list is divided into two sublists, sorted and
  unsorted, which are divided by an imaginary wall.
  
• We find the smallest element from the unsorted
  sublist and swap it with the element at the
  beginning of the unsorted data.
• After each selection and swapping, the imaginary
  wall between the two sublists move one element
  ahead, increasing the number of sorted elements
  and decreasing the number of unsorted ones.
• Each time we move one element from the unsorted
  sublist to the sorted sublist, we say that we
  have completed a sort pass.
• A list of n elements requires n-1 passes to
  completely rearrange the data.

42
Sorted
Unsorted
Original List
After pass 1
After pass 2
After pass 3
After pass 4
After pass 5
43
Selection Sort (cont.)

• template ltclass Itemgt
• void selectionSort( Item a, int n)
• for (int i 0 i lt n-1 i)
• int min i
• for (int j i1 j lt n j)
• if (aj lt amin) min j
• swap(ai, amin)
• template lt class Objectgt
• void swap( Object lhs, Object rhs )
• Object tmp lhs
• lhs rhs
• rhs tmp

44
Selection Sort -- Analysis

• In general, we compare keys and move items (or
  exchange items) in a sorting algorithm (which
  uses key comparisons).
• ? So, to analyze a sorting algorithm we should
  count the number of key comparisons and the
  number of moves.
• Ignoring other operations does not affect our
  final result.
• In selectionSort function, the outer for loop
  executes n-1 times.
• We invoke swap function once at each iteration.
• ? Total Swaps n-1
• ? Total Moves 3(n-1) (Each swap has three
  moves)

45
Selection Sort Analysis (cont.)

• The inner for loop executes the size of the
  unsorted part minus 1 (from 1 to n-1), and in
  each iteration we make one key comparison.
• ? of key comparisons 12...n-1 n(n-1)/2
• ? So, Selection sort is O(n2)
• The best case, the worst case, and the average
  case of the selection sort algorithm are same. ?
  all of them are O(n2)
• This means that the behavior of the selection
  sort algorithm does not depend on the initial
  organization of data.
• Since O(n2) grows so rapidly, the selection sort
  algorithm is appropriate only for small n.
• Although the selection sort algorithm requires
  O(n2) key comparisons, it only requires O(n)
  moves.
• A selection sort could be a good choice if data
  moves are costly but key comparisons are not
  costly (short keys, long records).

46
Comparison of N, logN and N2

• N O(LogN) O(N2)
• 16 4 256
• 64 6 4K
• 256 8 64K
• 1,024 10 1M
• 16,384 14 256M
• 131,072 17 16G
• 262,144 18 6.87E10
• 524,288 19 2.74E11
• 1,048,576 20 1.09E12
• 1,073,741,824 30 1.15E18

47
Insertion Sort

• Insertion sort is a simple sorting algorithm that
  is appropriate for small inputs.
• Most common sorting technique used by card
  players.
• The list is divided into two parts sorted and
  unsorted.
• In each pass, the first element of the unsorted
  part is picked up, transferred to the sorted
  sublist, and inserted at the appropriate place.
• A list of n elements will take at most n-1 passes
  to sort the data.

48
Sorted
Unsorted
Original List
After pass 1
After pass 2
After pass 3
After pass 4
After pass 5
49
Insertion Sort Algorithm

• template ltclass Itemgt
• void insertionSort(Item a, int n)
• for (int i 1 i lt n i)
• Item tmp ai
• for (int ji jgt0 tmp lt aj-1 j--)
• aj aj-1
• aj tmp

50
Insertion Sort Analysis

• Running time depends on not only the size of the
  array but also the contents of the array.
• Best-case ? O(n)
• Array is already sorted in ascending order.
• Inner loop will not be executed.
• The number of moves 2(n-1) ? O(n)
• The number of key comparisons (n-1) ? O(n)
• Worst-case ? O(n2)
• Array is in reverse order
• Inner loop is executed i-1 times, for i 2,3, ,
  n
• The number of moves 2(n-1)(12...n-1)
  2(n-1) n(n-1)/2 ? O(n2)
• The number of key comparisons (12...n-1)
  n(n-1)/2 ? O(n2)
• Average-case ? O(n2)
• We have to look at all possible initial data
  organizations.
• So, Insertion Sort is O(n2)

51
Analysis of insertion sort

• Which running time will be used to characterize
  this algorithm?
• Best, worst or average?
• Worst
• Longest running time (this is the upper limit for
  the algorithm)
• It is guaranteed that the algorithm will not be
  worst than this.
• Sometimes we are interested in average case. But
  there are some problems with the average case.
• It is difficult to figure out the average case.
  i.e. what is average input?
• Are we going to assume all possible inputs are
  equally likely?
• In fact for most algorithms average case is same
  as the worst case.

52
Bubble Sort

• The list is divided into two sublists sorted and
  unsorted.
• The smallest element is bubbled from the unsorted
  list and moved to the sorted sublist.
• After that, the wall moves one element ahead,
  increasing the number of sorted elements and
  decreasing the number of unsorted ones.
• Each time an element moves from the unsorted part
  to the sorted part one sort pass is completed.
• Given a list of n elements, bubble sort requires
  up to n-1 passes to sort the data.

53
Bubble Sort
Original List
After pass 1
After pass 2
After pass 3
After pass 4
54
Bubble Sort Algorithm

• template ltclass Itemgt
• void bubleSort(Item a, int n)
• bool sorted false
• int last n-1
• for (int i 0 (i lt last) !sorted i)
• sorted true
• for (int jlast j gt i j--)
• if (aj-1 gt aj
• swap(aj,aj-1)
• sorted false // signal exchange

55
Bubble Sort Analysis

• Best-case ? O(n)
• Array is already sorted in ascending order.
• The number of moves 0 ? O(1)
• The number of key comparisons (n-1) ? O(n)
• Worst-case ? O(n2)
• Array is in reverse order
• Inner loop is executed n-1 times,
• The number of moves 3(12...n-1) 3
  n(n-1)/2 ? O(n2)
• The number of key comparisons (12...n-1)
  n(n-1)/2 ? O(n2)
• Average-case ? O(n2)
• We have to look at all possible initial data
  organizations.
• So, Bubble Sort is O(n2)

56
Mergesort

• Mergesort algorithm is one of two important
  divide-and-conquer sorting algorithms (the other
  one is quicksort).
• It is a recursive algorithm.
• Divides the list into halves,
• Sort each halve separately, and
• Then merge the sorted halves into one sorted
  array.

57
Mergesort - Example
58
Merge

• const int MAX_SIZE maximum-number-of-items-in-ar
  ray
• void merge(DataType theArray, int first, int
  mid, int last)
• DataType tempArrayMAX_SIZE // temporary
  array
• int first1 first // beginning of first
  subarray
• int last1 mid // end of first subarray
• int first2 mid 1 // beginning of second
  subarray
• int last2 last // end of second subarray
• int index first1 // next available location
  in tempArray
• for ( (first1 lt last1) (first2 lt
  last2) index)
• if (theArrayfirst1 lt theArrayfirst2)
  
• tempArrayindex theArrayfirst1
• first1
• else
• tempArrayindex theArrayfirst2
• first2

59
Merge (cont.)

• // finish off the first subarray, if necessary
• for ( first1 lt last1 first1, index)
• tempArrayindex theArrayfirst1
• // finish off the second subarray, if
  necessary
• for ( first2 lt last2 first2, index)
• tempArrayindex theArrayfirst2
• // copy the result back into the original
  array
• for (index first index lt last index)
• theArrayindex tempArrayindex
• // end merge

60
Mergesort

• void mergesort(DataType theArray, int first,
  int last)
• if (first lt last)
• int mid (first last)/2 // index of
  midpoint
• mergesort(theArray, first, mid)
• mergesort(theArray, mid1, last)
• // merge the two halves
• merge(theArray, first, mid, last)
• // end mergesort

61
Mergesort - Example
divide
divide
divide
divide
divide
divide
divide
merge
merge
merge
merge
merge
merge
merge
62
Mergesort Example2
63
Mergesort Analysis of Merge
A worst-case instance of the merge step in
mergesort
64
Mergesort Analysis of Merge (cont.)
0 k-1
0 k-1

• Merging two sorted arrays of size k
• Best-case
• All the elements in the first array are smaller
  (or larger) than all the elements in the second
  array.
• The number of moves 2k 2k
• The number of key comparisons k
• Worst-case
• The number of moves 2k 2k
• The number of key comparisons 2k-1

0 2k-1
65
Mergesort - Analysis
Levels of recursive calls to mergesort, given an
array of eight items
66
Mergesort - Analysis
2m
level 0 1 merge (size 2m-1)
2m-1
2m-1
level 1 2 merges (size 2m-2)
level 2 4 merges (size 2m-3)
2m-2
2m-2
2m-2
2m-2
. . .
. . .
level m-1 2m-1 merges (size 20)
20
20
. . . . . . . . . . . . . . . . .
level m
67
Mergesort - Analysis

• Worst-case
• The number of key comparisons
• 20(22m-1-1) 21(22m-2-1) ...
  2m-1(220-1)
• (2m - 1) (2m - 2) ... (2m 2m-1) ( m
  terms )
• m2m
• m2m 2m 1
• n log2n n 1
• ? O (n log2n )

68
Mergesort Analysis

• Mergesort is extremely efficient algorithm with
  respect to time.
• Both worst case and average cases are O (n
  log2n )
• But, mergesort requires an extra array whose size
  equals to the size of the original array.
• If we use a linked list, we do not need an extra
  array
• But, we need space for the links
• And, it will be difficult to divide the list into
  half ( O(n) )

69
Quicksort

• Like mergesort, Quicksort is also based on
• the divide-and-conquer paradigm.
• But it uses this technique in a somewhat opposite
  manner,
• as all the hard work is done before the
  recursive calls.
• It works as follows
• First, it partitions an array into two parts,
• Then, it sorts the parts independently,
• Finally, it combines the sorted subsequences by
• a simple concatenation.

70
Quicksort (cont.)

• The quick-sort algorithm consists of the
  following three steps
• 1. Divide Partition the list.
• To partition the list, we first choose some
  element from the list for which we hope about
  half the elements will come before and half
  after. Call this element the pivot.
• Then we partition the elements so that all those
  with values less than the pivot come in one
  sublist and all those with greater values come in
  another.
•  2. Recursion Recursively sort the sublists
  separately.
•  3. Conquer Put the sorted sublists together.

71
Partition

• Partitioning places the pivot in its correct
  place position within the array.
• Arranging the array elements around the pivot p
  generates two smaller sorting problems.
• sort the left section of the array, and sort the
  right section of the array.
• when these two smaller sorting problems are
  solved recursively, our bigger sorting problem is
  solved.

72
Partition Choosing the pivot

• First, we have to select a pivot element among
  the elements of the given array, and we put this
  pivot into the first location of the array before
  partitioning.
• Which array item should be selected as pivot?
• Somehow we have to select a pivot, and we hope
  that we will get a good partitioning.
• If the items in the array arranged randomly, we
  choose a pivot randomly.
• We can choose the first or last element as a
  pivot (it may not give a good partitioning).
• We can use different techniques to select the
  pivot.

73
Partition Function

• template ltclass DataTypegt
• void partition(DataType theArray, int first,
  int last,
• int pivotIndex)
• // Partitions an array for quicksort.
• // Precondition first lt last.
• // Postcondition Partitions theArrayfirst..last
  such that
• // S1 theArrayfirst..pivotIndex-1 lt pivot
• // theArraypivotIndex pivot
• // S2 theArraypivotIndex1..last gt pivot
• // Calls choosePivot and swap.
• // place pivot in theArrayfirst
• choosePivot(theArray, first, last)
• DataType pivot theArrayfirst // copy pivot

74
Partition Function (cont.)

• // initially, everything but pivot is in
  unknown
• int lastS1 first // index of last
  item in S1
• int firstUnknown first 1 //index of 1st
  item in unknown
• // move one item at a time until unknown
  region is empty
• for ( firstUnknown lt last firstUnknown)
• // Invariant theArrayfirst1..lastS1 lt
  pivot
• // theArraylastS11..firstUnknow
  n-1 gt pivot
• // move item from unknown to proper region
• if (theArrayfirstUnknown lt pivot) //
  belongs to S1
• lastS1
• swap(theArrayfirstUnknown,
  theArraylastS1)
• // else belongs to S2
• // place pivot in proper position and mark its
  location
• swap(theArrayfirst, theArraylastS1)
• pivotIndex lastS1
• // end partition

75
Partition Function (cont.)
Invariant for the partition algorithm
76
Partition Function (cont.)
Initial state of the array
77
Partition Function (cont.)
Moving theArrayfirstUnknown into S1 by swapping
it with theArraylastS11 and by incrementing
both lastS1 and firstUnknown.
78
Partition Function (cont.)
Moving theArrayfirstUnknown into S2 by
incrementing firstUnknown.
79
Partition Function (cont.)
Developing the first partition of an array when
the pivot is the first item
80
Quicksort Function

• void quicksort(DataType theArray, int first,
  int last)
• // Sorts the items in an array into ascending
  order.
• // Precondition theArrayfirst..last is an
  array.
• // Postcondition theArrayfirst..last is
  sorted.
• // Calls partition.
• int pivotIndex
• if (first lt last)
• // create the partition S1, pivot, S2
• partition(theArray, first, last,
  pivotIndex)
• // sort regions S1 and S2
• quicksort(theArray, first, pivotIndex-1)
• quicksort(theArray, pivotIndex1, last)

81
Quicksort Analysis

• Worst Case (assume that we are selecting the
  first element as pivot)
• The pivot divides the list of size n into two
  sublists of sizes 0 and n-1.
• The number of key comparisons
• n-1 n-2 ... 1
• n2/2 n/2 ? O(n2)
• The number of swaps
• n-1 n-1 n-2 ... 1
• swaps outside of the for loop swaps inside of
  the for loop
• n2/2 n/2 - 1 ? O(n2)
• So, Quicksort is O(n2) in worst case

82
Quicksort Analysis

• Quicksort is O(nlog2n) in the best case and
  average case.
• Quicksort is slow when the array is sorted and we
  choose the first element as the pivot.
• Although the worst case behavior is not so good,
  its average case behavior is much better than its
  worst case.
• So, Quicksort is one of best sorting algorithms
  using key comparisons.

83
Quicksort Analysis
A worst-case partitioning with quicksort
84
Quicksort Analysis
An average-case partitioning with quicksort