Lecture 16 Big Oh and Sorting

1
Lecture 16 Big Oh and Sorting

• Comp 208 Computers in Engineering
• Yi Lin
• Winter, 2007

2
Lecture 16

• Objectives
• Learn about how to evaluate algorithms, O(..)
• Learn about sorting
• Selection sort
• Bubble sort
• Insertion sort
• Learn enough in-depth C to implement these

3
Algorithms

• What is an Algorithm?
• Algorithm is a well defined, systematic way to
  achieve a goal.
• Any method, technique, or paradigm to solve a
  problem.

4
Classification of Algorithms

• Numerical e.g. numerical integration
• root finding,
• solving differential equations
• Non-Numerical e.g. sorting, searching

5
Types of Algorithms

• 1. Direct Computation (e.g. Income Tax)
• 2. Enumeration (e.g. Sequential Search)
• 3. Divide and Conquer
• (e.g. Binary Search)
• 4. Iteration
• (e.g. Series and Taylor Expansions)
• 5. Trial Error (e.g. Newton Raphson)
• 6. Simulation
• (using Random Number Generators)

6
Algorithms

• Analysis of Algorithms
• Computer Scientists like to measure the
  performance of algorithms
• Accuracy
• Speed
• number of operations ( - / lt gt )
• Space
• Data memory usage (RAM disk)
• Code size (RAM disk)

7
Algorithmic Speed

• The study of algorithmic speed is often expressed
  in terms of the algorithm's run-time performance
  in the
• Best Case
• Average Case
• Worst Case Most interest

8
Big O Notation

• The Worst Case scenario leads to the measurement
  called Big Oh.
• The O refers to Order of magnitude a bounded
  approximation.
• This is written
• O(n) or O(n log n) or
• O(n2) or O(. . )

9
Big O Notation

1. O(1) The best time - Constant time
2. O(loge n) Next best time - Logarithmic time
3. O(n) Next best time - Linear time
4. O(nlogen) Next best time - Multiplied
   Logarithmic time
5. O(nk) Next is - Quadratic time (k is constant)
6. O(nm) The Worst Exponential time (m is
   variable)

10
Big O

• In many cases the Big O notation may have a
  constant that modifies the order.
• For instance
• O(gn) or O(n h) or O(kn log n)
• where g, h, and k are constants
• In general we can ignore the constants.

11
Big O Examples

• Calculate the Big O for the following algorithm
• y0
• for (x 0 x lt N x)
• y y x 10

12
Big O Examples

• The quick answer is that the algorithm and the
  program is O(N).
• There are 3N2 single statements. O(3N2)O(N)
• A more detailed measure might count the
  operations.
• N times of , , for y calculation
• N times of , lt for the loop
• Total 5N 2 if all operations equal
• O(5N2) O(N)

13
Big O Examples

• Calculate the Big O for the following algorithm
• for(i 2 i lt N i 2)
• Ai i

• Calculate the Big O for the following algorithm
• for(i N i gt 0 i / 2)
• Ai i

The order is O(logN)
14
Big O Examples

• Calculate the Big O for the following algorithm
• for(i 2 i lt N i i)
• Ai i

• Calculate the Big O for the following algorithm
• for(i 0 i lt N i)
• for(j 2 j lt N j 2) Ai ij

The order is O(log(logN))
The order is O(N log N)
15
Big O Examples

• Calculate the Big O for the following algorithm
• z0
• for (x 0 x lt N x)
• for (y 0 y lt M y)
• z z x y

The order is O(NM)
If MN, the order is O(N2)
16
Big O Examples

• Calculate the Big O for the following algorithm
• z0
• for (w0 wltK w)
• for (x 0 x lt N x)
• z z x
• for (y 0 y lt M y)
• z z y

The order is O(K(NM))
17
Big O Examples

• Calculate the Big O for the following algorithm
• z0
• for (w0 wltK w)
• for (x 0 x lt N x)
• z z x
• for (y 0 y lt M y)
• z z y

The order is O(KNM)
18
Sorting (selection, bubble, insertion)

• Comp 208 Computers in Engineering
• Yi Lin
• Winter, 2007

19
Motivation

• The motivations for sorting are
• Put a list of things in order such that it is
  easy for you to search something.
• For example
• Give a pile of POKER missing one card, sort it in
  order to find out which card is missing.
• I want to sort your midterm grades in order to
  see who get more than 90.

4 2 3 1
1 2 3 4
20
Selection Sorting

• Comp 208 Computers in Engineering
• Yi Lin
• Winter, 2007

21
What would you do?

• If you had to sort a stack of card what would you
  do?
• Possibly find the card which you know goes on
  top, then move it to the top
• Repeat this process for the second card, third
  card until there are no more cards.
• Selection sort does this, sort-of

22
Selection sort
32 15 38 56 23 35 74

• How many swaps in total?

15 32 38 56 23 35 74
First iteration
15 23 38 56 32 35 74
2nd iteration
15 23 32 56 38 35 74
3rd iteration
15 23 32 35 38 56 74
4th iteration
23
Example 1 with two functionsfindSmallest, swap

• // find the smallest element in the array
• int findSmallest(int b, int size)
• int i, pos
• pos 0
• for(i1 iltsize i)
• if(bi lt bpos)
• pos i
• return pos

• // swap two integer pointers
• void swap(int a, int b)
• int tmp
• tmp a
• a b
• b tmp

24
Example 2 selectionSort()

• void selectionSort(int a, int size)
• int i, j, pos
• / find the No. i smallest element and swap it
  with the element at No. i position /
• for(i0 iltsize i)
• // find the No. i smallest element
• pos findSmallest(ai, size-i)
• / swap it with the element at the No. i
  position /
• swap(ai, aposi)

• int main() // exmaple 2
• int i
• int a4 4, 2, 3, 1
• selectionSort(a, 4)
• // print the sorted array
• for(i0 ilt4 i)
• printf("d ", ai)
• return 0