Complexity%20Analysis:%20Examples

1
Complexity Analysis Examples
2
Recap

• Complexity analysis counts algorithm steps
• Q What is a step ?
• A Something independent of the input
  size (n)
• e.g., a multiplication, an addition, a swap
• a (recursive) function call

3
Simple Example Computing An

• Brute force method
• P 1
• for (i 1 i lt n i)
• P P A
• Complexity here is in terms of number of
  multiplications. Its O(n).

4
Computing An

• A recursive method ?
• Consider calculating A14
• A14 A7 x A7,
• A7 A3 x A3 x A,
• A3 A x A x A.
• Only 5 multiplications, not 14!

5
Computing An

• function raise(int A, int n)
• if (n 0) return 1
• if (n 1) return A
• temp raise(A, n/2)
• if (n mod 2 0)
• return temp temp
• else
• return temp temp A

6
Useful Fact

• The number of times n can be divided by 2 before
  reaching 0 or 1 is log2(n)

7
Computing An

• function raise(int A, int n)
• if (n 0) return 1
• if (n 1) return A
• temp raise(A, n/2)
• if (n mod 2 0)
• return temp temp
• else
• return temp temp A
• Complexity is determined by number of calls to
  raise.
• Its the number of times n can be divided by 2
  before reaching 0 or 1.
• So the complexity is O(log2n) O(log n).

8
Another Example Binary Search

• int binary_search(int n, int a, int key)
• int lo -1
• int hi n
• while (hi - lo ! 1)
• int mid (hi lo) / 2
• if (amid lt key)
• lo mid
• else
• hi mid
• return lo

9
Another Example Binary Search

• int binary_search(int n, int a, int key)
• int lo -1
• int hi n
• while (hi - lo ! 1)
• int mid (hi lo) / 2
• if (amid lt key)
• lo mid
• else
• hi mid
• We half the array and we do one comparison (step)
  each time
• Its the number of times n can be divided by 2
  O(log2n)

10
Methods

• Method 1 Guess solution or do the computations
• Method 2 Iterate T(n) recurrence

11
Binary Search Recurrence Equation

• T(n) T(n/2) 1
• T(1) 1
• T(n) 1 T(n/2) 1 1 T(n/4)
• 1 1 1 T(n/8)
• 1 1 1 . T(n/n) log(n)

12
More Recurrence Equations

• int recursive(int n)
• if (n lt 1)
• return 1
• else
• return recursive(n - 1) recursive(n - 1)
• T(n) 2T(n - 1)
• T(1) 1
• 
  

13
Recurrence Equations

• int recursive(int n)
• if (n lt 1)
• return 1
• else
• return recursive(n - 1) recursive(n - 1)
• T(n) 22T(n-2) 222T(n-3) 2n T(n -
  n) 2n
• 
  

14
Recurrence Equations

• MergeSort
• T(n) - time for merge sort ?
• Calls itself recursively on the two halves, so
  2T(n/2)
• Merges the two sorted halves (n steps)
• T(n) 2T(n/2) n
• T(1) 1

15
Recurrence Equations

• T(n) time for merge sort
• T(n) n 2T(n/2) n 2 (n/2 2T(n/4))
• n n 4T(n/4) n n 4(n/4 2T(n/8))
  
• n n n 8(n/8 2T(n/16))
• n n n n . base case 2logn
  T(n/2logn)
• n log(n)
• Assume n is a power of 2

16
Methods

• Method 1 Guess solution or do the computations
• Method 2 Iterate T(n) recurrence
• Method 3 Recursion trees

17
Recurrence Trees Method 3

• int fibonacci (int n)
• if (n lt 1) return 1
• return fibonacci(n 1) fibonacci (n - 2)
• 1 1 2 3 5 8 13

18
Best, average, worst case BubbleSort

• for (i n i gt 1 i--) / Line 1 /
• for (j 1 j lt i j) / 2 /
• if (aj-1 gt aj) / 3 /
• temp aj-1 / 4 /
• aj-1 aj / 5 /
• aj temp / 6 /
• Line 1 Executed n times.
• Lines 2 3 Executed n (n-1) (n-2) 2
  1 n(n 1) / 2 times.
• Lines 4, 5 6 Executed between 0 and n(n 1) /
  2 times depending on how often a j-1 gt a j .

19
Best, average, worst case

• Let p be the probability of a j-1 gt a j .
• In the worst case (i.e. maximum T(n)), p 1 for
  all j.
• In the best case (i.e. minimum T(n)), p 0 for
  all j.
• On average, p 0.5.

20
Sequential Search

• int seq_search(int n, int a, int key)
• int i 0
• while (i lt n ai ! key)
• i
• return i
• Whats the complexity?

21
More For loops

• void matmpy(int n, int c, int a, int b)
• int i, j, k
• for (i 0 i lt n i)
• for (j 0 j lt n j)
• cij 0
• for (k 0 k lt n k)
• cij aik bkj
• Whats the complexity?

22
More For Loops

• / Exercise 1 /
• void mystery(int n)
• int i, j, k
• for (i 1 i lt n - 1 i)
• for (j i 1 j lt n j)
• for (k 1 k lt j k)
• / Some statement taking O(1) time /

23
More For Loops

• / Exercise 2 /
• void veryodd(int n)
• int i, j, x, y
• x 0
• y 0
• for (i 1 i lt n i)
• if (i 2 1)
• for (j i j lt n j)
• x x 1
• for (j 1 j lt i j)
• y y 1

24
Exercising loops

• sum 0
• for (i 0 i lt n i)
• for (j 0 j lt i j)
• sum
• Whats the complexity?

25
Exercising loops

• sum 0
• for (i 0 i lt n i)
• for (j 0 j lt i j)
• sum
• Whats the complexity?
• Si 0 to n i n(n1)/2

26
Exercising loops

• sum 0
• for (i 0 i lt n i2)
• for (j 0 j lt i j)
• sum
• Whats the complexity?

27
Exercising loops

• sum 0
• for (i 0 i lt n i2) //Do log(n) times
• for (j 0 j lt n j) //Do n times
• sum
• Assume n is a power of 2
• Si 0 to logn n n log(n)

28
Exercising loops

• sum 0
• for (i 0 i lt n i2)
• for (j 0 j lt k j)
• sum
• Whats the complexity?
• Assume n is a power of 2

29
Exercising loops

• sum 0
• for (i 0 i lt n i2) //Do log(n) times
• for (j 0 j lt k j) //Do k times
• sum
• Assume n is a power of 2
• Si 0 to logn 2i 2logn1 1 2n -1