Analysis and Complexity of Algorithms

1
Analysis and Complexity of Algorithms
2
Analysis of Algorithms

• Quantifying the resources required.
• Measures of resource utilization (efficiency)
• Execution time ? time complexity
• Memory space ? space complexity
• Observation
• The larger the input data the more the resource
  requirement Complexities are functions of the
  amount of input data (input size).

Refer to first few chapters of the book
Introduction To Algorithms by Cormen,
Leiserson, Rivest and Stein
3
Yardstick?

• The same algorithm will run at different speeds
  and will require different amounts of space when
  run on different computers, different programming
  languages, different compilers.
• Algorithms usually consume resources in some
  fashion that depends on the size of the problem
  they solve.
• Need a machine independent complexity measure
  that is a function of input size, n. (Also we are
  interested at asymptotic behaviour, that is, when
  n becomes large.

4
Analyzing SelSort
int max_loc(int x, int k, int size) int j,
pos pos k for (jk1 jltsize
j) if (xj gt xpos)
pos j return pos int
selsort (int x, int size) int k, m,
temp for (k0 kltsize-1 k)
m max_loc(x, k, size)
temp xk xk xm
xm temp

• Computer 1
• f1(n) 0.0007772 n2 0.00305 n 0.001
• Computer 2
• f2(n) 0.0001724 n2 0.00040 n 0.100
• Note Both are quadratic functions of n
• The shape of the curve that expresses the running
  time as a function of the problem size stays the
  same.
• We say that Selection Sort is of complexity Order
  n2 or O(n2)

5
Complexity classes

• The running time for different algorithms fall
  into different complexity classes.
• Each complexity class is characterized by a
  different family of curves.
• All curves in a given complexity class share the
  same basic shape. (In the Asymptotic sense)
• The O-notation is used for talking about the
  complexity classes of algorithms.

6
Order of Complexity, O- notation

• For the quadratic function f(n)
  an2 bn cwe will say that f(n) is O(n2).
• We focus on the dominant term, and ignore the
  lesser terms then throw away the coefficient.
  Asymptotic Analysis
• Since constants are finally not important we may
  assume that each machine instruction takes one
  unit of time when we analyze the complexity of an
  algorithm.
• We may sometimes abstract this to unit time
  operators like add, sub, multiply, etc and do an
  operator count.
• We can perform worst case or average case
  analysis.

7
How execution time is affected by various
complexity measures

• Assume speed S is 107 instructions per second.

COMPLEX I TY
8
Maximum size solvable within 1 hour
9
Formal Definition

• T(N) O(f(N)) if there are positive constants c
  and n0 such that T(N) ? c f(N) when N ?
  n0.Meaning As N increases, T(N) grows no
  faster than f(N).The function T is
  eventually bounded by some multiple of f(N). f(N)
  gives an upper bound in the behavior of T(N).

• logen O(n)
• n2 O(n2).
• n2 O(n3).
• nlogn O(n2).
• 3 n2 5n 1 O(n2)

10
Some Worst Case Complexities

• Linear Search in an array ?
• Binary Search ?
• Selection Sort ?
• Mergesort ?
• Quicksort ?
• Stack ADT Push and Pop are ?
• Queue using a singly linked list ?? (What about
  a doubly linked list??)
• Multiplying 2 square matrices?
• The primes that we wrote in class?

11
Some Worst Case Complexities

• Linear Search in an array O(n)
• Binary Search O(log n)
• Selection Sort O(n2)
• Stack ADT Push and Pop are O(1)
• Queue using a singly linked list
• Insert (Enqueue) O(1)
• Delete (Dequeue) O(n)

12
More definitions

• T(N) ?(g(N)) if there are positive constants c
  and n0 such that T(N) ?c f(N) when N ? n0.
  Meaning As N increases, T(N) grows no slower
  than g(N) T(N) grows at least as fast as g(N).
• T(N) ?(h(N)) if and only if T(N) O (h(N)) and
  T(N) ?(h(N))Meaning As N increases, T(N)
  grows as fast as h(N).
• T(N) o(p(N)) Meaning As N increases, T(N)
  grows slower than p(N). lim
  n??T(N)/p(N) 0.

logen O(n) n10 o(2n) 3 n2 5n 1
?(n2)
13
Space Complexity

• Measures the work space or space required in
  addition to input storage
• Selection Sort O(1)
• Binary Search
• Recursive algorithm O(log n) Recursion Stack
• Iterative algorithm O(1)
• Mergesort O(n) Why Additional array. Here
  recursion stack is of depth log n
• Quicksort O(n) Why recursion stack in the
  worst case