CS 60: Theory of Algorithms ANALYSIS TOOLS

1
CS 60 Theory of AlgorithmsANALYSIS TOOLS

• Jay R.C. Fernandez, M.S.
• Instructor

2
Objectives

• To define Running Time and be familiar with the
  issues in measure this
• To apply a general methodology for measuring
  Running Time
• To be familiar with Pseudo-Coding
• To be familiar with common Algorithm
  Proof/Justification Derivations

3
Objectives

• To be able to analyze Algorithms and measure its
  Running Time
• To represent Running Times in mathematical
  notation
• To compare Algorithms based on their Running Times

4
Running Time

• the time for an algorithm to accomplish a certain
  task
• How do we measure it?
• Empirical Method
• Theoretical Method

5
Running Time

• Empirical Measuring
• Actual execution of the algorithm
• Considerations
• Hardware Environment Processor, Memory, Disk
  Access
• Software Environment OS, Programming Language
  used, Compiler, Interpreter

6
Running Time

• Theoretical Measuring
• determining mathematically the quantity of
  resources needed by each algorithm as a function
  of the size of the instances considered

7
Running Time

• Theoretical Measuring
• assuming all operations are accomplished in a
  uniform time, the running time is dependent on a
  primary parameter n, usually the size of the
  input to be processed.

8
(No Transcript)
9
Running Time

• Consideration in Measuring RT
• Experiments can be done only on a limited set of
  test inputs
• Comparison is significant when two algorithms are
  performed in the same hardware and software
  environments
• Necessity to implement and execute an algorithm
  in order to study its running time

10
Properties of Algorithms

• General Methodology
• Take into account all possible inputs
• Disregard hardware and software environment
  factors
• Study high-level description of the algorithm
  instead of implementing and executing algorithm

11
Running Time

• Unit of Measure associate an algorithm with a
  function f(n) that characterizes the running time
  of the algorithm as a function of the input size
  n.
• Examples n, n2

12
Running Time

• Algorithm A runs in time proportional to n2.
• Interpretation Running Time of Algorithm A on
  any input size n never exceeds cn, where c is a
  constant that depends on the hardware and
  software environment

13
Running Time

• Principle of Invariance
• Two different implementations of the same
  algorithm will not differ in efficiency by more
  than a multiplicative constant.

14
Running Time

• If two implementations take t1(n) and t2(n) sec.
  To solve an instance of size n, then there will
  always exist positive constants c and d such that
• t1(n) lt ct2(n) t2(n) lt dt1(n)

15
Running Time

• O(1)
• constant running time
• executed once or a few times
• O(n)
• linear running time
• running time is directly proportional to size n

16
Running Time

• O(n log n)
• logarithmic running time
• running time of algorithms that break a big
  instance into smaller instance (solving them
  independently and then combing the solution)

17
Running Time

• O(nx) x?1
• polynomial running time
• O(an) agt1
• exponential running time
• verifying possible combinations

18
Running Time
O(an)
O(nx)
O(n)
O(n log n)
Time to Finish
O(1)
Input Size (n)
19
Analysis

• Pseudo-Code
• a mixture of natural language and high-level
  programming constructs that describe the main
  ideas behind a generic implementation of a data
  structure or algorithm

20
Analysis

• Example of a Pseudo-Code
• Algorithm arrayMax(A,n)
• Input An array A storing n integers
• Output The maximum element in A
• current Max ? A0
• for i ? 1 to n-1 do
• if currentMax lt Ai then
• currentMax ? Ai
• return current Max

21
Analysis

• Notation
• operators
• ? Assignment
• ,-,,/ Math Operators
• ,lt,gt,lt,gt Logical Operators
• method/function declaration
• Algorithm ltnamegt (ltparam1gt, ltparam2gt,..)

22
Analysis

• Notation
• decision structures
• if ltcondgt then ltaction/sgt else ltaction/sgt
• loop structures
• while ltcondgt do
• ltaction/sgt
• for ltvargt ? ltstartgt to ltendgt do ltactionsgt

23
Analysis

• Notation
• array indexing
• Ai, A0, An-1
• method/function returns
• return ltvaluegt

24
Analysis

• Mathematical Review
• logba c iff a bc
• logbac logba logbc
• logba/c logba logbc
• logbac c logba
• logba (logca ) / logcb

25
Analysis

• Mathematical Review
• blogca alogcb
• (ba)c bac
• babc bac
• ba/bc ba-c

26
Analysis

• we perform our analysis on the high-level code or
  Pseudo-Code by counting primitive operations
  independent from any programming language

27
Analysis

• Primitive Operations
• assigning a value to a variable
• calling a method/function
• performing an arithmetic operation
• comparing 2 numbers
• indexing into an array
• following an object reference
• returning from a method

28
Analysis
worse-case time
average-case time
Running Time
best-case time
Input Instance
29
Analysis

• Best Case minimum number of primitive operations
  executed by the algorithm
• Worst Case maximum number of primitive
  operations executed by the algorithm
• Average Case average number of primitive
  operations executed by the algorithm

30
Analysis

• General Methods Issues
• Is the level of detail really needed?
• How important is it to figure out the exact
  number of primitive operations?
• What primitive operations should be considered?

31
Analysis

• Theoretical Methodology Asymptotic Notation
• O() Big O Notation
• ?() Big Omega Notation
• ?() Big Theta Notation

32
Asymptotic Notation

• Big O Notation
• Let f(n) and g(n) be functions mapping
  non-negative integers to real numbers. We say
  that f(n) is O(g(n)) if there is a real constant
  cgt0 and an integer constant n0?1 such that
  f(n)?cg(n) for every n ? n0.

33
Asymptotic Notation
cg(n)
f(n)
Running Time
n0
Input Size
34
Asymptotic Notation

• Example
• Show that 7n-3 is O(n)
• Solution find c gt 0 constant n0 ? 1 such that
  7n-3 ? cn for every integer n ? n0.
• Let c 7 and n0 1.
• 7(1)-3 ? 7(1)
• 7-3 ? 7
• 4 ? 7

35
Asymptotic Notation

• Asymptotic Rules
• f(n) is O(a f(n)) for any constant agt0
• If f(n) ? g(n) and g(n) is O(h(n)), then f(n) is
  O(h(n))
• If f(n) is O(g(n)) and g(n) is O(h(n)), then f(n)
  is O(h(n)
• f(n) g(n) is O(max f(n), g(n))

36
Asymptotic Notation

• Asymptotic Rules
• If g(n) is O(h(n), then f(n) g(n) is O(f(n)
  h(n))
• If g(n) is O(h(n)), then f(n)g(n) is O(f(n)h(n))
• If f(n) is a polynomial of degree d (i.e. f(n)
  a0 a1n adnd), then f(n) is O(nd)

37
Asymptotic Notation

• Asymptotic Rules
• nx is O(an) for any fixed xgt0 and agt1
• log nx is O(log n) for any fixed xgt0
• logx is O(ny) for any fixed constants xgt0 and ygt0

38
Asymptotic Notation

• Example
• g(n) 2n3 4n2 log n
• Show that g(n) is O(n3)
• log n is O(n) Rule 10
• 4n2 log n is O(4n3) Rule 6
• 2n3 4n2 log n is O(2n3 4n3) Rule 5
• 2n3 4n3 is O(n3) Rule 7
• 2n3 4n2 log n is O(n3) Rule 3

39
Asymptotic Notation

• Is the Big-O enough?
• No. The Big-O notation defines the upper bounds
  on the amount of resources required and not the
  lower bounds.
• You may reach Manila from Cagayan de Oro by
  plane in no longer than 5 days. True but
  Inaccurate

40
Asymptotic Notation

• Big Omega Notation ?()
• Let f (n) and g(n) be functions mapping
  non-negative integers to real numbers.
• f (n) is ?(g(n))
• if there are constants c and n0 such that
• f (n) ? cg(n) when n ? n0.
• defines the lower bounds on the amount of
  resources required.

41
Asymptotic Notation
f(n)
cg(n)
Running Time
n0
Input Size
42
Asymptotic Notation

• Big Theta Notation ?()
• f (n) is ?(g(n))
• if and only if
• f (n) O(g(n)) and
• f (n) ?(g(n))

43
Asymptotic Notation

• Homework
• Suppose we have two algorithms A and B. A has a
  running time of 3n2 f(n), where f(n) is a
  linear function in n. B has a running time of
  20n log n 3n 6.
• Explain clearly why it is not important to
  explicitly know what the function f(n) is when we
  want to compare the two algorithms.
• Under what conditions do we really need to know
  function f(n) to compare the running time of the
  two algorithms