Growth%20of%20Functions%201

1
Asymptotic Growth Rate
2
Function Growth

• The running time of an algorithm as input size
  approaches infinity is called the asymptotic
  running time
• We study different notations for asymptotic
  efficiency.
• In particular, we study tight bounds, upper
  bounds and lower bounds.

3
Outline

• Why do we need the different sets?
• Definition of the sets O, Omega and Theta
• Classifying examples
• Using the original definition
• Using limits
• General Properties
• Little Oh
• Additional properties

4
The sets and their use big Oh

• Big oh - asymptotic upper bound on the growth
  of an algorithm
• When do we use Big Oh?
• Theory of NP-completeness
• To provide information on the maximum number of
  operations that an algorithm performs
• Insertion sort is O(n2) in the worst case
• This means that in the worst case it performs at
  most cn2 operations
• Insertion sort is also O(n6) in the worst case
  since it also performs at most dn6 operations

5
The sets and their use Omega

• Omega - asymptotic lower bound on the growth of
  an algorithm or a problem
• When do we use Omega?
• 1. To provide information on the minimum number
  of operations that an algorithm performs
• Insertion sort is ?(n) in the best case
• This means that in the best case its instruction
  count is at least cn,
• It is ?(n2) in the worst case
• This means that in the worst case its instruction
  count is at least cn2

6
The sets and their use Omega cont.

• 2. To provide information on a class of
  algorithms that solve a problem
• Sort algorithms based on comparison of keys are
  ?(nlgn) in the worst case
• This means that all sort algorithms based only on
  comparison of keys have to do at least cnlgn
  operations
• Any algorithm based only on comparison of keys to
  find the maximum of n elements is ?(n) in every
  case
• This means that all algorithms based only on
  comparison of keys to find maximum have to do at
  least cn operations

7
The sets and their use - Theta

• Theta - asymptotic tight bound on the growth
  rate of an algorithm
• Insertion sort is ??(n2) in the worst and
  average cases
• The means that in the worst case and average
  cases insertion sort performs cn2 operations
• Binary search is ?(lg n) in the worst and average
  cases
• The means that in the worst case and average
  cases binary search performs clgn operations
• Note We want to classify an algorithm using
  Theta.
• In Data Structures used Oh
• Little oh - used to denote an upper bound that
  is not asymptotically tight. n is in o(n3). n is
  not in o(n)

8
The functions

• Let f(n) and g(n) be asymptotically nonnegative
  functions whose domains are the set of natural
  numbers N0,1,2,.
• A function g(n) is asymptotically nonnegative, if
  g(n)³0 for all n³n0 where n0ÎN

9
Asymptotic Upper Bound big O
c ? g (n)
Graph shows that for all n ? N, f(n) ? cg(n)
f (n)
Why only for n ? N ? What is the
purpose of multiplying by c gt 0?
N
f (n) O ( g ( n ))
10
Asymptotic Upper Bound O

• Definition Let f (n) and g(n) be asymptotically
  non-negative functions. We say f (n) is in O (
  g ( n )) if there is a real positive constant c
  and a positive Integer N such that for every n ³
  N 0 f (n) c ? g (n ).
• Or using more mathematical notation
• O ( g (n) ) f (n ) there exist positive
  constant c and a positive integer N such that
  0 f( n) c ? g (n ) for all n ³ N

11
n2 10 n ? O(n2) Why?
take c 2N 10 2n2 gt n2 10 n for all ngt10
1400
1200
1000
n2 10n
800
600
2 n2
400
200
0
0
10
20
30
12
Does 5n2 ÎO(n)?

• Proof From the definition of Big Oh, there must
  exist cgt0 and integer Ngt0 such that 0 5n2cn
  for all n³N.
• Dividing both sides of the inequality by ngt0 we
  get
• 0 52/nc.
• 2/n 2, 2/ngt0 becomes smaller when n increases
• There are many choices here for c and N.
• If we choose N1 then c ³ 52/1 7.
• If we choose c6, then 0 52/n6. So N ³ 2.
• In either case (we only need one!) we have a cgto
  and Ngt0 such that 0 5n2cn for all n ³ N. So
  the definition is satisfied and 5n2 ÎO(n)

13
Does n2Î O(n)? No.

• We will prove by contradiction that the
  definition cannot be
• satisfied.
• Assume that n2Î O(n).
• From the definition of Big Oh, there must exist
  cgt0 and integer Ngt0 such that 0 n2cn for all
  n³N.
• Dividing the inequality by ngt0 we get 0 n c
  for all n³N.
• n c cannot be true for any n gtmaxc,N ,
  contradicting our assumption
• So there is no constant cgt0 such that nc is
  satisfied for all n³N, and n2? O(n)

14
O ( g (n) ) f (n ) there exist positive
constant c and positive integer N such that
0 f( n) c ? g (n ) for all n ³ N

• 1,000,000 n2 ? O(n2) why/why not?
• (n - 1)n / 2 ? O(n2) why /why not?
• n / 2 ? O(n2) why /why not?
• lg (n2) ? O( lg n ) why /why not?
• n2 ? O(n) why /why not?

15
Asymptotic Lower Bound, Omega W
f (n)
c g (n)
N
f(n) ? ( g ( n ))
16
Asymptotic Lower Bound W

• Definition Let f (n) and g(n) be asymptotically
  non-negative functions. We say f (n) is W ( g (
  n )) if there is a positive real constant c and a
  positive integer N such that for every n ³ N
  0 c g (n ) f ( n).
• Or using more mathematical notation
• W ( g ( n )) f (n) there exist positive
  constant c and a positive integer N such that
  0 c g (n ) f ( n) for all n ³ N
  

17
Is 5n-20Î W (n)?

• Proof From the definition of Omega, there must
  exist cgt0 and integer Ngt0 such that 0 cn
  5n-20 for all n³N
• Dividing the inequality by ngt0 we get 0 c
  5-20/n for all n³N.
• 20/n 20, and 20/n becomes smaller as n grows.
• There are many choices here for c and N.
• Since c gt 0, 5 20/n gt0 and N gt4
• For example, if we choose c4, then 5 20/n ³ 4
  and N ³ 20
• In this case we have a cgto and Ngt0 such that 0
  cn 5n-20 for all n ³ N. So the definition is
  satisfied and 5n-20 Î W (n)

18
W ( g ( n )) f (n) there exist positive
constant c and a positive integer N such that
0 c g (n ) f ( n) for all n ³ N

• 1,000,000 n2 ? W (n2) why /why not?
• (n - 1)n / 2 ? W (n2) why /why not?
• n / 2 ? W (n2) why /why not?
• lg (n2) ? W ( lg n ) why /why not?
• n2 ? W (n) why /why not?

19
Asymptotic Tight Bound Q
d ? g (n)
f (n)
c ? g (n)
f (n) Q ( g ( n ))
N
20
Asymptotic Bound Theta Q

• Definition
• Let f (n) and g(n) be asymptotically
  non-negative functions. We say f (n) is Q( g ( n
  )) if there are positive constants c, d and a
  positive integer N such that for every n ³ N 0
  c ? g (n ) f ( n) d ? g ( n ).
• Or using more mathematical notation
• Q ( g ( n )) f (n) there exist positive
  constants c, d and a positive integer N such
  that 0 c ? g (n ) f ( n) d ? g ( n ).
  for all n ³ N

21
More on Q

• We will use this definition Q (g (n)) O( g
  (n) ) Ç W ( g (n) )

22

• We show

23
(No Transcript)
24
(No Transcript)
25
More Q

• 1,000,000 n2 ? Q(n2) why /why not?
• (n - 1)n / 2 ? Q(n2) why /why not?
• n / 2 ? Q(n2) why /why not?
• lg (n2) ? Q ( lg n ) why /why not?
• n2 ? Q(n) why /why not?

26
Limits can be used to determine Order

• c then f (n) Q ( g (n))
  if c gt 0
• if lim f (n) / g (n) 0 or c gt 0 then f
  (n) o ( g(n))
• or c gt 0 then f (n) W ( g (n))
• The limit must exist

n
27
Example using limits
28
LHopitals Rule

• If f(x) and g(x) are both differentiable with
  derivatives f(x) and g(x), respectively, and if

29
Example using limits
30
Example using limits
31
Example using limits
Î
k
n
O

where k is
a positiv
e integer.

n
n
ln
2
e
2



n
n
n
n
ln
ln
2
2
e
e
)'
(
ln
2
2
2
(
)'

ln2
-
k
k
1
n
kn


lim
lim
n
n
ln
2
2
2




n
n
-
-
2
k
k
k
n
k
(
)
!
1




lim
.
.
.
lim
0
n
n
k
2
ln
ln
2
2
2
2




n
n
32
Another upper bound little oh o

• Definition Let f (n) and g(n) be
  asymptotically non-negative functions.. We say f
  ( n ) is o ( g ( n )) if for every positive
  real constant c there exists a positive integer
  N such that for all n ³ N 0 f(n) lt c ? g (n
  ).
• o ( g (n) ) f(n) for any positive
  constant c gt0, there exists a positive integer N
  gt 0 such that 0 f( n) lt c ? g (n ) for all n ³
  N
• little omega can also be defined

33
main difference between O and o

• O ( g (n) ) f (n ) there exist positive
  constant c and a positive integer N such
  that 0 f( n) ? c ? g (n ) for all n ³ N
  
• o ( g (n) ) f(n) for any positive
  constant c gt0, there exists a positive integer
  N such that 0 f( n) lt c ? g (n ) for all n ³ N
  
• For o the inequality holds for all positive
  constants.
• Whereas for O the inequality holds for some
  positive constants.

34
Lower-order terms and constants

• Lower order terms of a function do not matter
  since lower-order terms are dominated by the
  higher order term.
• Constants (multiplied by highest order term) do
  not matter, since they do not affect the
  asymptotic growth rate
• All logarithms with base b gt1 belong to ?(lg n)
  since

35
Asymptotic notation in equations

• What does n2 2n 99 n2 Q(n)
• mean?
• n2 2n 99 n2 f(n)
• Where the function f(n) is from the set Q(n).
  In fact f(n) 2n 99 .
• Using notation in this manner can help to
  eliminate non-affecting details and clutter in an
  equation.
• ? 2n2 5n 21 2n2 Q (n ) Q (n2)

36
Transitivity

• If f (n) Q (g(n )) and g (n) Q (h(n ))
  then f (n) Q (h(n )) .
• If f (n) O (g(n )) and g (n) O (h(n ))
  then f (n) O (h(n )).
• If f (n) W (g(n )) and g (n) W (h(n ))
  then f (n) W (h(n )) .
• If f (n) o (g(n )) and g (n) o (h(n ))
  then f (n) o (h(n )) .
• If f (n) w (g(n )) and g (n) w (h(n ))
  then f (n) w (h(n ))

37
Reflexivity

• f (n) Q (f (n )).
• f (n) O (f (n )).
• f (n) W (f (n )).
• o is not reflexive

38
Symmetry and Transpose symmetry

• Symmetryf (n) Q ( g(n )) if and only if g
  (n) Q (f (n )) .
• Transpose symmetry f (n) O (g(n )) if and
  only if g (n) W (f (n )). f (n) o (g(n ))
  if and only if g (n) w (f (n )).

39
Analogy between asymptotic comparison of
functions and comparison of real numbers.

• f (n) O( g(n)) a b
• f (n) W ( g(n)) a ³ b
• f (n) Q ( g(n)) a b
• f (n) o ( g(n)) a lt b
• f (n) w ( g(n)) a gt b

40
Not All Functions are Comparable

• The following functions are not asymptotically
  comparable

41
(No Transcript)
42
Is O(g(n)) ?(g(n)) ? o(g(n))?

• We show a counter example
• The functions are
• g(n) n
• and
• f(n)?O(n) but f(n) ? ?(n) and f(n) ? o(n)

Conclusion O(g(n)) ? ?(g(n)) ? o(g(n))
43
General Rules

• We say a function f (n ) is polynomially bounded
  if f (n ) O ( nk ) for some positive
  constant k
• We say a function f (n ) is polylogarithmic
  bounded if f (n ) O ( lgk n) for some
  positive constant k
• Exponential functions
• grow faster than positive polynomial functions
• grow faster than polylogarithmic functions