DISCRETE MATHEMATICS Lecture 12

1
DISCRETE MATHEMATICSLecture 12

• Dr. Kemal Akkaya
• Department of Computer Science

2
Orders of Growth

• A way to measure the cost of an ALGORITHM
• A function is defined to determine the cost.
• For functions over numbers, we often need to know
  a rough measure of how fast a function grows.
• If f(x) is faster growing than g(x), then f(x)
  always eventually becomes larger than g(x) in the
  limit (for large enough values of x).
• Useful in engineering for showing that one
  design/algorithm scales better or worse than
  another.

3
Orders of Growth - Motivation

• Suppose you are designing a web site to process
  user data (e.g., financial records).
• Suppose database program A takes fA(n)30n8
  microseconds to process any n records, while
  program B takes fB(n)n21 microseconds to
  process the n records.
• Which program do you choose, knowing youll want
  to support millions of users?

4
Visualizing Orders of Growth

• On a graph, as you go to the right, the
  faster-growing function always eventually
  becomes the larger one...

fA(n)30n8
Value of function ?
fB(n)n21
Increasing n ?
5
Concept of order of growth

• We say fA(n)30n8 is (at most) order n, or O(n).
• It is, at most, roughly proportional to n.
• fB(n)n21 is order n2, or O(n2).
• It is (at most) roughly proportional to n2.
• Any function whose exact (tightest) order is
  O(n2) is faster-growing than any O(n) function.
• Later we will introduce T for expressing exact
  order.
• For large numbers of user records, the exactly
  order n2 function will always take more time.

6
Definition O(g), at most order g

• Let g be any function R?R.
• Define at most order g, written O(g), to be
  fR?R ?c,k ?xgtk f(x) ? cg(x).

• Beyond some point k, function f is at most a
  constant c times g (i.e., proportional to g).
• f is at most order g, or f is O(g), or
  fO(g) all just mean that f?O(g).
• Often the phrase at most is omitted.

7
Points about the definition

• Note that f is O(g) as long as any values of c
  and k exist that satisfy the definition.
• But The particular c, k, values that make the
  statement true are not unique Any larger value
  of c and/or k will also work.
• You are not required to find the smallest c and k
  values that work. (Indeed, in some cases, there
  may be no smallest values!)

However, you should prove that the values you
choose do work.
8
Big-O Proof Examples

• Show that 30n8 is O(n).
• Show ?c,k ?ngtk 30n8 ? cn.
• Let c31, k8. Assume ngtk8. Thencn 31n
  30n n gt 30n8, so 30n8 lt cn.
• Show that n21 is O(n2).
• Show ?c,k ?ngtk n21 ? cn2.
• Let c2, k1. Assume ngt1. Then cn2 2n2
  n2n2 gt n21, or n21lt cn2.

9
Big-O example, graphically

• Note 30n8 isnt less than n anywhere (ngt0).
• It isnt even less than 31n everywhere.
• But it is less than 31n everywhere tothe right
  of n8.

cn 31n
30n8
Value of function ?
30n8?O(n)
n
Increasing n ?
10
Useful Facts about Big O

• Big O, as a relation, is transitive f?O(g) ?
  g?O(h) ? f?O(h)
• O with constant multiples, roots, and logs...? f
  constants a,b?R, with b?0, af, f 1-b, and
  (logb f)a are all O(f).
• Sums of functionsIf g?O(f) and h?O(f), then
  gh?O(f).

11
More Big-O facts

• ?cgt0, O(cf)O(fc)O(f?c)O(f)
• f1?O(g1) ? f2?O(g2) ?
• f1 f2 ?O(g1g2)
• f1f2 ?O(g1g2) O(max(g1,g2))
  O(g1) if g2?O(g1)
• Very useful!

12
Order-of-Growth Expressions

• O(f) when used as a term in an arithmetic
  expression means some function f such that
  f?O(f).
• E.g. x2O(x) means x2 plus some function
  that is O(x).
• Formally, you can think of any such expression as
  denoting a set of functions
• x2O(x) ? g ?f?O(x) g(x) x2f(x)

13
Useful Facts about Big O

• ? f,g constants a,b?R, with b?0,
• af O(f) (e.g. 3x2 O(x2))
• fO(f) O(f) (e.g. x2x O(x2))
• Also
• x?1 O(x)
• log x O(x)
• x O(x log x)
• x log x ? O(x)
• 3 O(x)) but it is also true that 3 O(1)

14
Definition ?(g), exactly order g

• If f?O(g) and g?O(f), then we say g and f are of
  the same order or f is (exactly) order g and
  write f??(g).
• Another, equivalent definition?(g) ? fR?R
  ?c1c2kgt0 ?xgtk c1g(x)?f(x)?c2g(x)
• Everywhere beyond some
• point k, f(x) lies in between
• two multiples of g(x).

15
Rules for ?

• Mostly like rules for O( ), except
• ? f,ggt0 constants a,b?R, with bgt0, af ? ?(f),
  but ? Same as with O. f ? ?(fg) unless
  g?(1) ? Unlike O.f 1-b ? ?(f), and
  ? Unlike O. (logb f)c ? ?(f). ?
  Unlike O.
• The functions in the latter two cases we say are
  strictly of lower order than ?(f).

16
? example

• Determine whether
• Quick solution

17
Relations Between the Order-of-Growths

• Subset relations between order-of-growth sets.

R?R
?( f )
O( f )
f
?( f )
18
Review Orders of Growth

• Definitions of order-of-growth sets, ?gR?R
• O(g) ? f ? cgt0 ?k ?xgtk f(x) lt cg(x)
• ?(g) ? f ? cgt0 ?k ?xgtk f(x) cg(x)
• ?(g) ? O(g) ? ?(g)