Algorithms and Discrete Mathematics 20072008

1
Algorithms and Discrete Mathematics 2007/2008

• Lecture 4-5
• Big-O notation

Ioannis Ivrissimtzis
02-Nov-2007
2
Overview of the lecture

• Time complexity
• The Big-O notation

3
Time complexity

• For an input of given size n, how much time does
  it take a computer to
• solve a problem using an algorithm?
• This question is related to the time complexity
  of the algorithm.
• (Questions related to the required computer
  memory are related to
• space complexity.)
• Time complexity is described in terms of the
  number of basic
• operations required. Examples of such operations
  are
• Additions
• Multiplications
• Comparison of two numbers

4
Time complexity

• Example 5.1
• We run in our PC an algorithm which requires 100n
  basic operations.
• For the same problem, a 1,000 times faster
  supercomputer, runs an
• algorithm which requires n2 basic operations.
• Which computer will finish first for n
  1,000,000 ?

5
Time complexity

• Example 5.2
• Two algorithms solve the same problem. For an
  input of size n, they
• require 100n and n2 basic operations,
  respectively.
• Which algorithm should we use if there is an
  application specific
• constraint of 1000 basic computations?
• Which algorithm should we use if we upgrade our
  computer and can
• afford 1,000,000 basic computations?

6
Time complexity

• Example 5.3
• An algorithm requires 100n n2 basic operations.
  
• For n 10, the most computationally expensive
  part of the algorithm is
• the 100n.
• For n 1000, the most computationally expensive
  part of the algorithm
• is the n2.

7
Overview of the lecture

• Time complexity
• The Big-O notation

8
The Big-O notation

• Let f and g be functions from the set of integers
  or the set of real
• numbers to the set of real numbers. We say that
  f(x) is O(g(x)) if there
• are constants C and k such that
• whenever x gt k. This is read as f(x) is
  big-oh of g(x).
• Rosen, p.180

9
The Big-O notation

• The definition says that after a certain point,
  namely after k, the
• absolute value of f(x) is at most C times the
  absolute value of g(x).
• The absolute value of a number, is the number
  without its sign.
• Formally
• C is a fixed constant. We are not allowed to
  increase it as x increases.

10
The Big-O notation

• The constants C and k in the definition of big-O
  notation are called
• witnesses to the relationship f(x) is O(g(x)).
• If there is a pair of witnesses to the
  relationship f(x) is O(g(x)), then
• there are infinitely many pairs of witnesses to
  that relationship.
• Indeed, if C and k are one pair of witnesses,
  then any pair C and k,
• where C lt C and k lt k, is also a pair of
  witnesses.
• To establish that f(x) is O(g(x)) we need only
  one pair of witnesses to
• this relationship.

11
Examples

• Example 5.4 Rosen, p.181 Let
• f(x) x2 2x 1
• Prove that f(x) is in O(x2).
• Proof For x gt 1, we have 1 lt x lt x2. That gives
• f(x) x2 2x 1 x2 2x2 x2 4x2
• Because the above equation holds for every
  positive integer x gt 1,
• using k 1 and C 4 as witnesses, we get
• f(x) Cx2, for every x gt k
• showing that f(x) is O(x2).

12
Examples

• Example 5.5 Show that f(x) 3x3 7x2 - 4x
  2 is O(x3).
• Proof For all x gt 1, we have
• So, for the pair of witnesses k 1, C 16, we
  have f(x) Cx3 for
• every x gt k, giving f(x) is O(x3).

13
Examples

• Remark The inequality
• can be obtained with repetitive use of the
  triangle inequality
• Exercise 5.1 Using the triangle inequality show
  that

14
Big-O of polynomials

• Proposition 5.1 Rosen, p.184 Any polynomial
  function is Big-O its
• leading degree without leading constant.
• Proof Let f(x) be a polynomial of degree k, that
  is,
• with ak?0.

15
Big-O of polynomials

• We have
• With witnesses
  
• and k 1, we have f(x) Cxk, for every x gt
  k, giving f(x) is O(xk).

16
The Big-O notation

• Some orders that often occur in practice are

17
Summary of the lecture

• Time complexity
• Big-O notation
• Big-O of polynomials
• Common Big-O orders

18
Exercise

• Which of the following statements are correct ?

• 3x 4 O(x)
• 2x2 2x 1 O(x2)
• 3x 4 O(x2)
• x2 O(x)
• 3 2log(x) O(logx)
• 2x 1 O(4x9)
• 2x 4log(x) O(2x)
• xlogx O(logx)
• x4 32x O(2x )
• 25 O(1)

True True True False True True True False True Tru
e