Growth of Functions

1
Growth of Functions

• CS/APMA 202
• Rosen section 2.2
• Aaron Bloomfield

2
How does one measure algorithms

• We can time how long it takes a computer
• What if the computer is doing other things?
• And what happens if you get a faster computer?
• A 3 Ghz Windows machine chip will run an
  algorithm at a different speed than a 3 Ghz
  Macintosh
• So that idea didnt work out well

3
How does one measure algorithms

• We can measure how many machine instructions an
  algorithm takes
• Different CPUs will require different amount of
  machine instructions for the same algorithm
• So that idea didnt work out well

4
How does one measure algorithms

• We can loosely define a step as a single
  computer operation
• A comparison, an assignment, etc.
• Regardless of how many machine instructions it
  translates into
• This allows us to put algorithms into broad
  categories of efficientness
• An efficient algorithm on a slow computer will
  always beat an inefficient algorithm on a fast
  computer

5
Bubble sort running time

• The bubble step take (n2-n)/2 steps
• Lets say the bubble sort takes the following
  number of steps on specific CPUs
• Intel Pentium IV CPU 58(n2-n)/2
• Motorola CPU 84.4(n2-2n)/2
• Intel Pentium V CPU 44(n2-n)/2
• Notice that each has an n2 term
• As n increases, the other terms will drop out

6
Bubble sort running time

• This leaves us with
• Intel Pentium IV CPU 29n2
• Motorola CPU 42.2n2
• Intel Pentium V CPU 22n2
• As processors change, the constants will always
  change
• The exponent on n will not
• Thus, we cant care about the constants

7
An aside inequalities

• If you have a inequality you need to show
• x lt y
• You can replace the lesser side with something
  greater
• x1 lt y
• If you can still show this to be true, then the
  original inequality is true
• Consider showing that 15 lt 20
• You can replace 15 with 16, and then show that 16
  lt 20. Because 15 lt 16, and 16 lt 20, then 15 lt 20

8
An aside inequalities

• If you have a inequality you need to show
• x lt y
• You can replace the greater side with something
  lesser
• x lt y-1
• If you can still show this to be true, then the
  original inequality is true
• Consider showing that 15 lt 20
• You can replace 20 with 19, and then show that 15
  lt 19. Because 15 lt 19, and 19 lt 20, then 15 lt 20

9
An aside inequalities

• What if you do such a replacement and cant show
  anything?
• Then you cant say anything about the original
  inequality
• Consider showing that 15 lt 20
• You can replace 20 with 10
• But you cant show that 15 lt 10
• So you cant say anything one way or the other
  about the original inequality

10
Quick survey

• I felt I understand running times and inequality
  manipulation
• Very well
• With some review, Ill be good
• Not really
• Not at all

11
The 2002 Ig Nobel Prizes

• Biology
• Physics
• Interdisciplinary
• Chemistry
• Mathematics
• Literature
• Peace
• Hygiene
• Economics
• Medicine

Courtship behavior of ostriches towards humans
under farming conditions in Britain Demonstrati
on of the exponential decay law using beer
froth A comprehensive study of human belly
button lint Creating a four-legged periodic
table Estimation of the surface area of African
elephants The effects of pre-existing
inappropriate highlighting on reading
comprehension For creating Bow-lingual, a
computerized dog-to-human translation
device For creating a washing machine for cats
and dogs Enron et. al. for applying imaginary
numbers to the business world Scrotal
asymmetry in man in ancient sculpture
12
End of lecture on 3 March 2005
13
Review of last time

• Searches
• Linear n steps
• Binary log2 n steps
• Binary search is about as fast as you can get
• Sorts
• Bubble n2 steps
• Insertion n2 steps
• There are other, more efficient, sorting
  techniques
• In principle, the fastest are heap sort, quick
  sort, and merge sort
• These each take take n log2 n steps
• In practice, quick sort is the fastest, followed
  by merge sort

14
Big-Oh notation

• Let b(x) be the bubble sort algorithm
• We say b(x) is O(n2)
• This is read as b(x) is big-oh n2
• This means that the input size increases, the
  running time of the bubble sort will increase
  proportional to the square of the input size
• In other words, by some constant times n2
• Let l(x) be the linear (or sequential) search
  algorithm
• We say l(x) is O(n)
• Meaning the running time of the linear search
  increases directly proportional to the input size

15
Big-Oh notation

• Consider b(x) is O(n2)
• That means that b(x)s running time is less than
  (or equal to) some constant times n2
• Consider l(x) is O(n)
• That means that l(x)s running time is less than
  (or equal to) some constant times n

16
Big-Oh proofs

• Show that f(x) x2 2x 1 is O(x2)
• In other words, show that x2 2x 1 cx2
• Where c is some constant
• For input size greater than some x
• We know that 2x2 2x whenever x 1
• And we know that x2 1 whenever x 1
• So we replace 2x1 with 3x2
• We then end up with x2 3x2 4x2
• This yields 4x2 cx2
• This, for input sizes 1 or greater, when the
  constant is 4 or greater, f(x) is O(x2)
• We could have chosen values for c and x that were
  different

17
Big-Oh proofs
18
Rosen, section 2.2, question 2(b)

• Show that f(x) x2 1000 is O(x2)
• In other words, show that x2 1000 cx2
• We know that x2 gt 1000 whenever x gt 31
• Thus, we replace 1000 with x2
• This yields 2x2 cx2
• Thus, f(x) is O(x2) for all x gt 31 when c 2

19
Rosen, section 2.2, question 1(a)

• Show that f(x) 3x7 is O(x)
• In other words, show that 3x7 cx
• We know that x gt 7 whenever x gt 7
• Duh!
• So we replace 7 with x
• This yields 4x cx
• Thus, f(x) is O(x) for all x gt 7 when c 4

20
Quick survey

• I felt I understand (more or less) Big-Oh proofs
• Very well
• With some review, Ill be good
• Not really
• Not at all

21
Todays demotivators
22
A variant of the last question

• Show that f(x) 3x7 is O(x2)
• In other words, show that 3x7 cx2
• We know that x gt 7 whenever x gt 7
• Duh!
• So we replace 7 with x
• This yields 4x lt cx2
• This will also be true for x gt 7 when c 1
• Thus, f(x) is O(x2) for all x gt 7 when c 1

23
What that means

• If a function is O(x)
• Then it is also O(x2)
• And it is also O(x3)
• Meaning a O(x) function will grow at a slower or
  equal to the rate x, x2, x3, etc.

24
Function growth rates

• For input size n 1000
• O(1) 1
• O(log n) 10
• O(n) 103
• O(n log n) 104
• O(n2) 106
• O(n3) 109
• O(n4) 1012
• O(nc) 103c c is a consant
• 2n 10301
• n! 102568
• nn 103000

Many interesting problems fall into this category
25
Function growth rates
Logarithmic scale!
26
Integer factorization

• Factoring a composite number into its component
  primes is O(2n)
• This, if we choose 2048 bit numbers (as in RSA
  keys), it takes 22048 steps
• Thats about 10617 steps!

27
Formal Big-Oh definition

• Let f and g be functions. We say that f(x) is
  O(g(x)) if there are constants c and k such that
• f(x) C g(x)
• whenever x gt k

28
Formal Big-Oh definition
29
A note on Big-Oh notation

• Assume that a function f(x) is O(g(x))
• It is sometimes written as f(x) O(g(x))
• However, this is not a proper equality!
• Its really saying that f(x) C g(x)
• In this class, we will write it as f(x) is
  O(g(x))

30
The growth of combinations of functions

• Ignore this part

31
Big-omega and Big-theta

• Ignore this part

32
NP Completeness

• Not in the textbook this is additional material
• A full discussion of NP completeness takes 3
  hours for somebody who already has a CS degree
• We are going to do the 15 minute version of it
• Any term of the form nc, where c is a constant,
  is a polynomial
• Thus, any function that is O(nc) is a
  polynomial-time function
• 2n, n!, nn are not polynomial functions

33
Satisfiability

• Consider a Boolean expression of the form
• (x1 ? ?x2 ? x3) ? (x2 ? ?x3 ? x4) ? (?x1 ? x4 ?
  x5)
• This is a conjunction of disjunctions
• Is such an equation satisfiable?
• In other words, can you assign truth values to
  all the xis such that the equation is true?

34
Satisfiability

• If given a solution, it is easy to check if such
  a solution works
• Plug in the values this can be done quickly,
  even by hand
• However, there is no known efficient way to find
  such a solution
• The only definitive way to do so is to try all
  possible values for the n Boolean variables
• That means this is O(2n)!
• Thus it is not a polynomial time function
• NP stands for Not Polynomial
• Cooks theorem (1971) states that SAT is
  NP-complete
• There still may be an efficient way to solve it,
  though!

35
NP Completeness

• There are hundreds of NP complete problems
• It has been shown that if you can solve one of
  them efficiently, then you can solve them all
• Example the traveling salesman problem
• Given a number of cities and the costs of
  traveling from any city to any other city, what
  is the cheapest round-trip route that visits each
  city once and then returns to the starting city?
• Not all algorithms that are O(2n) are NP complete
• In particular, integer factorization (also O(2n))
  is not thought to be NP complete

36
NP Completeness

• It is widely believed that there is no
  efficient solution to NP complete problems
• In other words, everybody has that belief
• If you could solve an NP complete problem in
  polynomial time, you would be showing that P NP
• If this were possible, it would be like proving
  that Newtons or Einsteins laws of physics were
  wrong
• In summary
• NP complete problems are very difficult to solve,
  but easy to check the solutions of
• It is believed that there is no efficient way to
  solve them

37
Quick survey

• I sorta kinda get the hang of NP completeness
• Very well
• With some review, Ill be good
• Not really
• Not at all

38
Star Wars Episode III trailer

• No, really!

39
Quick survey

• I felt I understood the material in this slide
  set
• Very well
• With some review, Ill be good
• Not really
• Not at all

40
Quick survey

• The pace of the lecture for this slide set was
• Fast
• About right
• A little slow
• Too slow

41
Quick survey

• How interesting was the material in this slide
  set? Be honest!
• Wow! That was SOOOOOO cool!
• Somewhat interesting
• Rather borting
• Zzzzzzzzzzz