Recap: Order of Growth

1
Lecture 4

• Recap Order of Growth
• Fun with recursion
• Fast exponentiation
• Hanoi towers
• Perfect numbers (if time permits)

2
Orders of Growth

• Suppose n is a parameter that measures the size
  of a problem (the size of its input)
• R(n)measures the amount of resources needed to
  solve a problem of size n.
• Two common resources are space, measured by the
  number of deferred operations, and time, measured
  by the number of primitive steps.

3
Orders of Growth

• Want to estimate the order of growth of R(n)

R1(n) 100n2 R2(n) 2n210n2
R3(n) n2
Are all the same in the sense that if we multiply
the input by a factor of 2, the resource
consumption increases by a factor of about 4
Order of growth is proportional to n2
4
Order of Growth
5
Orders of Growth

• We say that R(n)? O(f(n)) if there is a
  constant cgt0 such that for all n1
• R(n) c f(n)

• We say that R(n)? ?(f(n)) if there is a
  constant cgt0 such that for all n1
• R(n) c f(n)

• We say that R(n)? Q(f(n)) if there are
  constants c1,c2gt 0 such that for all n1
• c1 f(n) R(n) c2 f(n)

6
The intuition of this notation
R(n)? O(f(n)) ?? f(n)? ?(R(n))
R(n)? Q(f(n)) ?? f(n)? Q(R(n))
7
f(n) O(g(n))
1
8
f(n) T (g(n))
1
9
Orders of Growth
True or False?
f
100n2 ? O(n)
t
100n2 ? ?(n)
f
100n2 ? Q(n)
t
100n2 ? Q(n2)
True or False?
f
2100 ? ?(n)
t
2100n ? O(n2)
f
2n ? Q(n)
t
210 ? Q(1)
10
The conditional form
(cond (lttest-1gt ltconsequent-1gt) (lttest-2gt
ltconsequent-2gt) . (lttest-ngt
ltconsequent-ngt) (else
ltconsequent-elsegt))
(define (abs x) (cond ((gt x 0) x) (( x 0)
0)
((lt x 0) (- x))))
(else (- x))))
11
computing ab

• (define exp-1
• (lambda (a b)
• (if ( b 0)
• 1
• ( a (exp-1 a (- b 1))))))

12
Iterative version

• (define (exp-2 a b)
• (define (exp-iter a counter product)
• (if ( counter 0)
• product
• (exp-iter a (- counter 1) ( a
  product)))
• (exp-iter a b 1))

13
Another algorithm for computing ab

• If b is even, then ab (a2)(b/2)
• If b is odd, then ab aa(b-1)
• Problem size reduced to half in at most two
  steps.

(define (exp-fast1 a b) computes ab (cond
(( b 0) 1) ((even? b) (exp-fast1 ( a
a) (/ b 2))) (else ( a (exp-fast1 a (- b
1)))))))
14
(exp-fast 3 56)
(define (exp-fast1 a b) (cond (( b 0) 1)
((even? b) (exp-fast1 ( a a) (/ b 2)))
(else ( a (exp-fast1 a (- b 1)))))))
compute 356 (exp-fast1 3 56)
b1110002 (exp-fast1 9 28)
b 111002 (exp-fast1
81 14) b
11102 (exp-fast1 6561 7)
b 1112 6561 (exp-fast1 6561 6)
b 1102 6561 (exp-fast1
43046721 3) b 112 6561
43046721 (exp-fast1 43046721 2) b
102 6561 43046721 (exp-fast1 1853020188851841
1) b 12 6561 43046721 1853020188851841
(exp-fast1 .. 0) b0 6561 43046721
1853020188851841 523347633027360537213511521 Note
scheme allows the use of very large numbers
15
How much time does exp-fast take?
If b is even T(b)
T(b/2)c and if b is odd then T(b)
T(b-1)c T((b-1)/2)2c
OK, but what is it? In theta notation, that is!
16
Lets start with a simpler case

• Say that T(b) T(b/2)c always
• For what values of b does this happen?
• T(b) T(b/2)c T(b/4)cc T(b/8)ccc .
  
• T(b/(2k)) kc
• When does b/(2k) 1?
• K log(b) gt T(b) T(1) log(b) Q (1)log(b)
  Q(log(b))

Note logbase1 x k logbase2 x for some
constant k gt 0
17
Counting odd steps

• We also make odd steps (where b becomes odd)
• But how much?
• Claim At most one odd step per each even
  step
• So we make no more than twice the number of
  steps!
• Still Q(log(b))
• Space complexity is Q(log(b)) as well

18
Fast exponentiation (Iterative Approach)
product ? 1, base a, exp b
Init
Loop
Even? b base ? base2 exp ?
exp/2 Odd? b product ? producta exp ? exp-1
19
Scheme code (Iterative Approach)
(define (exp-fast2 a b) (define (exp-fast-iter
a b product) (cond (( b 0) product)
((even? b) (exp-fast-iter ( a a) (/ b 2)
product)) (else (exp-fast-iter a (- b 1)
( a product))))) (exp-fast-iter a b 1))
20
Comparing the three exponentiation procedures
exp-fast1and exp-fast2 are exponentially faster
than exp-1 and exp-2
21
Towers of Hanoi

• Three posts, and a set of disks of different
  sizes.
• A disk can be placed only on a larger disk (or
  on bottom).
• At the beginning all the disks are on the left
  post.
• The goal is to move the disks one at a time,
  while preserving these conditions, until the
  entire stack has moved from the left post to
  another

You are allowed to move only the topmost disk at
a step
22
Use our paradigm

• Wishful thinking
• Smaller problem A problem with one disk less
• How do we use it ?

To move n disks from post A to post C (using B as
aux) Move top n-1 disks from post A to post
B (using C as aux) Move the largest disk
from post A to post C Move n-1 disks from
post B to post C (using A as aux)
We solve 2 smaller problems !
23
Towers of Hanoi
(define (move-tower size from to aux) (cond
(( size 1) (one-move from to)) (else
(move-tower (- size 1) from aux to)
(one-move from to) (move-tower (- size
1) aux to from))))
(define (one-move from to) (display "Move top
disk from ") (display from) (display " To
") (display to) (newline))
24
Tree Recursion
(mt 3 1 2 3)
25
Towers of Hanoi -- trace

• (move-tower 3 1 2 3)
• Move top disk from 1 to 2
• Move top disk from 1 to 3
• Move top disk from 2 to 3
• Move top disk from 1 to 2
• Move top disk from 3 to 1
• Move top disk from 3 to 2
• Move top disk from 1 to 2

26
Orders of growth for towers of Hanoi

• Denote by T(n) the number of steps that we need
  to take to solve the case for n disks.

T(n) 2T(n-1) 1 T(1) 1
This solves to T(n) 2n-1 Q (2n)
What does that mean ?
27
Hanoi Towers
Say we want to solve the problem for 400 disks.
Say it takes a second to move a disk . We need
about 2400 seconds. Thats about 2373
years. Thats about 2363 millenniums. Might be
longer then the age of the universe .
Infeasible !!!!
28
Lets buy a fast computerand make it feasible.
Our new computer can move giga billion (260 )
disks a second. Absolutely the last word in the
field of computing. We need about 2340
seconds. Thats about 2313 years. Thats about
2303 millenniums.
Does not help much. Infeasible !!!!
An algorithm with exponential time complexityis
not scalable
29
What about Space complexity?
30
Towers of Hanoi
(define (move-tower size from to aux) (cond
(( size 1) (one-move from to)) (else
(move-tower (- size 1) from aux to)
(one-move from to) (move-tower (- size
1) aux to from))))
Pending operations
(define (one-move from to) (display "Move top
disk from ") (display from) (display " To
") (display to) (newline))
31
Towers of Hanoi Space complexity
The number of pending operations is the height of
the recursion tree.
So the space complexity is
S(n) Q(n)
Note that the second recursive call is treated as
tail recursion, and forms an iteration (no
pending operation for these calls).
32
Tree Recursion
(mt 3 2 1 3)
33
Perfect Numbers
A perfect number equals the sum of its divisors
6 123
28 124714
496 1 2 4 8 16 31 62 124 248

Find the sum of all perfect numbers in the range
from, till
34
Perfect Numbers (contd)
A procedure to find the sum of divisors (define
(sum-divisors n) (define (sum-divisors-iter
div sum) (cond ( (gt div (sqrt n))
_____) ( ( (remainder
n div) 0) ____________________
__________________ ) (else
____________________________)
(sum-divisors-iter 2 1))
sum
(sum-divisors-iter ( div 1) ( sum div (/ n div))
(sum-divisors-iter ( div 1) sum)))
35
Perfect Numbers (contd)
A procedure to find the sum of divisors
correct version
(define (sum-divisors n) (define
(sum-divisors-iter div sum) (cond ( (gt div
(sqrt n)) sum)
( ( (remainder n div) 0)
(sum-divisors-iter ( div 1) ( sum

(if ( div (sqrt n))
div

( div (/ n div))))))
(else (sum-divisors-iter
( div 1) sum)))) (sum-divisors-iter 2 1))
36
Perfect Numbers (contd)
A procedure to find the sum of perfect
numbers (define (sum-perfect-numbers from till)
(define (sum-divisors n) )
(cond ( (gt from till) ______
) ( ( (sum-divisors from)
from) __________________________
___________ ) (else
_____________________________________)))

0
( from (sum-perfect-numbers ( from 1) till))
(sum-perfect-numbers ( from 1) till)
37
Perfect Numbers (contd)
Lets check the complexities of a call to
(sum-perfect-numbers 1 n)
Q (n3/2)
Time complexity?
Q (The number of perfect numbers no larger
than n)
Space complexity?