Chapter 7: Advanced Counting Techniques

1
Chapter 7Advanced Counting Techniques
2
7.1 Recurrence Relations

• A recurrence relation (R.R., or just recurrence)
  for a sequence an is an equation that expresses
  an in terms of one or more previous elements a0,
  , an-1 of the sequence, for all nn0.
• A recursive definition, without the base cases.
• A particular sequence (described non-recursively)
  is said to solve the given recurrence relation if
  it is consistent with the definition of the
  recurrence.
• A given recurrence relation may have many
  solutions.

7.1 Recurrence Relations
3
Recurrence Relation Example

• Consider the recurrence relation
• an 2an-1 - an-2 (n2).
• Which of the following are solutions? an
  3n an 2n
• an 5

7.1 Recurrence Relations
4
Example Applications

• Recurrence relation for growth of a bank account
  with P interest per given period
• Mn Mn-1 (P/100)Mn-1
• Growth of a population in which each organism
  yields 1 new one every period starting 2 periods
  after its birth.
• Pn Pn-1 Pn-2 (Fibonacci relation)

7.1 Recurrence Relations
5
Solving Compound Interest RR

• Mn Mn-1 (P/100)Mn-1
• (1 P/100) Mn-1
• r Mn-1 (let r 1 P/100)

7.1 Recurrence Relations
6
Tower of Hanoi Example

• Problem Get all disks from peg 1 to peg 2.
• Only move 1 disk at a time.
• Never set a larger disk on a smaller one.

Peg 1
Peg 2
Peg 3
7.1 Recurrence Relations
7
Hanoi Recurrence Relation

• Let Hn moves for a stack of n disks.
• Optimal strategy
• Move top n-1 disks to spare peg. (Hn-1 moves)
• Move bottom disk. (1 move)
• Move top n-1 to bottom disk. (Hn-1 moves)
• Note Hn 2Hn-1 1

7.1 Recurrence Relations
8
Solving Tower of Hanoi RR

• Hn 2 Hn-1 1

7.1 Recurrence Relations
9
Finding Recurrence Relation

• Ex Find a recurrence relation and give initial
  conditions for the number of bit strings of
  length n that do not have two consecutive 0s. How
  many such bit strings are there of length 5?

7.1 Recurrence Relations
10
Codeword Enumeration

• ExConsider a string of decimal digits a valid
  codeword if it contains an even number of 0
  digits. For example, 1230407869 is valid, whereas
  120987045608 is not valid. Let an be the number
  of valid n-digit codewords. Find a recurrence
  relation for an .

7.1 Recurrence Relations
11
Catalan Numbers

• Ex Find a recurrence relation for Cn , the
  number of ways to parenthesize the product of n1
  numbers, x0, x1,, xn, to specify the order of
  multiplication. For example, C3 5.

7.1 Recurrence Relations
12
7.2 Solving Recurrences
General Solution Schemas

• A linear homogeneous recurrence of degree k with
  constant coefficients (k-LiHoReCoCo) is a
  recurrence of the form an c1an-1
  ckan-k,where the ci are all real, and ck ? 0.
• The solution is uniquely determined if k initial
  conditions a0ak-1 are provided.

7.2 Solving Recurrences
13
Solving LiHoReCoCos

• Basic idea Look for solutions of the form an
  rn, where r is a constant.
• This requires the characteristic equation rn
  c1rn-1 ckrn-k, i.e., rk - c1rk-1 - - ck
  0
• The solutions (characteristic roots) can yield an
  explicit formula for the sequence.

7.2 Solving Recurrences
14
Solving 2-LiHoReCoCos

• Consider an arbitrary 2-LiHoReCoCo an c1an-1
  c2an-2
• It has the characteristic equation (C.E.) r2 -
  c1r - c2 0
• Thm. 1 If this CE has 2 roots r1?r2, then an
  a1r1n a2r2n for n0for some constants a1, a2.

7.2 Solving Recurrences
15
Example

• Solve the recurrence an an-1 2an-2 given the
  initial conditions a0 2, a1 7.
• Solution

7.2 Solving Recurrences
16
Example Continued

• To find a1 and a2, solve the equations for the
  initial conditions a0 and a1
• Simplifying, we have the pair of equations
• which we can solve easily by substitution
• Final answer

Check an0 2, 7, 11, 25, 47, 97
7.2 Solving Recurrences
17
Example

• Find an explicit formula for the Fibonacci
  numbers.

7.2 Solving Recurrences
18
The Case of Degenerate Roots

• Now, what if the C.E. r2 - c1r - c2 0 has only
  1 root r0?
• Theorem 2 Then, an a1r0n a2nr0n, for all
  n0,for some constants a1, a2.
• Ex

7.2 Solving Recurrences
19
k-LiHoReCoCos

• Consider a k-LiHoReCoCo
• Its C.E. is
• Thm.3 If this has k distinct roots ri, then the
  solutions to the recurrence are of the form
• for all n0, where the ai are constants.

7.2 Solving Recurrences
20
Example

• Ex

7.2 Solving Recurrences
21
Degenerate k-LiHoReCoCos

• Suppose there are t roots r1,,rt with
  multiplicities m1,,mt. Then
• for all n0, where all the a are constants.

7.2 Solving Recurrences
22
Example

• Ex

7.2 Solving Recurrences
23
LiNoReCoCos

• Linear nonhomogeneous RRs with constant
  coefficients may (unlike LiHoReCoCos) contain
  some terms F(n) that depend only on n (and not on
  any ais). General form
• an c1an-1 ckan-k F(n)

The associated homogeneous recurrence
relation(associated LiHoReCoCo).
7.2 Solving Recurrences
24
Solutions of LiNoReCoCos

• A useful theorem about LiNoReCoCos
• If an p(n) is any particular solution to the
  LiNoReCoCo
• Then all its solutions are of the form an
  p(n) h(n),where an h(n) is any solution to
  the associated homogeneous RR

7.2 Solving Recurrences
25
Example

• Find all solutions to an 3an-12n. Which
  solution has a1 3?
• Notice this is a 1-LiNoReCoCo. Its associated
  1-LiHoReCoCo is an 3an-1, whose solutions are
  all of the form an a3n. Thus the solutions to
  the original problem are all of the form an
  p(n) a3n. So, all we need to do is find one
  p(n) that works.

7.2 Solving Recurrences
26
Trial Solutions

• If the extra terms F(n) are a degree-t polynomial
  in n, you should try a degree-t polynomial as the
  particular solution p(n).
• This case F(n) is linear so try an cn d.
• (for all n)
  (collect
  terms) So
• So is a solution.
• Check an1 -5/2, -7/2, -9/2,

7.2 Solving Recurrences
27
Finding a Desired Solution

• From the previous, we know that all general
  solutions to our example are of the form
• Solve this for a for the given case, a1 3
• The answer is

7.2 Solving Recurrences
28
Example

• Ex

7.2 Solving Recurrences
29
7.3 Divide Conquer R.R.s

• Main points so far
• Many types of problems are solvable by reducing a
  problem of size n into some number a of
  independent subproblems, each of size ??n/b?,
  where a?1 and bgt1.
• The time complexity to solve such problems is
  given by a recurrence relation
• T(n) aT(?n/b?) g(n)

7.3 D-C Recurrence Relations
30
DivideConquer Examples

• Binary search Break list into 1 sub-problem
  (smaller list) (so a1) of size ??n/2? (so b2).
• So T(n) T(?n/2?)c (g(n)c constant)
• Merge sort Break list of length n into 2
  sublists (a2), each of size ??n/2? (so b2),
  then merge them, in g(n) T(n) time.
• So T(n) 2T(?n/2?) cn (roughly, for some c)

7.3 D-C Recurrence Relations
31
DivideConquer Examples

• Finding the Maximum and Minimum Break list into
  2 sub-problem (smaller list) (so a2) of size
  ??n/2? (so b2).
• So T(n) 2T(?n/2?)2 (g(n)2 constant)

7.3 D-C Recurrence Relations
32
Fast Multiplication Example

• The ordinary grade-school algorithm takes T(n2)
  steps to multiply two n-digit numbers.
• This seems like too much work!
• So, lets find an asymptotically faster
  multiplication algorithm!
• To find the product cd of two 2n-digit base-b
  numbers, c(c2n-1c2n-2c0)b and
  d(d2n-1d2n-2d0)b,
• First, we break c and d in half
• cbnC1C0, dbnD1D0, and then... (see
  next slide)

7.3 D-C Recurrence Relations
33
Derivation of Fast Multiplication

(Multiply out polynomials)
(Factor last polynomial)
7.3 D-C Recurrence Relations
34
Recurrence Rel. for Fast Mult.

• Notice that the time complexity T(n) of the fast
  multiplication algorithm obeys the recurrence
• T(2n)3T(n)?(n) i.e.,
• T(n)3T(n/2)?(n)
• So a3, b2.

Time to do the needed adds subtracts of
n-digit and 2n-digit numbers
7.3 D-C Recurrence Relations
35
The Master Theorem

• Consider a function f(n) that, for all nbk for
  all k?Z,,satisfies the recurrence relation
• f(n) a f (n/b) cnd
• with a1, integer bgt1, real cgt0, d0. Then

7.3 D-C Recurrence Relations
36
Examples

• Consider a function f(n) that, for all n2k for
  all k?Z,,satisfies the recurrence relation
• f(n) 5f(n/2) 3. Then
• Complexity of Merge Sort

7.3 D-C Recurrence Relations
37
Example

• Recall that complexity of fast multiply was
• T(n)3T(n/2)?(n)
• Thus, a3, b2, d1. So a gt bd, so case 3 of the
  master theorem applies, so
• which is ?(n1.58), so the new algorithm is
  strictly faster than ordinary T(n2) multiply!

7.3 D-C Recurrence Relations
38
Example

• The Closest-Pair Problema set of n points,
• How can this closest pair of points be found in
  an efficient way?
• T(n)2T(n/2)7n

7.3 D-C Recurrence Relations
39
7.4 Generating Functions

• Definitiongenerating function for the sequence
  of real numbers is the infinite
  series

7.4 Generating Functions
40
Examples

• What is the generating function of the sequence
  1,1,1,1,1,1?
• What is the generating function of the sequence
  ?

7.4 Generating Functions
41
Examples

• The function f(x)1/(1?x) is the generating
  function of the sequence 1,1,1,for xlt1.
• The function f(x)1/(1?ax) is the generating
  function of the sequence 1,a,a2,for axlt1.

7.4 Generating Functions
42
Theorem

Convolution of ak and bk
7.4 Generating Functions
43
Example

• What sequence has the generating function
• f(x)1/(1?x)2 ?

7.4 Generating Functions
44
Extended Binomial Coefficient

• Examples

7.4 Generating Functions
45
Extended Binomial Theorem

• Can be proved using Maclaurin series.
• Examples

7.4 Generating Functions
46
Example

• Find the number of solutions of
• Sol Find the coefficient of x17 ,
• The answer is

7.4 Generating Functions
47
Example

• Solve the recurrence relation
• Sol Let G(x) be the generating function of
  ak ,

7.4 Generating Functions
48
Example(Contd)

7.4 Generating Functions