Introduction to Divide and Conquer

1
Introduction to Divide and Conquer

• Lecture 8 More Greedy Examples
• Shortest Path Problems
• Dijkstras Algorithm
• The Knapsack Problem
• Linear Programs
• Lecture 9 Recap
• Recap Greedy Algorithms
• Discuss Papers
• Discuss Assignment 2

• Lecture 10 Introduction to Divide and Conquer
• Binary Search
• Compare to Linear Search
• Best/Worst Case Analysis
• More Recursion Relations
• Change of Variables
• Intro to Probability
• What does average mean?
• Average Case Analysis

2
Search-a-Lympics

• Phone Book Competition
• Sequential vs. Binary Search

• Tattoos--Removed

• Childrens Activities

• Notaries Public

• Go-Karts

• How do the Search Methods compare?

3
Sequential vs. Binary Search

• function binsearch(T1..n, x)
• if n0 or xgtTn then return n1
• else return binrec(T1..n,x)
• function binrec (Ti..j, x)
• if i j then return I
• k ?(ij)/2
• if x lt Tk then return binrec(Ti..k,x)
• else return binrec(Tk1..j, x)

• function sequential (T1..n, x)
• for i ? 1 to n do
• if Ti gt x then return i
• return n1

1 2 3 4 5 6 7 8
9 10 11
x 12 lt Tk?
i k
j no
i k j
yes
i k j
yes
ik j
no
ij ij
stop
4
Sequential vs. Binary Search

• function binsearch(T1..n, x)
• if n0 or xgtTn then return n1
• else return binrec(T1..n,x)
• function binrec (Ti..j, x)
• if i j then return I
• k ?(ij)/2
• if x lt Tk then return binrec(Ti..k,x)
• else return binrec(Tk1..j, x)

• function sequential (T1..n, x)
• for i ? 1 to n do
• if Ti gt x then return i
• return n1

Q(r) W(n) worst case O(1) best case

• If t(m) for binrec, mj-i1, then
• binsearch needs about t(n)

• For m even, t(m) t(m/2)g(m)
• where g(m) O(1) c

• How do we solve this recurrence?

5
More on Solving Recurrences Change of Variables

• Suppose we want to solve y(k) y(k/2)c, y(1)q,
  
• where k 2, 4, 8, 16,

• Linear
• Constant Co-efficient
• Non-homogeneous
• Not defined on all times k!!

6
More on Solving Recurrences Change of Variables

• Suppose we want to solve y(k) y(k/2)c, y(1)q,
  
• where k 2, 4, 8, 16,

• Linear
• Constant Co-efficient
• Non-homogeneous
• Not defined on all times k

• Need a change of variables

• Let k 2n. Then y(2n) y(2n-1) c gt y(n)
  y(n-1) c

gt y(n) - y(n-1) c
Solve for z(n) z(n) z(n-1) 0
gt y(n) y(n) z(n)
ln ln-1 0
l 1 0
7
More on Solving Recurrences Change of Variables

• Suppose we want to solve y(k) y(k/2)c, y(1)q,
  
• where k 2, 4, 8, 16,

• Linear
• Constant Co-efficient
• Non-homogeneous
• Not defined on all times k

• Need a change of variables

• Let k 2n. Then y(2n) y(2n-1) c gt y(n)
  y(n-1) c

gt y(n) - y(n-1) c
Solve for y(n) Each step adds c
gt y(n) y(n) d1n
Try y(n) cn
cn c(n-1)c ?
gt y(n) y(n) d
8
More on Solving Recurrences Change of Variables

• Suppose we want to solve y(k) y(k/2)c, y(1)q,
  
• where k 2, 4, 8, 16,

• Linear
• Constant Co-efficient
• Non-homogeneous
• Not defined on all times k

• Need a change of variables

• Let k 2n. Then y(2n) y(2n-1) c gt y(n)
  y(n-1) c

gt y(n) - y(n-1) c
Use initial condition to find d q c d gt
d q - c
gt y(n) y(n) d1n
gt y(n) y(n) d
y(n) c(n-1)q
gt y(n) cn d
9
More on Solving Recurrences Change of Variables

• Suppose we want to solve y(k) y(k/2)c, y(1)q,
  
• where k 2, 4, 8, 16, Let k 2n

y(n) c(n-1)q

• What is the order of growth of y(n)?

Q(n)

• What is the order of growth of y(k)?

Q(log2k)

• Recall expression for binrec for m even, t(m)
  t(m/2)g(m)
• 
  where g(m) O(1) c

10
Sequential vs. Binary Search

• function binsearch(T1..n, x)
• if n0 or xgtTn then return n1
• else return binrec(T1..n,x)
• function binrec (Ti..j, x)
• if i j then return I
• k ?(ij)/2
• if x lt Tk then return binrec(Ti..k,x)
• else return binrec(Tk1..j, x)

• function sequential (T1..n, x)
• for i ? 1 to n do
• if Ti gt x then return i
• return n1

Q(n) worst case Q(1) best case
Q(log n) worst case Q(log n) best case
What about average case?
11
What Does Average Mean?

• What is the complexity of a typical instance of
  size n?

• For each size n,
• there may be a set
• of outcomes that
• are possible
• Some of these may
• be more likely than
• others
• We use probability
• to make the notion
• of typical precise

y
n
12
Elementary Probability

• Consider a set S, called the sample space.
  Elements of the set S are called sample points.

S
13
Elementary Probability

• Consider a set S, called the sample space.
  Elements of the set S are called sample points.

S
14
Elementary Probability

• A random variable is a function that assigns an
  arbitrary number to each sample point.

Random Variable, X
Probability Measure

• PrXx means the probability
• of the event Xx, e.g
• PrX -1 .1
• PrX5 .4
• PrX 0 .1

Set S
-1 0 4 -1.5 2 -3 5 5 9 3 2
.1 .1 .2 .1 .05 .05 .1 .1 .05 .1 .05
.1
.1
.2
.1
.05
.05
.1
.1
.05
.1
.05
15
Elementary Probability

• The expectation of X (or expected value, or mean,
  or average) is given by

Random Variable, X
Probability Measure
Set S

• Like the center of mass.
• In this example E(X) 2.45

-1 0 4 -1.5 2 -3 5 5 9 3 2
.1 .1 .2 .1 .05 .05 .1 .1 .05 .1 .05
.1
.1
.2
.1
.05

• The probability mass function
• of random variable X is just
• p(x) PrXx

.05
.1
.1
.05
.1
.05
16
Sequential vs. Binary Search

• function binsearch(T1..n, x)
• if n0 or xgtTn then return n1
• else return binrec(T1..n,x)
• function binrec (Ti..j, x)
• if i j then return I
• k ?(ij)/2
• if x lt Tk then return binrec(Ti..k,x)
• else return binrec(Tk1..j, x)

• function sequential (T1..n, x)
• for i ? 1 to n do
• if Ti gt x then return i
• return n1

Q(n) worst case Q(1) best case
Q(log n) worst case Q(log n) best case
What about average case?
17
Sequential Search Average Case
Probability Measure
Random Variable, X
Set S
1 2 3 4 5 n
1 2 3 4 5 n
1/n 1/n 1/n 1/n 1/n 1/n

• function sequential (T1..n, x)
• for i ? 1 to n do
• if Ti gt x then return i
• return n1

Q(n) worst case Q(1) best case
Expected value of X is
18
Sequential Search Average Case

• The loop will terminate after
• (n1)/2 iterations when
• searching for the expected
• instance for a problem of
• size n.

• function sequential (T1..n, x)
• for i ? 1 to n do
• if Ti gt x then return i
• return n1

Average case Q(n), Assuming the probability
mass function for X is uniform
Q(n) worst case Q(1) best case
Expected value of X is
19
Sequential vs. Binary Search

• function binsearch(T1..n, x)
• if n0 or xgtTn then return n1
• else return binrec(T1..n,x)
• function binrec (Ti..j, x)
• if i j then return I
• k ?(ij)/2
• if x lt Tk then return binrec(Ti..k,x)
• else return binrec(Tk1..j, x)

• function sequential (T1..n, x)
• for i ? 1 to n do
• if Ti gt x then return i
• return n1

Q(n) worst case Q(n) average case Q(1) best case
Q(log n) worst case Q(log n) average case Q(log
n) best case
Need to look at constants on order of growth to
decide at what size n It makes sense to switch
from sequential sort to binary sort.
20
Bonus Slide!!(No Extra Charge!)

• Given the breathing difficulties today and your
  new mastery of probability, Lets play the
  Forrest Fire Game!
• Given
• An n by n grid
• Each square of the grid is a tree
• If lightening strikes a tree, all connected trees
  burn
• If lightening strikes a cut-down tree, nothing
  happens
• Connected trees are those up, down, left, or
  right (not diagonal)
• May cut trees down before the storm to disconnect
  high risk areas
• Want to maximize the trees left after a storm
• Cut the trees (if any) you think protect your
  forest. Then roll a dice to see which of the
  following storms you have

21
Bonus Slide!!(No Extra Charge!)

• (1,2) Light rainno lightening strikes
• (3,4) Thunderstormlightening strikes twice
  wildly!
• For each strike, choose two random numbers (i, j)
  uniformly distributed between 1 and n. The tree
  at the coordinate (i, j) is hit by lightening and
  all connected trees burn.
• (5) A Sign from Heavenlightening strikes the
  square given by your birthday coordinates (month,
  day). If either is greater than n, simply wrap
  around until you find the coordinate.
• (6) Lightening strikes once randomly as indicated
  above

22
Bonus Slide!!(No Extra Charge!)

• Fix your design to protect your forest
• Play the game enough times to compute an Average
  number of Surviving Trees such that if you
  played a few more times and recomputed the
  average, it would not change much.

225 Total Trees. Used 22 to protect the rest from
fire. How good is this design?