Stochastic Advantage

1
Stochastic Advantage

• Lecture 36
• Stochastic Advantage
• Coin Toss Primality Test
• Compare with Fermat
• Biased Algorithms
• Amplification of Stochastic Advantage

• Lecture 35 Primality Testing
• Fermats Theorem
• Primality Testing
• Miller-Rabin

2
Grading Final Presentations

• Grades based on content and style
• Content
• Comparison of individual work
• Insights and conclusions
• Creativity in new algorithm design
• Results
• Style
• Clarity in presentation
• Delivery10 min, response to questions
• Attendance and participation matters
• Come prepared to support each other

3
Advantage

• Let p be the probability that algorithm ALG is
  correct.
• If p is less than 1/2, no advantage of rerunning
  ALG.
• If so, you could solve any problem with several
  coin tosses.
• Let p-1/2 be the advantage of ALG.

4
Coin Toss Primality Test
function CoinToss (n) if coin-toss heads
then return n is prime else return n is
composite What is the probability that this
algorithm is correct?
Sample Space
Probability Measure
5
Coin Toss Primality Test
function CoinToss (n) if coin-toss heads
then return n is prime else return n is
composite What is the probability that this
algorithm is correct?
Sample Space
Probability Measure
.5
Probability measure depends on the probability of
n being prime or not!
.5
.5
.5
6
Coin Toss Primality Test
function CoinToss (n) if coin-toss heads
then return n is prime else return n is
composite What is the probability that this
algorithm is correct?
Sample Space
Probability Measure
.1 .1 .4 .4
.5
.5
.2
.8
.5
.5
7
Coin Toss Primality Test
function CoinToss (n) if coin-toss heads
then return n is prime else return n is
composite What is the probability that this
algorithm is correct?
.5
Sample Space
Probability Measure
Correct?
.1 .1 .4 .4
Y N N Y
.5
.5
.2
.8
.5
.5
8
Coin Toss Primality Test
function CoinToss (n) if coin-toss heads
then return n is prime else return n is
composite Does the probability of being correct
change if we repeat the test and take a majority
vote of the answer?
9
Sample Space
pH
H
pH
T
H
pH
H
pT
T
H
T
pH
H
pT
T
H
pT
H
prime
T
.2
pT
T
T
H
H
nH
.8
H
not prime
T
nH
T
H
nH
T
nT
T
H
H
nH
T
nT
H
T
nT
T
nT
10
Sample Space
pH
H
pH
T
H
pH
H
pT
T
H
T
pH
H
pT
T
H
pT
H
prime
T
.2
pT
T
T
H
H
nH
.8
.5
H
not prime
T
nH
T
H
nH
T
nT
T
H
H
nH
T
nT
H
T
nT
T
nT
11
Coin Toss Primality Test
function CoinToss (n) if coin-toss heads
then return n is prime else return n is
composite Does the probability of being correct
change if we repeat the test and take a majority
vote of the answer?
.5
NO. Repeating the test does not
increase our confidence because this
algorithm has advantage .5 - 1/2 0.
12
Compare Coin Flip with Fermat
Correct
Probability Measure
Sample Space
.1 .1 .4 .4
Y N N Y
.5
.5
.2
.8
.5
.5
Sample Space
Correct
Probability Measure
13
Compare Coin Flip with Fermat
Correct
Probability Measure
Sample Space
.1 .1 .4 .4
Y N N Y
.5
.5
.2
.8
.5
.5
Sample Space
Correct
Probability Measure
pass
1
prime
.2
pass
.8
.9
not prime
.1
fail
14
Compare Coin Flip with Fermat
Correct
Sample Space
Y N N Y
.5
Both Algorithms are Monte Carlo. When one of
the answers of a Monte Carlo algorithm is always
correct, we call the algorithm biased
.5
.2
.8
.5
.5
Sample Space
Correct
pass
1
prime
.2
pass
.8
.9
not prime
.1
fail
15
Sample Space
p
p
1
f
p
p
1
f
p
f
p
1
f
p
p
prime
f
.2
f
f
p
p
.9
.9
p
.8
p
not prime
.1
f
.1
f
.9
f
p
.9
f
.1
.1
f
f
f
p
p
.9
.9
f
.1
f
f
.1
p
f
.9
f
f
f
.1
16
Sample Space
p
p
1
f
p
p
1
f
p
f
.28
p
1
f
p
p
prime
f
.2
.4168
f
f
p
p
.9
.9
p
.8
p
not prime
.1
f
.1
f
.9
f
p
.9
f
1 - e
.1
.1
f
f
f
p
p
.9
.9
f
.1
f
f
.1
p
f
.9
f
f
f
.1
17
Biased Algorithms

• Biased algorithms can increase the probability
  of success
• arbitrarily close to 1 through a suitable
  number of repetitions
• So what was the advantage of Miller Rabin over
  Fermat?

• Fermat was not p-correct for any p, so you
  dont
• know how many repetitions you need to
  guarantee
• a certain level of confidence in your results
• Miller Rabin could guarantee a certain level of
• confidence for a fixed number of repetitions

• This improvement in our confidence of the
  results of our
• algorithm by simply repeating it on the same
  instance is
• called amplification of stochastic advantage.
  
• Can unbiased algorithms exhibit stochastic
  advantage?

18
Stochastic Advantage

• Etymology Greek stochastikos skillful in
  aiming, from stochazesthai to aim at, guess at,
  from stochos target, aim, guess -- more at
  STINGDate 19231 RANDOM specifically
  involving a random variable lta stochastic
  processgt2 involving chance or probability
  PROBABILISTIC lta stochastic model of
  radiation-induced mutationgt
• Stochastic advantage advantage that has to do
  with a random variable.

19
Amplification of Stochastic Advantage

• Rerun an algorithm several times to increase the
  stochastic advantage.
• How much amplification do you get?
• Depends on how many times the algorithm is
  repeated.
• Whether or not the algorithm is biased.
• Advantage of the algorithm.

20
Sample Space
.2
C C C
p
1
C C W
f
p
C W C
p
1
C W W
f
p
f
W C C
p
1
W C W
f
p
W W C
p
prime
f
.2
W W W
f
f
p
p
1/4
1/4
.8
.8x(1/4)x(1/4)x(1/4)
W W W
p
not prime
3/4
f
3/4
1/4
f
p
1/4
W C W
3/4
3/4
f
W C C
f
p
p
1/4
1/4
C W W
3/4
f
3/4
C W C
p
f
1/4
C C W
f
3/4
.8x(3/4)x(3/4)x(3/4)
C C C
21
Small Example
Given the 3/4-correct biased algorithm.
C Correct, W Wrong Have a 79/80 (.9875)
probability of being correct, for this biased
algorithm. Not bad.
22
Small Example
Given a 3/4-correct unbiased algorithm.
C Correct, W Wrong Have a 27/32 (.84)
probability of being correct, for an unbiased
algorithm.
23
3/4
Sample Space
.2x(3/4)x(3/4)x(3/4)
C C C
Y
3/4
C C W
N
Y
C W C
Y
3/4
C W W
N
Y
N
W C C
Y
W C W
N
Y
W W C
Y
yes
N
.2
.2x(1/4)x(1/4)x(1/4)
W W W
1/4
N
N
Y
Y
1/4
.8
.8x(1/4)x(1/4)x(1/4)
W W W
Y
no
N
W W C
N
Y
W C W
N
W C C
N
Y
Y
C W W
3/4
N
C W C
Y
N
C C W
3/4
N
.8x(3/4)x(3/4)x(3/4)
C C C
3/4
24
Now You Try
A 5/8-correct unbiased algorithm, or B
5/8-correct biased algorithm, or C
15/16-correct unbiased algorithm, or D
15/16-correct biased algorithm, or E
1/16-correct biased algorithm How correct are
three trials of each algorithm?
25
Closed Form Solution

• For k trials of an e-advantage algorithm,
• Expect to get the right answer i times
• With probability

CCC CCW CWC CWW WCC WCW WWC WWW
26
Closed Form Solution

• For k trials of an e-advantage algorithm,
• Expect to get the right answer i times
• With probability

CCC CCW CWC CWW WCC WCW WWC WWW
27
Closed Form Solution

• For k trials of an e-advantage algorithm,
• Expect to get the right answer i times
• With probability

CCC CCW CWC CWW WCC WCW WWC WWW
28
Error Probability on k Trials

• Probability that X ? k/2
• (wrong most of the time)

29
Want Error Probability lt 5
Since the distribution on X is a sum of
distributions, the Central Limit Theorem tells
us that the distribution for X becomes Gaussian
as the number of trials, k, becomes large. If we
assume it is Gaussian, we can use a table to find
the number of trials necessary to achieve a
specified error probability given a specific
advantage. The relationship becomes
k number of times to repeat algorithm ?
advantage of the algorithm. 2.706 value
from normal (Gaussian) distribution table
30
Suppose ? .05
So a .55-correct unbiased algorithm must be
repeated 269 times to get an error probability
less than 5. What about a biased .55-correct
algorithm? Only need 4 repetitions.