Finite probability space

1
Finite probability space
set ? (sample
space) function P ?? R (probability
distribution)
? P(x) 1
x??
2
Finite probability space
set ? (sample
space) function P ?? R (probability
distribution)
? P(x) 1
x??
elements of ? are called atomic events subsets of
? are called events
probability of an event A is
? P(x)
P(A)
x?A
3
Examples
1. Roll a (6 sided) dice. What is the probability
that the number on the dice is even?
2. Flip two coins, what is the probability
that they show the same symbol?
3. Flip five coins, what is the probability
that they show the same symbol?
4. Mix a pack of 52 cards. What is the
probability that all red cards come before all
black cards?
4
Union bound
P(A ? B) ? P(A) P(B)
P(A1? A2? ? An) ? P(A1) P(A2)P(An)
5
Union bound
P(A1? A2? ? An) ? P(A1) P(A2)P(An)
Suppose that the probability of winning in a
lottery is 10-6. What is the probability
that somebody out of 100 people wins?
Ai i-th person wins somebody wins ?
6
Union bound
P(A1? A2? ? An) ? P(A1) P(A2)P(An)
Suppose that the probability of winning in a
lottery is 10-6. What is the probability
that somebody out of 100 people wins?
Ai i-th person wins somebody wins
A1?A2??A100
7
Union bound
P(A1? A2? ? An) ? P(A1) P(A2)P(An)
Suppose that the probability of winning in a
lottery is 10-6. What is the probability
that somebody out of 100 people wins?
P(A1?A2??A100) ? 10010-6 10-4
8
Union bound
P(A1? A2? ? An) ? P(A1) P(A2)P(An)
Suppose that the probability of winning in a
lottery is 10-6. What is the probability
that somebody out of 100 people wins?
P(A1?A2??A100) ? 10010-6 10-4
P(A1?A2??A100) 1P(AC1? AC2?? AC100)
1-P(AC1)P(AC2)P(AC100) 1-(1-10-6)100?
0.9910-4
9
Independence
Events A,B are independent if
P(A ? B) P(A) P(B)
10
Independence
Events A,B are independent if
P(A ? B) P(A) P(B)
observing whether B happened gives no
information on A
B
A
11
Independence
Events A,B are independent if
P(A ? B) P(A) P(B)
observing whether B happened gives no
information on A
B
P(AB) P(A?B)/P(B)
A
conditional probability of A, given B
12
Independence
Events A,B are independent if
P(A ? B) P(A) P(B)
P(AB) P(A)
13
Examples
Roll two (6 sided) dice. Let S be their sum. 1)
What is that probability that S7 ? 2) What is
the probability that S7, conditioned on
S being odd ? 3) Let A be the event that S is
even and B the event that S is odd. Are
A,B independent? 4) Let C be the event
that S is divisible by 4. Are A,C
independent? 5) Let D be the event that S is
divisible by 3. Are A,D independent?
14
Examples
A
B
C
Are A,B independent ?Are A,C independent ? Are
B,C independent ? Is it true that
P(A?B?C)P(A)P(B)P(C)?
15
Examples
Events A,B,C are pairwise independent but
not (fully) independent
A
B
C
Are A,B independent ?Are A,C independent ? Are
B,C independent ? Is it true that
P(A?B?C)P(A)P(B)P(C)?
16
Full independence
Events A1,,An are (fully) independent If for
every subset S?n1,2,,n P (
? Ai ) ? P(Ai)
i?S
i?S
17
Testing equality of strings
n-bits
Alice A 0001110100010101000111 Bob B
0001110100010101000111
slow network
n-bits
QUESTION Is AB?
18
Testing equality of strings
n-bits
n-bits
Alice A 0001110100010101000111
Bob B 0001110100010101000111
slow network
QUESTION Is AB?
Protocol 1. Alice picks a random prime p ?
n2. 2. Alice computes a(A mod p), and sends
p and a to Bob. 3. Bob computes b(B
mod p), and checks whether ab.
19
Testing equality of strings
How many bits are communicated?
Protocol 1. Alice picks a random prime p ?
n2. 2. Alice computes a(A mod p), and sends
p and a to Bob. 3. Bob computes b(B
mod p), and checks whether ab.
20
Testing equality of strings
What is the probabilty of failure?
Protocol 1. Alice picks a random prime p ?
n2. 2. Alice computes a(A mod p), and sends
p and a to Bob. 3. Bob computes b(B
mod p), and checks whether ab.
21
Testing equality of strings
What is the probabilty of failure?
BAD EVENT p divides A-B
Protocol 1. Alice picks a random prime p ?
n2. 2. Alice computes a(A mod p), and sends
p and a to Bob. 3. Bob computes b(B
mod p), and checks whether ab.
22
Testing equality of strings
What is the probabilty of failure?
BAD EVENT p divides A-B
How many (different) primes can divide an n-bit
number?
How many primes ? n2 are there?
23
Testing equality of strings
What is the probabilty of failure?
BAD EVENT p divides A-B
How many (different) primes can divide an n-bit
number?
2n ? Mp1p2pk ? 2k
k ? n
How many primes ? n2 are there?
Prime Number Theorem ? (m) ? m/ln m
number of primes ? m
24
Testing equality of strings
If AB then the algorithm always answers
YES If A?B then the algorithms answers
NO with probability ? 1- (ln n)/n
Monte Carlo algorithm with 1-sided error
25
Random variable
set ? (sample
space) function P ?? R (probability
distribution)
? P(x) 1
x??
A random variable is a function Y ? ? R
The expected value of Y is EX ?
P(x) Y(x)
x??
26
Examples
Roll two dice. Let S be their sum. If S7 then
player A gives player B 6 otherwise player B
gives player A 1
2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12
27
Examples
Roll two dice. Let S be their sum. If S7 then
player A gives player B 6 otherwise player B
gives player A 1
2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12
-1 , -1,-1 ,-1, -1, 6 ,-1 ,-1 , -1 , -1 , -1
Y
Expected income for B
EY 6(1/6)-1(5/6) 1/6
28
Linearity of expectation
EX Y EX EY
EX1 X2 Xn EX1 EX2EXn
29
Linearity of expectation
Everybody pays me 1 and writes their name on a
card. I mix the cards and give everybody one
card. If you get back the card with your name I
pay you 10.
Let n be the number of people in the class. For
what n is the game advantageous for me?
30
Linearity of expectation
Everybody pays me 1 and writes their name on a
card. I mix the cards and give everybody one
card. If you get back the card with your name I
pay you 10.
X1 -9 if player 1 gets his card back
1 otherwise EX1 ?
31
Linearity of expectation
Everybody pays me 1 and writes their name on a
card. I mix the cards and give everybody one
card. If you get back the card with your name I
pay you 10.
X1 -9 if player 1 gets his card back
1 otherwise EX1 -9/n 1(n-1)/n
32
Linearity of expectation
Everybody pays me 1 and writes their name on a
card. I mix the cards and give everybody one
card. If you get back the card with your name I
pay you 10.
X1 -9 if player 1 gets his card back
1 otherwise X2 -9 if player 2 gets his card
back 1 otherwise
EX1Xn EX1EXn n ( -9/n
1(n-1)/n ) n 10.
33
Expected number of coin-tosses until HEADS?
34
Expected number of coin-tosses until HEADS?
1/2 1 1/4 2 1/8 3 1/16
4 .
?
? n.2-n 2
n1
35
Expected number of coin-tosses until HEADS?
S
S 1 ½S
S2
36
Expected number of dice-throws until you get 6
S
37
Expected number of dice-throws until you get 6
S
S 1 (5/6)S
S6
38
Coupon collector problem
n coupons to collect What is the expected number
of cereal boxes that you need to buy?
39
Expected number of coin-tosses until 3
consecutive HEADS?
40
Markovs inequality
A group of 10 people have average income 20,000.
At most how many people in the group can have
average income at least 40,000?
A group of 10 people have average income 20000.
At most how many people in the group can have
average income at least 100,000?
41
Markovs inequality
A group of 10 people have average income 20,000.
At most how many people in the group can have
average income at least 40,000?
Let X be a random variable such that X ? 0. Then
P(X ? aEX) ? 1/a
42
Example
Alice has an algorithm A which runs in expected
running time T(n). Bob uses Alices algorithm to
construct his own algorithm B. 1. Run
algorithm A for 2T(n) steps. 2. If A
terminates then B outputs the same,
otherwise goto step 1.
What is the expected running time of B? What is
the probability that A terminates after 100T(n)
steps? What is the probability that B terminates
after 100T(n) steps?