Law of Total Probability Bayes Theorem

1
Chapter 2, Section 10

• Law of Total ProbabilityBayes Theorem

? John J Currano, 02/09/2009
2
The Law of Total Probability

• S sample space
• B1, B2, ??, Bk partition S that is
• A is any event (any subset of S)
• Then A ? B1, A ? B2, ? , A ? Bk partition A
  that is

3
The Law of Total Probability

• S sample space
• B1, B2, ? , Bk partition S
• A is any event (any subset of S)
• Then A ? B1, A ? B2, ? , A ? Bk partition A,
  and
• P(A) P(A ? B1) ??? P(A ? Bk)
• P(B1) P(A B1) ??? P(Bk) P(A
  Bk)

This result follows from 1. countable
additivity (since the A ? Bi partition A), and
2. the multiplicative law of probability P(A ?
Bi)P(Bi) P(A Bi).
4
The Law of Total Probability

• B1, B2, ??, Bk partition S A ? S ?
• P(A) P(A ? B1) ??? P(A ? Bk)
• P(B1) P(A B1) ??? P(Bk) P(A Bk)

• This theorem is useful in the following fairly
  common situation
• We need to compute the probability of an event A,
  and
• the sample space is partitioned into k?? 2
  events, B1, ? , Bk whose probabilities are
  either known or easy to compute
• the conditional probability of A given each Bi
  is either known or easy to compute.

5

• Law of Total Probability B1, B2, B3 partition
  S ?
• P(A) P(A ? B1) P(A ? B2) P(A ? B3)
• P(B1) P(AB1) P(B2) P(AB2) P(B3)
  P(AB3)

• Example. Suppose that a bag contains 12 coins
• 5 are fair
• 4 are biased with probability of heads 1/3 and
  
• 3 are two-headed.
• A coin is chosen at random from the bag and
  tossed.
• Find the probability that the coin is heads.
• Given that the coin is heads, find the
  conditional probability of each coin type.

6

• Law of Total Probability B1, B2, B3 partition
  S ?
• P(A) P(A ? B1) P(A ? B2) P(A ? B3)
• P(B1) P(AB1) P(B2) P(AB2) P(B3)
  P(AB3)

P(B1) 5/12 P(B2) 4/12 P(B3) 3/12 P(A B1)
1/2 P(A B2) 1/3 P(A B3) 1

• Example. Suppose that a bag contains 12 coins
• 5 are fair
• 4 are biased with probability of heads 1/3
• 3 are two-headed.
• A coin is chosen at random and tossed.
• Find the probability of event A the coin is
  heads

B1 B2 B3
P(A) P(B1) P(AB1) P(B2) P(AB2) P(B3)
P(AB3) (5/12)(1/2)
(4/12)(1/3) (3/12)(1)
(5/24) (4/36) (3/12)
41/72
7

• Law of Total Probability B1, B2, B3 partition
  S ?
• P(A) P(A ? B1) P(A ? B2) P(A ? B3)
• P(B1) P(AB1) P(B2) P(AB2) P(B3)
  P(AB3)

• P(B1) 5/12, P(B2) 4/12, P(B3) 3/12,
• P(A B1) 1/2, P(A B2) 1/3, P(A B3)
  1,
• P(A) P(B1) P(AB1) P(B2) P(AB2) P(B3)
  P(AB3) 41/72

2. Given that the coin is heads (A), find the
conditional probability of each coin type.
8

• Bayes Theorem. Given
• S sample space
• B1, B2, ??, Bk partition S that is
• A is any event (any subset of S)
• Then for i 1, 2, , k,

9

• Example. If a water specimen contains nitrates
  and is tested using a colormetric test, it will
  turn red 95 of the time. When the test is used
  on specimens that do not contain nitrates, the
  water turns red 10 of the time. From past
  experience we are confident that 30 of the water
  specimens tested at a particular lab contain
  nitrates.
• A turns red B contains nitrates
• If a water specimen is randomly chosen from those
  sent to the lab for testing, find the probability
  that it will turn red when tested. 0.355
• If a water specimen is randomly selected and
  turns red, what is the probability that it
  actually contains nitrates? 0.8028

B1 B B2 B
Find P (A)
Find P ( B A)
Detailed solutions using the formulas are on the
class website. Well use a tree.
10
Example. If a water specimen contains nitrates
and is tested using a colormetric test, it will
turn red 95 of the time. When the test is used
on specimens that do not contain nitrates, the
water turns red 10 of the time. From past
experience we are confident that 30 of the water
specimens tested at a particular lab contain
nitrates. A turns red B contains nitrates
a. Find P (A) b. Find P ( B A) Given
information 0.95 0.10 0.30
P(B)
B1 B B2 B
P(A B) P(A B) P(B)
0.70
11
Example. Using the texts tree applet B
contains nitrates B A turns red P(B)
0.30 P(AB) 0.95 P(AB) 0.10