Probability

1
Probability

• Sample spaces, events, probabilities, conditional
  probabilities, independence, Bayes formula

2
Sample spaces and events

• Envision an experiment for which the result is
  unknown.
• The collection of all possible outcomes is called
  the sample space.
• Sample spaces can be discrete HH, HT, TH, TT
  for two coins
• Or continuous, e.g., 0, ?),
• A set of outcomes, or subset of the sample space,
  is called an event. If E and F are events,
• E?F is the event that either E or F (or both)
  occurs
• E?F EF is the event that both E and F occur.
  If EF ? then E and F are called mutually
  exclusive.
• The complement of E is the event
  that E does not occur

3
Probability

• A probability space is a three-tuple (S ,?, P)
  where S is a sample space, ? is a collection of
  events from the sample space and P is a
  probability law that assigns a number to each
  event in ?. P(?) must satisfy
• P(S) 1
• 0 ? P(A) ? 1
• For any collection of mutually exclusive events
  E1, E2, ,
• If S is discrete, then ? is the set of all
  subsets of S
• If S is continuous, then ? can be defined in
  terms of basic events of interest, e.g., if S
  0, 1 the basic events could be all (a, b)
  with 0 ? a lt b ? 1. Then ? would be the set of
  intervals along with all their countable unions
  and intersections.

4
Probability

• If S consists of n equally likely outcomes, then
  the probability of each is 1/n.
• Since E and Ec are mutually exclusive and E ? Ec
  S,
• P(E ? F) P(E) P(F) P(EF)
• P E ? F ? G) P(E) P(F) P(G) P(EF) P(EG)
  P(FG) P(EFG)
• Generalizes to any number of events.
• Use Venn diagrams!

5
Conditional Probabilities

• If A and B are events with P(B) ? 0, the
  conditional probability of A given B is
• This formula also tells how to find the
  probability of AB
• A and B are independent events if
• or equivalently if
• If events are mutually exclusive, are they
  independent? Vice versa?
• A set of events E1, E2, , En are independent if
  for every subset E1, E2, , Er,
• (pairwise independence is not enough see
  Example 1.10)

6
Bayes Formula

• The law of total probability says that if E and F
  are any two events, then since E EF ? EFc,
• It can be generalized to any partition of S F1,
  F2, , Fn mutually exclusive events with
• Bayes formula is relevant if we know that E
  occurred and we want to know which of the Fs
  occurred.