Probability Notes

1
Probability Notes

• Math 309

2
Some Definitions

• Experiment - means of making an observation
• Sample Space (S) - set of all outcomes of an
  experiment listed in a mutually exclusive and
  exhaustive manner
• Event - subset of a sample space
• Simple Event - an event which can only happen in
  one way (or can be thought of as a sample point
  - a one element subset of S)

3
Since events are sets, we need to understand the
basic set operations

• Intersection (A B) - everything in A and B
• Union (A B) - everything in A or B or both
• Complement ( ) - everything not in A

4

• You should be able to sketch Venn diagrams to
  describe the intersections, unions, complements
  of sets.
• Note that these set operations obey the
  commutative, associative, and distributive laws

5
DeMorgans Laws

• Convince yourself that these are reasonable with
  Venn diagrams!

6
Another definition -

• A and B are mutually exclusive iff
• A ? B ?

7
Axioms of Probability(these are FACT, no proof
needed!)

• Let A represent an event, S the sample space,
• P(A) 0
• P(S) 1
• For pairwise mutually exclusive events, the
  probability of their union is the sum of their
  respective probabilities, i.e.

8
Theorems(You should be able to prove these using
the axioms and definitions.)

• Thm 1.1 - The probability of the empty set is
  zero.
• Thm 1.2 Let A1, A2, . . . ,An be a mutually
  exclusive set of events. Then
• P(A1?A2? . . . ?An) P(A1) P(A2) . . .
  P(An)

9
Propositions(You should be able to prove these
using the axioms and definitions.)

• Let E and F be any two events.
• 2.1 0 P(E) 1
• 2.2 If E is a subset of F, then P(E) P(F).
  
• 2.3 For mutually exclusive events,
  P(E F) P(E) P(F)

10
Theorems(You should be able to prove these using
the axioms and definitions.)

• Let E and F be any two events.
• Thm 2.5 P( ) 1 - P(E)
• Thm 2.6 P(E F) P(E) P(F) - P(E F)

11
Unions get complicated if events are not
mutually exclusive!
P(A ? B ? C) P(A) P(B) P(C)
- P(A ? B) - P(A ? C) - P(B ? C)
P(A ? B ? C)
B
12
More Theorems

• Let A and B be any two events.
• Thm 1.5- If A ? B,
• then P(B-A) P(B ?AC) P(B)
  P(A)
• Corollary If A is a subset of B, then P(A) lt
  P(B).
• Thm 1.7 P(A) P(A ? B) P(A ? BC)
  (generalize)

13
Sample Spaces with Equally Likely Outcomes

• In an experiment where all simple events (sample
  points) are equally likely, one can find the
  probability of an event by counting two sets.

14
Combinatorial Methods

• Math 309

15
Combinatorics

• Basic Principle of Counting
• (a.k.a. Multiplication Principle)
• Permutations
• Permutations with indistinguishable objects
• Combinations

16
Basic Counting Principle

• If experiment 1 has m outcomes and experiment 2
  has n outcomes, then there are mn outcomes for
  both experiments.
• The principle can be generalized for r
  experiments. The number of outcomes of r
  experiments is the product of the number of
  outcomes of each experiment.

17

• We define experiment as a means of making an
  observation (e.g. flip a coin, choose a color).
• Each experiment could be making a choice from a
  different set.

18
Permutations

• of arrangements of one set, order matters
• application of the basic counting principle where
  we return to the same set for the next selection
• P(n,r) n!/(n-r)!

19
Permutations with Indistinguishable Objects

• Order the objects as if they were distinguishable
• Then divide out those arrangements that look
  identical.

20
Combinations

• the number of selections, order doesnt matter
• C(n,r) n!/(n-r)!r!
• the number of arrangements can be counted by
  selecting the objects and then ordering them
• i.e. P(n,r) C(n,r)r!

21
Observations about Combinations

• C(n, r) C(n, n-r)
• C(n, n) C(n, 0) 1
• C(n, 1) n C(n, n-1)
• C(n, 2) n(n-1)/2

22
Combining Counting Techniques

• If we are careful with language,
• when we say AND, we multiply
• AND ? multiplication ? intersection
• when we say OR, we add
• OR ? addition ? union

23
Conditional Probability and Independence
24
Conditional Probability P(AB)

• P(AB) is read,
• the probability of A given B
• B is known to occur.

25
The multiplication rule and ?
intersection?multiply

• P(A ? B) P(A)P(BA)
• P(B)P(AB)
• Intersections get more complicated when there are
  more events, e.g.
• P(A?B?C?D)
• P(A) P(BA)P(CA?B)P(DA ?B?C)

26
Independent Events

• A and B are independent if any of the following
  are true
• P(A?B) P(A)P(B)
• P(AB) P(A)
• P(BA) P(B)
• You need to check probabilities to determine if
  events are independent.
• If A, B, C, D are pairwise independent,
• P (A?B ?C ?D) P(A)P(B)P(C)P(D)

27
Conditional Probability P(AB) Formula

• P(AB) P(A ? B) / P(B), if P(B) gt 0
• (Note that this is an algebraic manipulation of
  the formula for the probability of the
  intersection of 2 events.)
• i.e. the conditional probability is the
  probability that both occur divided by what is
  given occurs