Basic Axioms of Probability

1
Basic Axioms of Probability

• Lecture II

2
Basis of Probability

• Using the example from Birenens Chapter 1 Assume
  we are interested in the game Texas lotto
  (similar to Florida lotto).
• In this game, players choose a set of 6 numbers
  out of the first 50. Note that the ordering does
  not count so that 35,20,15,1,5,45 is the same of
  35,5,15,20,1,45.
• How many different sets of numbers can be drawn?
• First, we note that we could draw any one of 50
  numbers in the first draw.

3

• However for the second draw we can only draw 49
  possible numbers (one of the numbers has been
  eliminated). Thus, there are 50 x 49 different
  ways to draw two numbers
• Again, for the third draw, we only have 48
  possible numbers left. Therefore, the total
  number of possible ways to choose 6 numbers out
  of 50 is

4

• Finally, note that there are 6! ways to draw a
  set of 6 numbers (you could draw 35 first, or 20
  first, ). Thus, the total number of ways to draw
  an unordered set of 6 numbers out of 50 is
• This is a combinatorial. It also is useful for
  binomial arithmetic

5

• Basic definitions
• Sample Space The set of all possible outcomes.
• In the Texas lotto scenario, the sample space is
  all possible 15,890,700 sets of 6 numbers which
  could be drawn.
• Event A subset of the sample space.
• A subset of the sample space. In the Texas lotto
  scenario, possible events include single draws
  such as 35,20,15,1,5,45 or complex draws such
  as all possible lotto tickets including
  35,20,15. Note that this could be
  35,20,15,1,2,3, 35,20,15,1,2,4,.

6

• Simple Event An event which cannot be a union of
  other events
• An event which cannot be a union of other events.
  In the Texas lotto scenario, this is a single
  draw such as 35,20,15,1,5,45.
• Composite Event An event which is not a simple
  event.

7
Axiomatic Foundations

• A set
• of different combinations of outcomes is called
  an event. These events could be simple events or
  compound events. In the Texas lotto case, the
  important aspect is that the event is something
  you could bet on (for example, you could bet on
  three numbers in the draw 35,20,15).

8

• A collection of events F is called a family of
  subsets of sample space O. This family consists
  of all possible subsets of O including O itself
  and the null-set F.
• Following the betting line, you could bet on all
  possible numbers (covering the board) so that O
  is a valid bet.
• Alternatively, you could bet on nothing, or F is
  a valid bet.

9

• Next, we will examine a variety of closure
  conditions. These are conditions that guarantee
  that if one set is an contained in a family,
  another related set must also be contained in
  that family.
• First, we note that the family is closed under
  complementarity

10

• Second, we note that the family is closed under
  union
• Definition 1.1 (Bierens) A collection F of
  subsets of a nonempty set O satisfying closure
  under complementarity and closure under union is
  called an algebra.
• Adding closure under infinite union defined as

11

• Definition 1.2 (Bierens) A collection F of
  subsets of a nonempty set O satisfying closure
  under complementarity and infinite union is
  called a s-algebra (sigma-algebra) or a Borel
  Field.

12
(No Transcript)
13

• We typically think of this as an odds function
  (i.e., what are the odds of a winning lotto
  ticket? 1/15,890,700).
• To be mathematically precise, suppose we define a
  set of events
• say that we choose n different numbers. The
  probability of winning the lotto is

14

• Our intuition would indicate that ,
• or the probability of winning given that you
  have covered the board is equal to one (a
  certainty).
• Further, if you dont bet the probability of
  winning is zeros or

15

• Definition 1.2.2 (Cassella and Berger) Given a
  sample space O and an associated Borel field B, a
  probability function is a function P with domain
  B that satisfies
• P (A)?0 for all A?B.
• P (O)1.
• If A1,A2,?B are pairwise disjoint, then
  P(??i1Ai)??i1P (Ai)

16

• Axioms of Probability
• P(A) ? 0 for any event A.
• P(S) 1 where S is the sample space.
• If Ai, i1,2,, are mutually exclusive (that
  is, Ai?Aj? for all i?j), then P(A1?A2?)P(A1)P(
  A2)

17

• In a little more detail from Casella and Berger
• Definition 1.1.1 The set, S, of all possible
  outcomes of a particular experiment is called the
  sample space for the experiment.
• Definition 1.1.2 An event is any collection of
  possible outcomes of an experiment, that is, any
  subset of S (including S itself).

18

• Defining the subset relationship
• A ? B ? x ? A ? x?B
• A B ? A ? B and B ? A
• Union The union of A and B, written A ? B, is
  the set of elements that belong to either A or B.
• Intersection The intersection of A and B,
  written A ? B, is the set of elements that belong
  to both A and B.

19

• Complementation The complement of A, written Ac,
  is the set of all elements that are not in A.

20

• Theorem 1.1.1 For any three events A, B, and C
  defined on a sample space S,
• Commutativity A ? BB ? A, A ? BB ? A.
• Associativity A ? (B ? C)(A ? B) ? C, A ? (B ?
  C)(A ? B) ? C
• Distributative Laws A ?(B ?C )(A ?B )?(A ? C
  ),A ?(B ?C )(A ? B )?(A ?C )
• DeMorgans Laws (A ?B )cAc ? Bc, (A ?B )cAc
  ?Bc

21
Counting Techniques

• Simple Evens with Equal Probabilities
• the probability of event A is simply the
  number possible occurrences of A divided by the
  number of possible occurrences in the sample.

22

• Definition 2.3.1 The number of permutations of
  taking r elements from n elements is a number of
  distinct ordered sets consisting of r distinct
  elements which can be formed out of a set of n
  distinctive elements and is denoted Pnr.

23

• The first point to consider is that of
  factorials. For example, if you have two objects
  A and B, how many different ways are there to
  order the object? Two
• A, B or B, A
• If you have three orderings how many ways are
  there to order the objects? Six
• A, B, C, A, C, B, B, A, C, B, C, A, C,
  A, B, or C, B, A

24

• The sequence then becomes two objects can be
  drawn in two sequences, three objects can be
  drawn in six sequences (2 x 3). By inductive
  proof, four objects can be drawn in 24 sequences
  (6 x 4).
• The total possible number of sequences is then
  for n objects is n! defined as
• n!n (n -1)(n -2)1

25

• Theorem 2.3.1 Pnrn!/(n-r)!.
• Definition 2.3.2 The number of combinations of
  taking r elements from n elements is the number
  of distinct sets consisting of r distinct
  elements which can be formed out of a set of n
  distinct elements and is denoted Cnr.

26
Conditional Probability and Independence

• In order to define the concept of a conditional
  probability it is necessary to discuss joint
  probabilities and marginal probabilities.
• A joint probability is the probability of two
  random events. For example, the draw of two
  cards from a deck of cards. There are 52x512652
  different combinations of the first two cards
  from the deck.

27

• The marginal probability is overall probability
  of a single event or the probability of drawing a
  given card.
• The conditional probability of an event is the
  probability of that event given that some other
  event has occurred.
• In the textbook, what is the probability of the
  die being a one if you know that the face number
  is odd? (1/3).
• Note if you know that the role of the die is a
  one, then the probability of the role being odd
  is 1.

28

• Axioms of Conditional Probability
• P (AB )?0 for any event A.
• P (AB )1 for any event A ? B.
• If Ai?B, i1, 2, are mutually exclusive, then
  P(A1?A2?B )P(A1B )P(A2B).
• If B ?H and B ?G and P (G )?0, then

29
Bayes Theorem

• Theorem 2.4.1 P (AB )P (A?B )/P (B) for any
  pair of events A and B such that P (B)gt0.
• Theorem 2.4.2 (Bayes Theorem) Let Events A1, A2,
  , An be mutually exclusive such that P
  (A1?A2??An)1 and P (Ai) gt0 for each i. Let E
  be an arbitrary event such that P (E)gt0. Then

30

• Another manifestation of this theorem is from the
  joint distribution function
• The bottom equality reduces the marginal
  probability of event E

31

• This yields a friendlier version of Bayes theorem
  based on the ratio between the joint and marginal
  distribution function

32
Statistical Independence

• Statistical independence is when the probability
  of one random variable is independent of the
  probability of another random variable.
• Definition 2.4.1 Events A and B are said to be
  independent if P (A)P (AB ).

33

• Definition 2.4.2 Events A, B, and C are said to
  be mutually independent if the following
  equalities hold
• P (A?B )P (A )P (B )
• P (A?C )P (A )P (C )
• P (B?C )P (B )P (C )
• P (A?B ?C )P (A )P (B)P (C )