Probability Theory

1
Probability Theory

• We ended last time by saying that in order to
  reason effectively and accurately about questions
  of the form How likely is it that? we need to
  use some formal methods.
• These methods are those of probability theory
   the mathematical approach to the concept of
  chance and related concepts.

2
A Priori Probability

• Weve already looked at one way of trying to
  reason about likelihood by looking at the
  properties of samples and inferring properties of
  populations.
• A second way is by reasoning about the sample.
• This is called the a priori approach (because it
  doesnt involve experimenting with the
  population by random sampling).

3
Illustration

• What is the probability of drawing an ace from a
  standard deck of 52 cards?
• Argue as follows there are four aces in a
  standard deck. So the probability of pick an ace
  is four in 52 (one in 13).

4
Calculating Probabilities

• Our method of calculating the probability of
  picking an ace can be represented as follows
• Pr(picking an ace) number of aces
• number of cards
• In general, the probability of some event
  occurring or some hypothesis being true can be
  represented as follows
• Pr(h) favorable outcomes
• total outcomes
• Note the outcomes have to be equally likely
  otherwise this will not work. (E.g., the
  probability of a coin landing heads is only 1/2
  if the coin is unbiased.)

5
Rule 1 Non-Occurrence

• Probabilities are conventionally represented on a
  scale from 0 to 1.
• An event with probability 1 will certainly occur
  an event with probability 0 will certainly not
  occur.
• From this we get the first rule of probability
• Rule 1 The probability that an event will not
  occur is 1 minus the probability that it will
  occur
• Pr(not h) 1 - Pr(h)

6
Rule 2 Joint Pr

• Rule 2 Given two independent events, the
  probability of their both occurring is the
  product of their individual probabilities
• Pr(h1 h2) Pr(h1) x Pr(h2)
• Intuition the probability of the conjunction is
  less than the probability of each conjunct.

7
Independence

• Rule 2 Pr(h1 h2) Pr(h1) x Pr(h2)
• It is crucial here that h1 and h2 are independent
  that is, that occurrence of h1 provides no
  information about the occurrence of h2.
• Why?
• Suppose the in the population at large, the
  following holds
• The probability of taking early retirement is
  .33
• The probability of smoking three packs a day is
  .33
• The probability of getting emphysema is .005

8
Independence

• Now compare
• h1 Ian takes early retirement
• Pr(h1) .33
• h2 Ian gets emphysema
• Pr(h2) .005
• and
• h1 Ian smokes three packs a day
• Pr(h1) .33
• h2 Ian gets emphysema
• Pr(h2) ?

9
Conditional Pr

• In order to deal with cases in which two events
  are not independent, we need to introduce the
  notion of conditional probability.
• The probability of an event, h2 occurring given
  that h1 has occurred is called the conditional
  probability of h2 on h1 and is represented
• Pr(h2h1)
• Note The symbol / sometimes used in
  representations of conditional probability is not
  the symbol for arithmetic division.

10
Rule 2G Conditional Pr

• Given this idea, we can modify Rule 2 for events
  that are not independent
• Rule 2G Given two events, the probability of
  their both both occurring is the probability of
  the first occurring times the probability of the
  second occurring, given that the first has
  occurred
• Pr(h1 h2) Pr(h1) x Pr(h2h1)
• Note We can extend the rules above for more than
  two events.

11
Rule 2G Conditional Pr

• Rule 2G Pr(h1 h2) Pr(h1) x Pr(h2h1)
• Pr(Ian smokes three packs a day) (Ian gets
  emphysema)
• Pr(Ian smokes three packs a day) x Pr(Ian gets
  emphysema given that Ian smokes three packs a
  day)
• Notice that the probability that Ian gets
  emphysema given that Ian smokes three packs a day
  is considerably higher than the mere probability
  that Ian gets emphysema (which might be very rare
  in the population at large).
• So Pr (h1) x Pr(h2h1) will be higher in this
  case than Pr(h1) x Pr(h2).

12
Rule 3 Exclusion

• What about events that are mutually exclusive?
  That is, if h1 occurs, then h2 does not and vice
  versa?
• An example would be rolling dice if you roll a
  six, you cant roll a two.
• Rule 3 The probability that at least one of two
  mutually exclusive events will occur is the sum
  of the probabilities that each of them will
  occur
• Pr(h1 or h2) Pr(h1) Pr(h2)
• Intuition The probability of rolling either a
  six or a two is higher than the probability of
  rolling either one of them.

13
Rule 3G Non-Exclusion

• What about if the events are not mutually
  exclusive?
• Consider the case in which half class is over 19
  and half is under and half are men and half
  women (and the age distribution is the same for
  men and women).
• If we used the rule Pr(h1 or h2) Pr(h1)
  Pr(h2) for calculating the chances of finding
  either a woman or someone over 19, wed get 1  a
  certainty!

14
Rule 3G Non-Exclusion
But this isnt the case
women men
Green under 19 White over 19
A woman or over 19
15
Rule 3G Non-Exclusion

• We therefore need a different rule for these
  cases
• Rule 3G The probability that at least one of two
  events will occur is the sum of the
  probabilities that each of them will occur,
  minus the probability that they will both occur
• Pr(h1 or h2) Pr(h1) Pr(h2) - Pr(h1 h2)

Yellow woman over 19
16
Rule 4 Single Occurrence

• What are the chances of tossing a heads once in
  eight tosses?
• That is equivalent to the probability that it
  will not come up tails eight times in a row.
• The probability that it will come up eight times
  in a row is 1/2 multiplied by itself eight times.
  
• And the probability that this will not occur is 1
  minus that value.

17
Rule 4 Single Occurrence

• We can generalize this idea as follows
• Rule 4 The probability that an event will occur
  at least once in a series of independent trials
  is simply 1 minus the probability that it will
  not occur in that number of trials
• The probability that h will occur at least once
  in n trials
• 1 - Pr(not h)n