Discrete Probability Probability Theory

1
Discrete ProbabilityProbability Theory
2
Learning Objectives

• Define the terminology of discrete probability
  theory experiment, sample space, event.
• Define the probability of an event.
• Define the probability of combinations of events.
• Example of probabilistic reasoning.
• Define the probability of an event as the sum of
  individual outcomes probabilities.
• Define a conditional probability.
• Define the probability of independent events.
• Define Bernoulli binomial distribution.
• Define random variables and their values.

3
Discrete Probability

• Probability theory relies on combinatorics
  (counting groups of elements with permutations,
  arrangements and combinations).
• Experiment an experiment is a procedure that
  yields one of a given set of possible outcomes.
• Sample space the set of possible outcomes of an
  experiment.
• Event an event is a subset of the sample space.
• Finite probability for experiments having
  finitely many, equally likely, outcomes.

4
Discrete Probability

• Finite probability the probability of an event
  E, which is a subset of a finite sample space S
  of equally likely outcomes is p(E) E / S
• (number of favorable outcomes / number of
  outcomes).
• Example
• What is the probability, when one die is
  rolled, that number 6 shows ?
• p(E) 1 / 6

5
Discrete Probability

• Example
• What is the probability, when two dice are
  rolled, that the sum is 12 ?
• p(E) ?
• What is the probability, when two dice are
  rolled, that the sum is 7 ?
• Favorable events ?
• p(E) ?

6
Discrete Probability

• Probability of combinations of events let E be
  an event in a sample space S. The probability of
  the event , the complementary of E, is
• Example what is the probability that two dice
  never come up with a sum of 7 ?
• p(E) p(sum7) 1 / 6
• p( ) 1 - p(E) 1 - 1 / 6 5 / 6

7
Discrete Probability

• Probability of the union of two eventsLet E1
  and E2 be two events in the same space S. Then
• Example probability of having 1 on the first die
  or 2 on the second when die is thrown twice?
• p(E1) 1 . 6 / 36 6 / 36
• p(E2) 6 . 1 / 36 6 / 36
• p(E1?E2) 6/36 6/36 - 1/36 11 / 36

8
Discrete Probability

• Probabilistic reasoning Lets make a deala
  large prize is behind one of three doors you are
  asked to select one door, but not to open it the
  game show host then opens another door which is
  not a winning door, and asks you whether you want
  to choose another door. What should you do ?

9
Discrete Probability

• p(winning door) 1/3
• p(winning door after choosing to keep it) 1/3
• p(losing door) 2/3
• p(changing door wins after losing door) 2/3
  (the price is behind the other door).
• It is better to change door.

10
Probability Theory

• A sample space S is a set of individual outcomes
  s.
• Properties of p(s)
• We study only finite sample spaces the number
  of outcomes is finite.

11
Probability Theory

• Example
• tossing a fair coin p(H) p(T) 1/2
• tossing a biased coin where heads come up twice
  as often as tails p(H) 2 p(T)
• p(H) p(T) 1
• p(T) 1/3
• p(H) 2/3

12
Probability Theory

• For an event, we have the property
• the probability of an event is the sum of the
  probabilities of the individual events.
• For pairwise disjoint events

13
Probability Theory

• Conditional probability
• Let E and F be events with p(F) gt 0. The
  conditional probability of E given F, denoted
  p(E/F), is defined as
• Independent events
• Two events E and F are said to be independent if
  and only if p(E?F) p(E) . p(F)

14
Probability Theory

• Examples
• tossing a fair coin three times, what is the
  probability of the event E that an odd number of
  tails appears, knowing that the first coin showed
  tails (event F) ?
• p(E/F) p(E?F)/p(F)
• E?FTTT, THH
• p(E?F) 2/8 1/4
• p(F) 1/2
• p(E/F) 2/4 1/2

15
Probability Theory

• E TTT, THH, HTH, HHT
• p(E) 4/8 1/2
• p(E?F) 1/4
• p(F) 1/2
• We notice that E and F are independent because
  p(E?F) p(E) . p(F)

16
Probability Theory

• Bernoulli binomial distribution
• a Bernoulli trial is an experiment with two
  possible outcomes.
• An outcome of a Bernoulli trial is called a
  success or failure.
• The probability of k successes in n independent
  Bernoulli trials, with probability of success p
  and probability of failure q 1-p is
  C(n,k)pkqn-k.

17
Probability Theory

• Proof n Bernoulli trials can be represented in
  a n-tuple (t1, t2, , tn). The n trials are
  independent, so the probability will be the
  product of the probabilities for each trial
  pkqn-k.
• There are C(n,k) ways of choosing the k trials
  that have success. Hence the result.
• Example tossing a coin 100 times, what is the
  probability of seeing 20 heads ?
• p(E) C(100,20)(1/2)20(1/2)80

18
Probability Theory

• Random variables a random variable is a
  function from the sample space of an experiment
  to the set of real numbers. It assigns a real
  number to each outcome.
• Example a coin is tossed 3 times. X(t) is the
  number of heads that appear when t is the
  outcome.
• X(HHT) 2
• X(TTT) 0

19
Probability Theory

• Example the birthday problem
• what is the minimum number of people who need
  to be in a room so that the probability that at
  least two of them have the same birthday is
  greater than ½ ?
• birthdays of people are independent
• pn probability that n people have different
  birthdays

20
Probability Theory

• Monte Carlo algorithms
• Probabilistic algorithm an algorithm that makes
  random choices at one or more steps (non
  deterministic).
• Monte Carlo algorithm an algorithm that produces
  answers to problems, but a small probability
  remains that these answers my be incorrect. Uses
  sequences of random numbers to perform computer
  simulations.

21
Probability Theory

• Example calculate the area under a curve as a
  fraction of a rectangular box containing the
  curve.Estimate the
• area under
• the curve
• by the
• fraction of
• points under
• the curve.

22
Probability Theory

• Independent random variables
• The random variables X and Y on a sample space S
  are independent if
• p(X(s)r1 AND Y(s)r2)p(X(s)r1).p(X(s)r2)
• Example a die is rolled twice. X1 is the number
  of spots on first roll, and X2 is the number of
  spots on second roll.
• p(X1i and X2j) 1/36
• p(X1i) 1/6 and p(X2j) 1/6
• Thus the variables are independent.

23
Probability Theory

• The probabilistic method (theorem) if the
  probability that an element of a set S does not
  have a particular property is less than 1, there
  exists an element in S with this property.