Computing Fundamentals 2 Lecture 6 Probability

1
Computing Fundamentals 2Lecture 6 Probability

• Lecturer Patrick Browne
• http//www.comp.dit.ie/pbrowne/

2
Probability

• If a die is thrown we consider it certain that it
  will land, with a random chance that it will show
  a 6. With s successes out of n experiments fs/n
  is called the relative frequency of success. It
  becomes stable in the long run. It is this long
  term stability (limit) that forms the basis of
  probability.

3
Sample Space and Events

• The sample space S is the set of all possible
  outcomes of a given experiment.
• An element or outcome in S is called a sample
  point (or sample).
• An event A is a set of outcomes, it is a subset
  of the sample space.
• The singleton a where a ? S is called an
  elementary event.
• The empty set, ?, sometimes represents an
  impossible event.

4
Sample Space and Events

• An event gives rise to a set hence we can use set
  operations to combine events.
• A ? B is the event that occurs whenever A occurs
  or B occurs (or both)
• A ? B is the event that occurs whenever A and B
  both occur.
• Ac is the event that occurs whenever A does not
  occur (called the complement of A)
• Two events are mutually exclusive if they are
  disjoint A ? B ?.

5
Sample Space and Events

• Toss a die and observe the top number
  S1,2,3,4,5,6
• A even number event, B odd number event, C prime
  number event.
• A2,4,6 B1,3,5 C2,3,5
• A ? C 2,3,4,5,6
• B ? C 3,5
• Cc 1,4,6
• A and B are mutually exclusive.

6
Sample Space and Events

• Toss 3 coins and observe the H T sequence
• SHHH,HHT,HTH,HTT,THH,THT,TTH,TTT
• Let A be the two consecutive heads event, B same
  outcome event.
• AHHH,HHT,THH BHHH,TTT
• A ? B HHH is the elementary event with only
  heads.

7
Probability Spaces

• A probability space is a triple (S, A, P), where
• S sample space, all possible outcomes.
• A event space, sample events/outcomes
• P is a probability measure.
• We also have a set of probability axioms e.g. the
  probability of an event is a non-negative real
  number.

8
Probability Spaces

• A probability space consists of a sample space
  together with a positive, additive measure,
  called a probability measure, which sums to one
  the points of the sample space represent the
  different possible outcomes of the phenomenon,
  and the probability measure assigns probabilities
  to sets of outcomes.

9
Finite Probability Spaces

• Let S be a finite sample space
• Sa1,a2,a3...an. A finite probability space
  is obtained by assigning to each sample point
  ai?S a real number pi, called the probability of
  ai satisfying the following conditions
• Each pi is non-negative.
• The sum of pi is one.
• We write P(A) for the sum of the probabilities
  sample points in A.

10
Finite Probability Spaces

• Three runners A,B,C A is twice a likely to win
  as B, and B is twice as likely to win as C. What
  is P(A),P(B),P(C) winning?
• Let P(C) p
• P(B) 2p
• P(A)4p
• p2p4p1 therefore p 1/7
• P(A)4/7, P(B)2/7, P(C)1/7
• P(B,C) P(B)P(C)3/7

11
Equiprobable Spaces

• If all the sample points within a given finite
  probability space are equal to each other, then
  it is known as an equiprobable space. An example
  would be a fair die, where each number is equally
  possible
• P(1) P(2) P(3) P(4) P(5) P(6) 1/6

12
Equiprobable Spaces

• If S contains n points, then the probability of
  each point is 1/n. If an event A contains r
  points then its probability is
• r ? 1/n r/n
• P(A) number of elements in A
• number of elements in S

13
Equiprobable Spaces

• S cards in the deck 52
• A card is spade
• B card is a face
• P(A) 13/52
• P(B) 12/52
• P(A?B) 3/52

14
Axioms of Finite Probability Spaces

• For every event A, 0?P(A)?1
• P(S)1, where S is sample space,
• If events A and B are mutually exclusive (or
  disjoint), then
• P(A?B) P(A) P(B)

15
Theorems of Finite Probability Spaces

• P(?) 0
• P(Ac) 1 P(A)
• P(A\B)P(A) - P(A?B)
• A?B implies P(A)?P(B)
• P(A) ? 1
• P(A?B) P(A) P(B) - P(A?B)
• P(A?B) P(A) P(BA)
• Where P(BA) reads the probability B given A

16
Addition

• P(A?B) P(A) P(B) - P(A?B)
• Sums are used when we have two events, and we
  want to know the probability that either event
  occurs (Event A union Event B). In the Addition
  Rule, A and B may or may not be disjoint.
  Mutually exclusive or disjoint events cannot
  occur together, so we have P(A ? B) 0.
• Then the addition rule reduces to P(A U
  B) P(A) P(B)

17
Addition Rule Example

• Suppose a student is selected at random from 100
  students where 30 are taking maths, 20 are taking
  chemistry, and 10 are taking maths and chemistry.
  Find the probability p that the student is taking
  maths or chemistry.
• P(M) 30/100, P(C)20/100
• P(M ? C) 10/100
• P(M?C)P(M) P(C) - P(M?C)
• P(M?C) 30/10020/10010/1002/5

18
Rule of Multiplication

• Is used when we want to know the probability that
  two events occur (Event A intersection Event B).
• Rule of Multiplication The probability that
  Events A and B both occur is equal to the
  probability that Event A occurs times the
  probability that Event B occurs, given that A has
  occurred.
• P(A?B) P(A) P(BA)

19
Rule of Multiplication

• A bag contains 6 red marbles and 4 blue marbles.
  Two marbles are drawn without replacement from
  the bag. What is the probability that both of the
  marbles are blue?
• A first marble is blue, B second marble is
  blue.
• Therefore, P(A) 4/10, P(BA) 3/9.
• Using P(A n B) P(A) P(BA)
• P(A n B) (4/10) (3/9) 12/90 2/15

20
Rule of Multiplication

• A bag contains 6 red marbles and 4 blue marbles.
  Two marbles are drawn with replacement from the
  bag. What is the probability that both of the
  marbles are blue?
• A first marble is blue, B second marble is
  blue.
• Therefore, P(A) 4/10, P(BA) 4/10.
• Using P(A n B) P(A) P(BA)
• P(A n B) (4/10) (4/10) 16/100 4/25

21
Conditional Probability

• E is an event in S with P(E)gt0. Conditional
  probability of A is defined as the probability
  that A has occurred after E has occurred. We say
  the conditional probability of A given E
• P(AE) P(A?E)
• P(E)
• P(AE) number of elements in A?E
• number of elements in E
  

22
Example Conditional Probability

• Alternatively
• P(AE) number of ways A and E can occur
• number of ways E can occur
• Given the sum of a pair of tossed die is 6.
• Esum is 6,5 ways
• (1,5),(2,4),(3,3),(4,2),(5,1)
• A has at least one two,2 ways
• (2,4),(4,2)
• P(AE)2/5

23
Example 2 Conditional Probability

• P(AE) number of ways A and E can occur
• number of ways E can occur
• From a class has 12 boys and 4 girls, 3 students
  are selected. What is the probability that they
  are all boys?
• PComb(12,3)/Comb(16,3)11/28
• Alternatively
• P(12/16)(11/15)(10/14) 11/28

24
Independence

• Two events are independent if the occurrence of
  one of the events gives us no information about
  whether or not the other event will occur that
  is, the events have no influence on each other.
• We say that two events, A and B, are independent
  if the probability that they both occur is equal
  to the product of the probabilities of the two
  individual events, i.e.
• P(A?B) P(A) ? P(B)
• If two events are independent then they cannot be
  mutually exclusive (disjoint) and vice versa.

25
Example Independence

• Events A and B are independent if
• P(AnB) P(A) P(B)
• otherwise they are dependent.
• A coin tossed three times
• SHHH,HHT,HTH,HTT,THH,THT,TTH,TTT
• Afirst toss head
• Bsecond toss head
• Cexactly 2 heads tossed in a row

26
Example Independence

• Continuing, coin tossed three times
• P(A)HHH,HHT,HTH,HTT4/8 (1st head)
• P(B)HHH,HHT,THH,THT4/8 (2nd head)
• P(C)HHT,THH 1/4 (2 heads in row)
• P(A?B)P(HHH,HHT) 1/4
• P(A?C)P(HHT) 1/8
• P(B?C)P(HHT,THH) 1/4

27
Example Independence

• Continuing, coin tossed three times
• P(A?B)P(HHH,HHT) 1/4
• P(A?C)P(HHT) 1/8
• P(B?C)P(HHT,THH) 1/4
• P(A)P(B)(1/2)?(1/2)(1/4) P(A?B)
• P(A)P(C)(1/2)?(1/4)(1/8) P(A?C)
• P(B)P(C)(1/2)?(1/4)(1/8)? P(B? C)

Not independent, B and C are dependent.
28
Repeated Trials

• The Law of Averages states, in the long run, over
  repeated trials, random fluctuations eventually
  average out and the average of our observations
  will approach the expected value. But at the same
  time with increasing numbers of observations, the
  number of observations that differ from what we
  expect will be larger.

29
Repeated Trials

• Let S be a finite probability space. By n
  independent or repeated trials we mean the
  probability space S consisting of all ordered
  n-tuples of elements of S, with the probability
  of n-tuple defined to be the product of the
  probabilities of its components.
• P(s1,s2,s3...sn)P(s1)?P(s2)? ? ?P(sn)

30
Repeated Trials

• Let probability space SP(a),p(b),P(c)
  represents probabilities three runners winning a
  race. Their probabilities of winning are
  P(a)1/2, P(b)1/3, P(c)1/6.
• If there are two races then the sample space S
  consisting of two repeated trials is
• Saa,ab,ac,ba,bb,bc,ca,cb,cc

31
Repeated Trials

• Saa,ab,ac,ba,bb,bc,ca,cb,cc
• The probability of the sample points of S are
• P(aa)(1/2)?(1/2)1/4
• P(ab)(1/2)?(1/3)1/6
• P(ac)(1/2)?(1/6)1/12
• P(ba)(1/3)?(1/2)1/6
• P(bb)(1/3)?(1/3)1/9
• P(bc)(1/3)?(1/6)1/18
• P(ca)(1/6)?(1/2)1/12
• P(cb)(1/6)?(1/3)1/18
• P(cc)(1/6)?(1/6)1/36
• The probability of c winning first race and a the
  second is P(ca)1/12
• EXCEL (1/4)(1/6)(1/12)(1/6)(1/9)(1/18)(1/12
  )(1/18)(1/36)

32
Bernoulli Trials with 2 possible outcomes.

• A Bernoulli trial is a random experiment in which
  there are only two possible outcomes - success
  and failure. If p is the probability of success,
  then q1-p is the probability of failure. Often
  we are interested in the number of successes
  without considering their order. The probability
  of exactly k successes in n repeated trials is
• b(k,n,p) pkqn-k

33
Example Trials with 2 possible outcomes.

• A coin is tossed 6 times, Hsuccess ,Tfailure.
• n6, pq1/2
• The probability of two heads, (k2)
• Binomial coefficient
• b(2,6,1/2) (1/2)2(1/2)4 15/64

34
Identically Distributed variableSame probability
distributions