Lecture 5: Chance Variability

1
Lecture 5 Chance Variability

• The Law of Averages
• The Expected Value and Standard Error
• The Probability Histogram
• The Normal Approximation for Probability
  Histograms
• The Central Limit Theorem

2
How many heads?

• Suppose you toss a fair coin 10,000 times.
  Roughly how many heads will you
  get?

3
Chance Fluctuations

• Around 5,000, but not exactly. The actual
  number will fluctuate, due to chance. The chance
  error is
• number of heads 5,000
• How large is the chance error as the number of
  tosses increases indefinitely?

4

• Kerrichs coing-tossing experiment.
  Chance error vs number of tosses.

5

• Same data. Percentage chance error vs number
  of tosses.

6
The Law of Averages

• As the number of tosses goes up,
• the chance error is likely to get larger, but
  relative to the number of tosses, it gets
  smaller.
• Equivalently, the percentage of heads is
  likely to become closer to 50, though not
  exactly.

7
Chance Processes

• Coin tossing
• Roulette
• Sampling survey
• Each experiment gives a different outcome,
  because of chance.

8
To what extent are the results influenced by
chance?

• (a) find an analogy between the process and
  drawing numbers at random from a box.
• (b) connect the variability of interest with
  the chance variability in the sum of the numbers
  drawn from the box.

9
The Sum of Draws

• A shorthand for this process
• (a) draw a number of tickets at random with
  replacement from a box of numbers.
• (b) add the numbers on the tickets.
• Simple, but will be used many times later.

10
Example 1

• Roll a die 25 times, and take the sum. This is
  like the sum of 25 random draws with replacement
  from a box with tickets labeled 1 to 6.
• Repeat this many times. You get a bunch of
  numbers (observed values) ranging from 25 to 150.

11

• 10 computer simulations gave 88 84 80 90 83
  78 95 94 80 89. All between 75 and 100.
• What is the chance that the sum is between 75 and
  100? This can be answered using the box model.

12
Making a Box Model

• What numbers go into the box?
• How many of each kind?
• How many draws?

13
Las Vegas Roulette

• A Las Vegas roulette wheel has 38 pockets 0, 00,
  1, 2,, 36. 18 are red, 18 are black. 0 and 00
  are green.
• You bet 1 on red. If the ball lands on a red
  pocket, you get a 1 otherwise you lose 1.
  Consider your net gain in 10 games. What is the
  box model?

14
Solution

• Box contains
• 18 tickets labelled 1
• 20 tickets labelled 1
• 10 draws are made
• (at random with replacement)

15
More Roulette

• You bet number 7. You win 35 if the ball
  lands on 7, otherwise you lose 1. What are the
  tickets in the box?
• 1 ticket labelled 35
• 37 tickets labelled 1

16
A little Order in Chaos

• A chance process is generating a long list of
  numbers. What can we say about it?
• The numbers vary around the expected value, and
  the size of the chance error is measured by the
  standard error.

17
Expected Value of Sum

• A box has 3 tickets labeled 1 and 1 ticket
  labeled 5.
• Make n draws at random with replacement.
• Every time you do this, you get a different sum.
  What is the EV?

18

• About 3/4 of the n draws should give 1, and
  about 1/4 should give 5.
• The EV for the sum of n draws is

19
EV of Sum Formula

• But 2 is the average of the box, i.e., the
  average of the tickets.
• For any box, the EV of the sum of n draws with
  replacement is
• n x average of box

20
Standard Error of Sum

• The sum of n draws hovers around the EV, and
  is likely to be off by a chance error about the
  size of the SE, given by
• The explanation for this square-root law is
  beyond this course.

21
SE of sum and number of draws

• It is intuitive that the larger the box SD,
  the more variable the sum. As the number of
  draws increases, the variability also goes up,
  but more slowly. For instance, the sum of 100
  draws is only 10 times as variable as a single
  draw.

22
Sum of 25 rolls of a die

• Roll a die 25 times. The sum of 25 rolls varies
  if we repeat the experiment.
• We expect the sum of 25 rolls to be around ___,
  give or take ___.
• The EV and the SE are wanted here.

23
Solution

• Box model tickets in box 1, 2, 3, 4,
  5, 6.
• Average of box 3.5, SD of box 1.7.
• EV 25 3.5 87.5
• SE 5 1.7 8.5
• So the sum of 25 rolls is around 87.5, give or
  take 8.5 or so.

24
Short-cut to compute SE

• If there are only two kinds of tickets in the
  box, the SD can be found more easily. Let the
  two numbers be a and b, where a respective fractions be f and g. Then the SD is

25
Betting 7 on roulette

• Box has 1 ticket labeled 35, 37 labeled ?1.
  Average of box ?0.05, SD of box 5.76.
• You make 100 bets. EV ?5, SE 58. The big
  SE means you have some reasonable chance to win
  big, but on average you lose, and lose big.

26
SD and SE

• SD measures the spread of a list of numbers.
• SE measures the variability of a chance process.

27
Box Model, SD and SE

• When drawing from a box, the SE of the sum is
  related the SD of the box by the square-root law.
  
• Otherwise, the relationship is more
  complicated (e.g., the SE of the product).

28
Example

• Box has tickets 0, 2, 3, 4, 6. What
  are the EV and SE of the sum of 25 draws at
  random with replacement?
• Ave of box 3, SD of box 2. So
• EV 25 3 75, SE 5 2 10.
• The sum of 25 draws is likely to be around 75,
  give or take 10.

29
(No Transcript)
30
Simulation vs Theory

• Every number in the previous table is an
  observed value for the sum of the draws. The 100
  observed values differ from the EV 75 due to
  chance. Though the observed values range from 0
  to 150, the values in the table shows remarkably
  little spread. In fact, all except one of them
  are within 2.5 SEs of the EV.

31
Rule of Thumb

• Observed values are rarely more than 2 or 3
  SEs away from the EV.

32
Chance and the Normal Curve

• A large number of draws are made at random
  with replacement from a box. What is the chance
  that the sum will lie in a given range?
• Convert to standard units using the EV and
  the SE and use the normal curve.

33
Box 0, 2, 3, 4, 6

• 25 draws are made at random with replacement.
  This process is repeated many times.
• About what percentage of the observed values
  should be between 50 and 100?
• Same question Whats the chance that the sum
  will be between 50 and 100?

34

• EV 75, SE 10.
• 50 is ?2.5 in standard units, and 100 is 2.5 in
  SU.
• The chance is roughly the area between ?2.5 and
  2.5 under the normal curve 99. In fact 99 of
  the 100 observed values in the table are between
  50 and 100.

35

• About 68 should be between 65 and 85 in fact 73
  are.
• About 95 should be between 55 and 95 98 are.
  The approximation is quite good.

36
Exercise

• A die is rolled 60 times.
• (a) The total number of spots should be
  around ____, give or take ____ or so.
• (b) The number of 6s should be around ____,
  give or take ____ or so.

37
Solution (a)

• Tickets are 1, 2, 3, 4, 5, 6.
  Total number of spots is like the sum of 60
  draws.
• Average of box 3.5, SD of box 1.71.
• EV 60 3.5 210, SE 60 1.71 13.

38
Solution (b)

• EV 60 1/6 10.
• For SE, need a box model.
• Previous box is irrelevant. Instead of adding,
  each roll is classified is it a 6 or not? If
  6, add 1. Otherwise, add 0.
• Tickets 0, 0, 0, 0, 0, 1. SD of box
  0.37. SE 60 0.37 3.

39
Adding or Classifying

• Two operations on draws (1) adding, (2)
  classifying or counting.
• EV and SE can be found by the same method, with
  the right box.
• For (2), the box should have only 0s or 1s.
  1 is for what you want 0 for what you dont
  want.

40
Coin Tosses

• Toss a coin 100 times. Estimate the chance of
  getting between 40 and 60 heads.
• We are not summing, but counting the box has
  only 0s and 1s.
• Since H and T are equally likely, it has one 0
  and one 1.

41
Coin Tosses

• Average of box 0.5, SD of box 0.5.
• EV 100 0.5 50, SE 10 0.5 5.
• Chance is roughly the area under the normal curve
  between 2 and 2 95.

42
The Square-root Law and the Law of Averages

• In n coin tosses, heads n/2 chance error.
• Law of averages the chance error grows as n
  increases, but shrinks relative to n.
• SE
• The square-root law is the mathematical
  explanation of the law of averages.

43
What is the Chance?

• Whats the chance that you get exactly 50 heads
  in 100 coin tosses? Exactly 51 heads? Between
  50 and 70 heads?
• The binomial formula can be used, but is very
  tedious.
• A good approximation is via the normal curve. To
  understand why this works, we need to look at
  probability histograms.

44
5 Coin Tosses

• In 5 coin tosses, there are 25 32 equally
  likely outcomes.
• number of heads number of
  patterns
• 0
  1
• 1
  5
• 2
  10
• 3
  10
• 4
  5
• 5
  1
• A histogram for number of heads is a
  probability histogram.

45
Probability Histograms

• A probability histogram of a chance process shows
  the chance of getting a particular outcome. The
  total area under a probability histogram is 100.
• The EV and SE summarise the probability
  histogram, just as the Average and SD summarise
  the data histogram.

46
Constructing Probability Histograms

• The probability histogram for the number of heads
  in 5 coin tosses is quite easy to make.
• For more complicated chance processes, a computer
  can be used.

47
A Pair of Dice

• The computer is programmed to roll a pair of dice
  and record the total number of spots. This is
  repeated many times.
• Make a histogram of the simulated data. As the
  number of repetition becomes large, the data
  histogram converges to the probability histogram.
  

48
(No Transcript)
49
Another Probability Histogram

• A similar method can be used to get the
  probability histogram for the product of the
  number of spots on a pair of dice.

50
(No Transcript)
51
Probability Histograms and the Normal Curve

• The probability histogram for the number of heads
  in 100 coin tosses is symmetric, and sort of fits
  the normal curve. The fit gets better with 400,
  and 900 tosses.
• The conclusion if the number of tosses is large
  enough, the probability histogram is close to the
  normal curve.

52
(No Transcript)
53
The Normal Approximation

• A coin will be tossed 100 times. Estimate the
  chance of getting
• (a) exactly 50 heads.
• (b) between 45 and 55 heads inclusive.
• (c) between 45 and 55 heads exclusive.

54
Solution

• EV 50, SE 5.
• (a) The chance is the area of the block above 50
  in the probability histogram. This block goes
  from 49.5 to 50.5. In SU, they become ?0.1 and
  0.1. The chance is approximately the area under
  the normal curve between ?0.1 and 0.1 7.97.
  Actual 7.96.

55

• (b) Chance area under the probability histogram
  between 44.5 and 55. This is approximately the
  area under the normal curve between ?1.1 and 1.1
  72.87. Actual 72.87.
• (c) The area between 45.5 and 54.5 is about that
  under the normal curve between ?0.9 and 0.9
  63.19. Actual 63.18.

56
Continuity Correction

• The continuity correction was used. For
  example, in finding the chance that the number of
  heads is between 45 and 55 inclusive, we look at
  the area under the probability histogram between
  44.5 and 55.5.

57
Continuity Correction

• If you are not told whether the endpoints are
  included, then a compromise is not to do a
  correction.
• For example, the chance that the number of heads
  is between 45 and 55 is approximated by the area
  under the probability histogram between 45 and
  55.

58
Drawing from a Box

• The number of heads in a coin-tossing process is
  like the sum of repeated draws from a box with
  tickets 0 and 1.
• How about other boxes? A lopsided box with 9
  0s and 1 1.

59
(No Transcript)
60
Lopsided Box

• With 25 draws, the probability histogram is a lot
  higher than the normal curve on the left, and
  lower on the right the normal approximation is
  not so good.
• With 100 draws, the approximation is much better.
  Even better with 400 draws.

61
Another Box

• A box with three tickets 1, 2 and 3. This
  box is symmetric, so even with 25 draws, the
  normal approximation is quite good.

62
(No Transcript)
63
Last Box

• A box with three tickets 1, 2 and 9. The
  histogram of the tickets does not look normal at
  all.
• But with enough draws, say 100, the approximation
  is excellent.

64
(No Transcript)
65
How about the product?

• Roll a die 10 times, and record the product of
  the 10 numbers.
• Does the probability histogram for the product
  converge to the normal curve, as we increase the
  number of rolls?

66
(No Transcript)
67
The Central Limit Theorem

• When the number of draws from a box is large, the
  probability histogram for the sum is well
  approximated by the normal curve.
• This mathematical fact holds for ANY box, and
  plays an important role in many statistical
  procedures. Hence the name.

68
How many draws?

• How many draws is enough for the approximation
  to be good? It depends on the box. However, for
  many boxes, the probability histogram for the sum
  of 100 draws will be close enough to the normal
  curve.

69
Probability Histogram, EV, SE

• When the probability histogram is close to the
  normal curve, it is completely determined by the
  EV and SE. The EV gives the centre, and the SE
  tells you the spread of the probability histogram.

70
Two Convergences

• The first convergence has to do with constructing
  a single probability histogram. Here, the number
  of draws is fixed, and the process is repeated
  many times. The data histogram converges to the
  probability histogram.
• This applies not just to sums, e.g., products.

71
Two Convergences (cont.)

• The second kind concerns the behaviour of
  probability histograms as the number of draws
  goes to infinity. Here the Central Limit Theorem
  says that the probability histogram converges to
  the normal curve.
• The CLT only applies to sums.

72
Two Convergences (cont.)

• In the first convergence, the chance process is
  fixed (e.g., number of draws), and the number
  repetitions goes to infinity.
• In the second convergence, the number of draws
  contributing to the sum goes to infinity.

73
Normal Approximation for Data

• When the number of repetitions (samples) is
  large, the first convergence ensures that the
  data histogram is close to the probability
  histogram.
• When the number of draws is large, the
  probability histogram is close to the normal
  curve.
• When both are large, the data histogram is well
  approximated by the normal curve. This can be
  proved mathematically.

74
Applying the Theory

• Is the data-generating process like taking the
  sum of numbers drawn from a box?
• If so, then when the number of draws are large,
  you can make very precise predictions.
• Establishing this analogy is beyond mathematics.
  It is a question of fact.