Samples, Good and Bad

1
Samples, Good and Bad

• Chapter 2

2
Review Where Do Data Come From?

• A statistical study records information about
  individuals (people, animals, households, things
  ) by giving the value for one or more variables
  for each individual
• Some variables are numeric, some are not
• Some variables are useful, others not at all
• The most important fact about any statistical
  study is how the data were produced

3
Review Where Do Data Come From?

• A census attempts to measure every individual in
  a population i.e. a complete sample survey
• An experiment actually does something to the
  individuals in order to observe a response
• The usual aim is to see if a treatment actually
  causes a specific response on average

4
Review Where Do Data Come From?

• Observational studies try to gather information
  without disturbing the scene they are observing
• Sample surveys are a very important kind of
  observational study
• A sample survey chooses a sample from a specific
  population and uses the sample to get information
  about the entire population

5

• Questions from Last time ??

6
The Town Talk takes and opinion poll

• A local paper wants assess the communitys
  reaction to breaking the local ambulance
  companys monopoly they have a call-in survey
• 3,763 calls
• 638 from existing ambulance company numbers
• Company VP admits Weve got employees who are
  concerned about this situation
• What likely happened here?
• What kinds of people called in to respond to the
  survey?

7
How to sample badly

• The Town Talk relied on voluntary response to
  define their sample
• Only people who really care phoned in (maybe more
  than once!), and most of those were people who
  would lose their job if the monopoly was broken
  people who are upset and will strongly favor
  keeping the monopoly
• This is a biased sample because it is
  overweighted with people who favor the monopoly
• There are other ways to sample badly

8
Example 1 Interviewing at the mall

• Manufacturers and advertisers often use
  interviews at shopping malls to inform them of
  consumers opinions
• This is a sample of convenience fast, easy,
  cheap just go the mall and interview people
• Who comes to the mall?
• People with money
• People with free time teenagers, the elderly
• Who do interviewers select?
• Neat, friendly, safe-looking people
• Who gets left out?
• Poor people, middle-aged people who are at work
  and scary or awkward-looking people
• Who does this sample represent?

9
Example 2 Write-in opinion polls

• Ann Landers If you had it to do over again,
  would you have children?
• 10,000 responses!
• 70 said NO!
• ??? Whats going on, is it really true that 70
  of parents dont like their kids?
• This is a voluntary response poll
• The bulk of respondents are parents who have had
  difficulties with their kids and want to tell
  someone about it
• This is a strongly biased poll that does not
  represent all parents, but rather just those who
  were upset and cared enough to write in

10
(No Transcript)
11
Biased Sampling Methods

• Write-in/Call-in opinion polls almost always very
  biased
• Only 15 of population has ever responded to one

12
Simple random samples

• In voluntary response polls write-in, call-in
  people choose whether to respond
• In a convenience sample i.e. at the mall the
  interviewer chooses who to interview
• In both cases personal choice produces bias
• The solution is to let random chance choose the
  sample
• Allows neither favoritism on the part of the
  interviewer nor self-selection on the part of the
  respondent
• Gives everyone rich, poor, black, white, male,
  female, tall, short , an equal chance of
  getting into the sample

13
Simple Random Sample

• The simplest way to choose a random sample is to
  put names (the population) into a hat, shake it
  (randomization) and then pull out a handful (the
  sample)
• The is a Simple Random Sample (SRS)

14
Simple Random Sample

• An SRS gives every individual an equal chance to
  be chosen thus eliminating bias
• An SRS also gives each sample an equal chance of
  being chosen
• Every sample of ten names from the hat has the
  same chance of being drawn as every other sample
  of ten names
• The idea of an SRS is that both the individuals
  and the samples have the same chance of being
  drawn
• Obviously drawing names from a hat is not a
  practical way of drawing an SRS from a large
  population
• We need to start with a randomizer

15
Random Digits

• Table A at the back of the book contains a list
  of random digits

16
Table A Random digits
17
Example 3 how to choose a SRS (1)

• Joan wants to draw a SRS of size 5 from the 30
  clients of her small firm
• Step 1, Labeling. Give each client a numerical
  ID, using as few digits as possible
• 01 A-1 Plumbing
• 02 Accent Printing
• 03 Action Sport Shop
• 20 MagicTan
• 30 Vons Video Store

18
Example 3 how to choose a SRS (2)

• Step 2, Randomizing. Enter Table A (or any other
  list of random digits) anywhere and read off
  two-digit groups
• For example, starting at line 103 in Table A
• 45467 71709 77558 00095 32863 .. .. becomes
• 45 46 77 17 09 77 55 80 00 95 32 86 ..
  .. .. ..
• Every two-digit group from Table A is as likely
  as each of the 100 possible two-digit groups
• Joan uses labels 01-30, so we can ignore all
  other two digit groups
• Read along the list and take the first five
  two-digit groups that fall within 01-30, this is
  Joans SRS of size 5

19
Example 3 how to choose a SRS (3)

• The first two in Joans SRS are IDs 17 and 09,
  Johnsons Commodities and Blue Print Specialties
• Choosing and SRS has these two steps

20
Choosing a SRS

• You can assign labels (IDs) in any convenient
  order, such as alphabetical by name
• As long as each label has the same number of
  digits, each individual will have the same chance
  of being chosen
• Use the shortest labels possible
• If you have ten less than 10 individuals in the
  population, use 1, 2, 3,
• If you have 10-20, use 01, 02, , 10, 11,
• Begin with 01, or 001, etc.

21
Example 4 Choosing a SRS using software

• In practice SRSes are drawn using software
• There is no real difference between the random
  digit table and the software approach
• Software is just easier, especially when the
  population and sample are large
• You can find a nice randomizer at
  www.randomizer.org
• We will use the Randomizer to pick a few more
  SRSes of various size from Joans list of clients

22
(No Transcript)
23
Can you trust a sample?

• The Town Talk, Ann Landers and mall interviews
  produce samples
• We cannot trust results from these samples
  because they are chosen in ways that invite bias
• We have more confidence in results from a SRS
  because it uses impersonal chance to avoid bias
• When you encounter a sample, ask Is this sample
  chosen at random?
• All properly conducted surveys use some form of
  random sampling

24
Example 5 Gallup Poll

• In your view does the entertainment industry
  need to make a serious effort to significantly
  reduce the amount of sex and violence in its
  movies?
• Results based on telephone interviews with a
  randomly selected national sample of 1,008 adults
  18 and older conducted Feb 6-8, 2004
• Result 75 of Americans think the entertainment
  industry should make a serious effort
• Can we trust this result?

25
Summary (1)

• We select a sample to get information about a
  population
• We want a sample that fairly represents the
  population
• Convenience samples and voluntary response
  samples are common but do not produce
  trustworthy, representative results because they
  are usually biased
• Bias is the systematic favoring of one part of
  the population (and their opinions) over other
  parts of the population

26
Summary (2)

• Using chance to choose a sample is one of the
  fundamental ideas of statistics
• Random samples use chance to choose the sample
• In the Simple Random Sample
• Every individual in the population has the same
  chance of being in the sample
• Every sample of the same size has the same chance
  of being chosen
• To choose a SRS
• Use a table of random digits, or
• Use software that produces random digits

27
Exercise 2.6

• A call-in opinion poll. Should the United
  Nations continue to have its headquarters in the
  United States? A television program asked its
  viewers to call in with their opinions on that
  questions. There were 186,000 callers, 65 of
  whom said No. A nationwide random sample of
  500 adults found that 72 answered Yes to the
  same question.
• Explain to someone who knows no statistics why
  the opinions of only 500 randomly chosen
  respondents are a better guide to what all
  Americans think than the opinions of 186,000
  callers.

28
Solution 2.6

• Call-in polls, and voluntary response polls in
  general, tend to attract responses from those who
  have strong opinions on the subject, and
  therefore are often not representative of the
  population as a whole.
• On the other hand, there is no reason to believe
  that the 500 randomly chosen adults over
  represent any particular group, so the 72 yes
  from that poll is more reliable as an estimate of
  the true population proportion.

29
Exercise 2.10

• Is this an SRS? A university has 1,000 male and
  500 female graduate students. A survey of
  graduate student opinion concerning health care
  benefits for graduate students selects 100 of the
  1,000 men at random and then separately selects
  50 of the 500 women at random. The 150 graduate
  students chosen make up the sample.
• (a) Explain why this sampling method gives each
  graduate student an equal chance to be chosen.
• (b) Nonetheless, this is not an SRS. Why not?

30
Solution 2.10

• (a) For men, 100/1000 or 10 are chosen, and for
  women, 50/500 or 10 are chosen. So any graduate
  student, male or female, has 1 chance in 10 of
  being selected.
• (b) Each sample contains exactly 100 men and 50
  women. This is not an SRS because not all samples
  of size 150 are even possible, let alone equally
  likely e.g., one could not come up with a
  sample containing 101 men and 49 women.