Determining the Size of a Sample

1
Determining the Size of a Sample
2
Sample Accuracy

• Sample accuracy refers to how close a random
  samples statistic is to the true populations
  value it represents
• Important points
• Sample size is not related to representativeness
• Sample size is related to accuracy

3
Sample Size and Accuracy

• Intuition Which is more accurate a large
  probability sample or a small probability sample?
  
• The larger a probability sample is, the more
  accurate it is (less sample error).

4
A Picture Says 1,000 Words

n 550 - 2000 1,450 4 - 2 2
Probability sample accuracy (error) can be
calculated with a simple formula, and expressed
as a number.
5
How to Interpret Sample Accuracy

• From a report
• The sample is accurate 7 at the 95 level of
  confidence
• From a news article
• The accuracy of this survey is 7

6
How to Interpret Sample Accuracy

• Interpretation
• Finding 60 are aware of our brand
• So between 53 (60-7) and 67 (607) of the
  entire population is aware of our brand

7
Sample Size Axioms

• To properly understand how to determine sample
  size, it helps to understand the following axioms

8
Sample Size Axioms

• The only perfectly accurate sample is a census.
• A probability sample will always have some
  inaccuracy (sample error).
• The larger a probability sample is, the more
  accurate it is (less sample error).
• Probability sample accuracy (error) can be
  calculated with a simple formula, and expressed
  as a - number.

9
Sample Size Axioms

• You can take any finding in the survey, replicate
  the survey with the same probability sample size,
  and you will be very likely to find the same
  finding within the - range of the original
  finding.
• In almost all cases, the accuracy (sample error)
  of a probability sample is independent of the
  size of the population.

10
Sample Size Axioms

• A probability sample can be a very tiny
  percentage of the population size and still be
  very accurate (have little sample error).

11
Sample Size and Population Size

• Where is N (size of the population) in the sample
  size determination formula?

In almost all cases, the accuracy (sample error)
of a probability sample is independent of the
size of the population.
12
Sample Size and Population Size

• Does the size of the population, N, affect sample
  size or sample error?

A probability sample size can be a very tiny
percentage of the population size and still be
very accurate (have little sample error).
13
Sample Size Axiom

• The size of the probability sample depends on the
  clients desired accuracy (acceptable sample
  error) balanced against the cost of data
  collection for that sample size.

14
Putting It All Together

• MR What level accuracy do you want?
• MM I dont have a clue.
• MR National opinion polls use 3.5.
• MM Sounds good to me.
• MR Okay, that means we need a sample of 1,200.
• MM Gee Whiz. That small?
• MR Yup, and at a cost of 20 per completion, it
  will be 24,000.
• MM Holy Cow! That much?
• MR I could do 500 for 10,000, and that would
  be 4.4 accurate, or 300 for 6,000 at 5.7.
• MM 500 sounds good to me.

The size of a probability sample depends on the
clients desired accuracy (acceptable sample
error) balanced against the cost of data
collection for that sample size.
15

• There is only one method of determining sample
  size that allows the researcher to PREDETERMINE
  the accuracy of the sample results

The Confidence Interval Method of Determining
Sample Size
16
The Confidence Interval Method of Determining
Sample Size

• This method is based upon the Confidence Interval
  and the Central Limit Theorem
• Confidence interval range whose endpoints define
  a certain percentage of the response to a question

17
The Confidence Interval Method of Determining
Sample Size

• Confidence interval approach applies the
  concepts of accuracy, variability, and confidence
  interval to create a correct sample size
• Two types of error
• Nonsampling error pertains to all sources of
  error other than sample selection method and
  sample size
• Sampling error involves sample selection and
  sample size

18
The Confidence Interval Method of Determining
Sample Size

• Sample error formula

19
The Confidence Interval Method of Determining
Sample Size

• The relationship between sample size and sample
  error

20
Computations Help Page
1.96
50 times 50
Lets try 3 ns 1000 500
100
Answers this way
21
And the answers are
1.96
50 times 50
Lets try 3 ns 1000 3.1 500 4.4
100 9.8
22
Review What does sample accuracy mean?

• 95 Accuracy
• Calculate your samples finding, p
• Calculate your samples accuracy, e
• You will be 95 confident that the population
  percentage (p) lies between p e

23
Review What does sample accuracy mean?

• Example
• Sample size of 1,000
• Finding 40 of respondents like our brand
• Sample accuracy is 3 (via our formula)
• So 37 - 43 like our brand

24
The Confidence Interval Method of Determining
Sample Size

• Variability refers to how similar or dissimilar
  responses are to a given question
• P percent
• Q 100-P
• Important point the more variability in the
  population being studied, the higher the sample
  size needed to achieve a stated level of accuracy.

25

• With nominal data (i.e. yes, no), we can
  conceptualize variability with bar chartsthe
  highest variability is 50/50

26
Confidence Interval Approach

• The confidence interval approach is based upon
  the normal curve distribution.
• We can use the normal distribution because of the
  CENTRAL LIMITS THEOREMregardless of the shape of
  the populations distribution, the distribution
  of samples (of n at least 30) drawn from that
  population will form a normal distribution.

27
Central Limits Theorem

• The central limits theorem allows us to use the
  logic of the normal curve distribution.
• Since 95 of samples drawn from a population will
  fall or 1.96 x sample error (this logic is
  based upon our understanding of the normal curve)
  we can make the following statement

28

• If we conducted our study over and over, 1,000
  times, we would expect our result to fall within
  a known range. Based upon this, we say that we
  are 95 confident that the true population range
  value falls within this range.

29
The Confidence Interval Method of Determining
Sample Size

• 1.96 x s.d. defines the endpoints of the
  distribution.

30

• We also know that, given the amount of
  variability in the population, the sample size
  will affect the size of the confidence interval.

31
So, what have we learned thus far?

• There is a relationship between
• The level of confidence we wish to have that our
  results would be repeated within some known range
  if we were to conduct the study again, and
• Variability in the population and
• The amount of acceptable sample error (desired
  accuracy) we wish to have and
• The size of the sample!

32
Sample Size Formula

• Fortunately, statisticians have given us a
  formula which is based upon these relationships.
• The formula requires that we
• Specify the amount of confidence we wish
• Estimate the variance in the population
• Specify the amount of desired accuracy we want.
• When we specify the above, the formula tells us
  what sample we need to usen

33
Sample Size Formula

• Standard sample size formula for estimating a
  percentage

34
Practical Considerations in Sample Size
Determination

• How to estimate variability (p times q) in the
  population
• Expect the worst cast (p50 q50)
• Estimate variability Previous studies? Conduct a
  pilot study?

35
Practical Considerations in Sample Size
Determination

• How to determine the amount of desired sample
  error
• Researchers should work with managers to make
  this decision. How much error is the manager
  willing to tolerate?
• Convention is or 5
• The more important the decision, the more
  (smaller number) the sample error.

36
Practical Considerations in Sample Size
Determination

• How to decide on the level of confidence desired
• Researchers should work with managers to make
  this decision. The more confidence, the larger
  the sample size.
• Convention is 95 (z1.96)
• The more important the decision, the more likely
  the manager will want more confidence. 99
  confidence, z2.58.

37
ExampleEstimating a Percentage in the Population

• What is the required sample size?
• Five years ago a survey showed that 42 of
  consumers were aware of the companys brand
  (Consumers were either aware or not aware)
• After an intense ad campaign, management wants to
  conduct another survey and they want to be 65
  confident that the survey estimate will be within
  5 of the true percentage of aware consumers
  in the population.
• What is n?

38
Estimating a Percentage What is n?

• Z1.96 (95 confidence)
• p42
• q100-p58
• e5
• What is n?

39
Estimating a Percentage What is n?
N374

• What does this mean?
• It means that if we use a sample size of 374,
  after the survey, we can say the following of the
  results (assume results show that 55 are aware)
• Our most likely estimate of the percentage of
  consumers that are aware of our brand name is
  55. In addition, we are 95 confident that the
  true percentage of aware customers in the
  population falls between 50 and 60.

40
Estimating a Mean

• Estimating a mean requires a different formula
  (See MRI 13.2, p. 378)

• Z is determined the same way (1.96 or 2.58)
• E is expressed in terms of the units we are
  estimating (i.e., if we are measuring attitudes
  on a 1-7 scale, we may want error to be no more
  than .5 scale units
• S is a little more difficult to estimate

41
Estimating s

• Since we are estimating a mean, we can assume
  that our data are either interval or ratio. When
  we have interval or ratio data, the standard
  deviation, s, may be used as a measure of
  variance.

42
Estimating s

• How to estimate s?
• Use standard deviation from a previous study on
  the target population.
• Conduct a pilot study of a few members of the
  target population and calculate s.
• Estimate the range the value you are estimating
  can take on (minimum and maximum value) and
  divide the range by 6.

43
Estimating s

• Why divide the range by 6?
• The range covers the entire distribution and 3
  (or 6) standard deviations cover 99.9 of the
  area under the normal curve. Since we are
  estimating one standard deviation, we divide the
  range by 6.

44
ExampleEstimating the Mean of a Population

• What is the required sample size?
• Management wants to know customers level of
  satisfaction with their service. They propose
  conducting a survey and asking for satisfaction
  on a scale from 1 to 10. (since there are 10
  possible answers, the range10).
• Management wants to be 99 confident in the
  results and they do not wan the allowed error to
  be more than .5 scale points.
• What is n?

45
Estimating a Mean What is n?

• S10/6 or 1.7
• Z2.58 (99 confidence)
• e.5 scale points
• What is n?

46
Estimating a Percentage What is n?
N77

• What does this mean?
• After the survey, management may make the
  following statement (assume satisfaction mean is
  7.3)
• Our most likely estimate of the level of
  consumer satisfaction is 7.3 on a 10-point scale.
  In addition, we are 99 confident that the true
  level of satisfaction in our consumer population
  falls between 6.8 and 7.8 on a 10-point scale

47
Other Methods of Sample Size Determination

• Arbitrary percentage of thumb sample size
• Arbitrary sample size approaches rely on
  erroneous rules of thumb.
• Arbitrary sample sizes are simple and easy to
  apply, but they are neither efficient nor
  economical.

48
Other Methods of Sample Size Determination

• Conventional sample size specification
• Conventional approach follows some convention or
  number believed somehow to be the right sample
  size.
• Using conventional sample size can result in a
  sample that may be too large or too small.
• Conventional sample sizes ignore the special
  circumstances of the survey at hand.

49
Other Methods of Sample Size Determination

• Statistical analysis requirements of sample size
  specification
• Sometimes the researchers desire to use
  particular statistical technique influences
  sample size

50
Other Methods of Sample Size Determination

• Cost basis of sample size specification
• All you can afford method
• Instead of the value of the information to be
  gained from the survey being primary
  consideration in the sample size, the sample size
  is determined by budget factors that usually
  ignore the value of the surveys results to
  management.

51
Special Sample Size Determination Situations

• Sampling from small populations
• Small population sample exceeds 5 of total
  population size
• Finite multiplier adjustment factor for sample
  size formula
• Appropriate use of the finite multiplier formula
  will reduce a calculated sample size and save
  money when performing research on small
  populations.

52
Special Sample Size Determination Situations

• Sample size using nonprobability sampling
• When using nonprobability sampling, sample size
  is unrelated to accuracy, so cost-benefit
  considerations must be used.

53
Practice Examples

• We will do some examples from the questions and
  exercises at the end of the chapter on sample
  sizequestion 5 on page 386.

54
Practice Examples

• 5a. Using the formula provided in your text,
  determine the approximate sample sizes for each
  of the following cases, all with precision
  (allowable error) of 5
• Variability of 30, confidence level
  of 95

55
Practice Examples

• 5b. Using the formula provided in your text,
  determine the approximate sample sizes for each
  of the following cases, all with precision
  (allowable error) of 5
• Variability of 60, confidence level
  of 99

56
Practice Examples

• 5c. Using the formula provided in your text,
  determine the approximate sample sizes for each
  of the following cases, all with precision
  (allowable error) of 5
• Unknown variability, confidence level
  of 95