Introduction to Sample Surveys

1
Introduction to Sample Surveys
2
Sample Surveys

• Sample survey has widespread applications,
  ranging from surveys conducted for business
  purposes to collection of data for use in public
  affairs and social studies.
• In order to ensure the reliability of survey
  results, it is essential that great care in
  addition to adequate professional knowledge be
  applied in the planning and conduct of surveys
  and in the analysis of data.

3
Major Steps in Conductinga Sample Survey

• Overall planning
• Design and selection of sample
• Design of questionnaire
• Collection of raw data
• Compilation and analysis of statistics and
  dissemination of survey results

4
Overall Planning

• Clearly defined survey objectives
• Clearly defined target population
• Clearly defined statistical standards (definition
  of terms, counting rules, classification,
  statistical methodologies)
• Respondents should be able to provide the
  information required.
• Adequate resources such as manpower and time must
  be available.

5
Design and Selection of Sample

• Sampling deals with the selection of a number of
  elements, i.e. a sample.
• A sample can be drawn using different methods. A
  probability sample, with each element having a
  known, non-zero chance of being included, should
  be used as far as practicable so that scientific
  inference can be drawn from the survey results.

6
Design and Selection of Sample

• Non-probability sampling methods, such as
  haphazard sampling, should be avoided. Such
  methods are not scientific and bias usually
  exists in survey results.
• A complete and up-to-date sampling frame should
  be acquired.

7
Design and Selection of Sample

• In case of unequal probability of selection, it
  is necessary to ensure that proper weighting
  methods are applied to survey results.
• Using established statistical methodology to
  compute the required sample size.
• Once the sampling units are selected, alterations
  are not allowed.

8
Common ProbabilitySampling Methods

• Simple random sampling
• Systematic sampling
• Stratified sampling
• Clustering sampling
• Multi-stage sampling

9
Non-probability Sampling Methods

• Selection of elements is based on subjective
  judgment and experience, but not a random manner
• Common non-probability sampling methods
• Street interview/mall intercept
• Quota sampling
• Respondent-initiated telephone polling
• Internet survey

10
Non-Probability Sampling Methods

• Why unreliable?
• Scientific inference cannot be drawn from the
  sample data (results are confined to describing
  the group of respondents, but cannot be extracted
  to the entire population)
• Level of precision of the estimates cannot be
  scientifically assessed
• Biases will most likely exist in the survey
  results

11
Non-Probability Sampling Methods

• Why unreliable (contd)?
• No way to know how exactly the group of
  respondents is formed
• In street interviews, the interviewers tend to
  select those friendly faces for interview.
• In self-selecting polls, only those who have
  strong views on the survey may volunteer to
  participate.
• The results may be inaccurate and misleading,
  hence not of much use.

12
General Principles in Designing Questionnaires

• Questions should be relevant to the survey
  objectives
• Questions should be arranged in a proper order
• Use screening question to enhance the flow of the
  questions.
• Use an appropriate language
• Clear instructions

13
General Principles in Designing Questionnaires

• Questions wordings should be appropriate,
  specific and precise
• Avoid leading questions or questions being loaded
  in favour of a particular response
• Avoid difficult vocabulary
• Avoid composite and double negative questions
• Beware of memory error

14
General Principles in Designing Questionnaires

• Dont know/No opinion should be included as
  appropriate
• Long questionnaires are undesirable.
• Questionnaires should be tested on some
  prospective respondents before finalized.

15
Collection of Raw Data

• Methods of data collection for surveys include
• Self-administered questionnaires by mail
• Personal interviews
• Telephone interviews
• Computer Assisted Telephone Interviewing (CATI)
• Very often, mixed modes of data collection can be
  used.

16
Collection of Raw Data

• An appropriate mode should be selected by
  carefully considering respondents' willingness to
  co-operate, the degree of complexity of the
  subject of enquiry and other relevant factors
  (e.g. practicability of using personal or
  telephone interviews).

17
Comparison of Different Data Collection Methods
Characteristics Personal interview Telephone interview Mail questionnaire
Cost High Medium Low
Manpower High Medium Low
Time consuming High Medium Low
Non-response/non-contact rate Relatively lower Medium to high High
Suitability for scattered population Costly Less costly Medium
Interviewer bias Yes Yes No
Response quality High Medium Low
Length of questionnaire May be longer Preferably short Preferably short
Design of questionnaire May be more complicated Simple Simple
Asking embarrassing questions Not suitable Easier Easier
18
Collection of Raw Data

• Interviewers should be trained before they start
  working and closely supervised during fieldwork
  to ensure their quality of work.
• Identity and information supplied by individual
  respondents should be kept confidential. The
  survey results are to be presented in the form of
  aggregate statistics.

19
Collection of Raw Data

• Every effort should be made to achieve a high
  response rate (or reducing the number of
  non-responses).
• Methods to reduce non-responses
• Keep the questionnaire brief and concise
• Consider rewards to respondents
• Better publicity measures (e.g. advertisement,
  advance letters)
• Assurance of confidentiality of individual data

20
Collection of Raw Data

• Methods to reduce non-responses (Contd)
• More experienced interviewers, especially when
  handling refusals
• Visit households in evening time
• Visit at different time of different days
• Increase the number of re-visits/call backs
• Use self-administered questionnaires in case of
  non-contact

21
Compilation and Analysis of Statistics

• Data should be carefully and thoroughly checked
  before compilation .
• Appropriate statistical methodology should be
  adopted in compiling and analysing data.

22
Reporting and Assessingthe Reliability of Surveys

• To enable readers to make judgment on whether the
  findings are credible, a good survey report
  should include
• Sponsorship of the survey
• Population covered
• Sampling method
• Mode of data collection
• Time period of data collection
• Wording of questions

23
Reporting and Assessingthe Reliability of Surveys

• A good survey report should include (contd)
• Sample size and response rate
• Point estimates and confidence intervals (if
  possible)
• Likely sources of non-sampling errors
• Information supplied by individual respondents
  should not be disclosed.

24
Reporting and Assessingthe Reliability of Surveys

• One may access the reliability of a survey by
  asking the following questions
• Has probability sampling methods been used ?
• Is the sample size reasonably large ?
• Is the questionnaire design proper ? Any leading
  questions or wordings ?

25
Reporting and Assessingthe Reliability of Surveys

• One may access the reliability of a survey by
  asking the following questions (contd)
• What is the interviewing method ? Any improper
  influence by interviewers or third parties during
  the interview ?
• Is the response rate too low ?
• Are the sampling errors of acceptable magnitude ?
  How about non-sampling errors and biases ?