Outline

1
Outline

• Definition
• Three major characteristics
• When and why to use surveys
• Two distinct forms of survey research
• Advantages of survey research
• How to create a survey

2
Surveys - Definition

• A survey is a procedure for systematically
  collecting information about attitudes,
  preferences, knowledge, or behavior by asking
  people questions.

3
Three major characteristics

• Survey research has 3 major characteristics

• Procedure
• Analysis
• Interpretation

4
Three major characteristics

• Procedure

• Survey research involves asking people questions
  using a formal procedure

5
Three major characteristics

• Analysis

• Survey research uses a quantitative method
  requiring standardized information as input
• The formal procedure produces standardized
  information

6
Three major characteristics

• Interpretation

• Survey research generalizes results from a
  sample to the population it was drawn from

7
When to Use Surveys

• When you want information about a population, but
  you cannot measure the whole population.

• Measuring the whole population is usually
  impossible
• Doing so is also usually unnecessary

8
2 distinct forms of survey research

• Exploratory

• Used in new research areas where little theory
  has developed
• descriptive

9
2 distinct forms of survey research

• Explanatory

• Looking for evidence of cause-effect relations
  among variables
• If C causes D, then C and D should be
  correlated.

10
Advantages of survey approach

• Surveys are
• Cheap

• Relative to measuring whole population

11
Advantages of survey approach

• Surveys are
• Practical

• Can be done in a short time
• Can be done over phone or through mail or
  internet

12
Advantages of survey approach

• Surveys are
• Accurate

• Results true of population within a small margin
  of error

13
Margin of Error

• A range of values
• True population value is likely to be in this
  range.
• If range is large, survey results are not useful

• Size of range depends upon sample size and
  confidence level chosen for estimating population
  value.

14
Value obtained from sample
15
How to create a survey

• Research questions
• Design
• Survey questions
• Sampling procedure
• Administration procedure
• Analyzing the results

16
Research question

• Start with a behavioral theory.
• Use it to generate research questions.

• Specify type of information that will (or wont)
  meet your scientific objectives

17
Research question

• Be as specific as possible
• Think about interpretation

• In reports, you wont discuss the actual
  questions on your survey instrument youll
  discuss some theoretical construct
• What is it?

18
Design

• Longitudinal

• Same samples measured at different times
• Potential problem with loss of subjects
• Do differences across time reflect change in
  society or change specific to subjects?

19
Design

• Cross-sectional

• One or more samples measured at one time
• If groups differ, its difficult to say why
• E.g., compare 20 year olds to 50 year olds do
  results differ because of age or because of
  cohort effect?

20
Design

• Successive independent samples

• Different samples at different times
• Samples supposed to be drawn from same population
  but that may be arguable if interval is long
• This design cannot measure change in individuals

21
Survey questions

• If a survey exists that does the job, use it
• Otherwise

• Write a first draft

22
Survey questions

• If a survey exists that does the job, use it
• Otherwise

• Write a first draft
• Get feedback and revise draft

23
Survey questions

• If a survey exists that does the job, use it
• Otherwise

• Write a first draft
• Get feedback and revise draft
• Pretest your questionnaire

24
Survey questions

• If a survey exists that does the job, use it
• Otherwise

• Write a first draft
• Get feedback and revise draft
• Pretest your questionnaire
• Edit questions

25
Survey questions

• If a survey exists that does the job, use it
• Otherwise

• Write a first draft
• Get feedback and revise draft
• Pretest your questionnaire
• Edit questions
• Formalize procedures

26
Survey questions

• Avoid bias

• Do you believe in killing unborn babies?
• Should women be forced to bear unwanted
  children?

27
Question from an ABC poll

• Schiavo suffered brain damage and has been on
  life support for 15 years. Doctors say she has no
  consciousness and her condition is irreversible.
  Her husband and her parents disagree about
  whether she would have wanted to be kept alive.
  Florida courts have sided with the husband and
  her feeding tube was removed on Friday. What's
  your opinion on this case - do you support or
  oppose the decision to remove Schiavo's feeding
  tube?

28
A Gallup Poll question the same week

• As you may know, on Friday the feeding tube
  keeping Terri Schiavo alive was removed. Based on
  what you have heard or read about the case, do
  you think that the feeding tube should or should
  not have been removed? 

29
Fox News poll 18 days earlier

• Terri Schiavo has been in a so-called 'persistent
  vegetative state' since 1990. Terri's husband
  says his wife would rather die than be kept alive
  artificially and wants her feeding tube removed.
  Terri's parents believe she could still recover
  and want the feeding tube to remain.  If you were
  Terri's guardian, what would you do? Would you
  remove the feeding tube or would you keep the
  feeding tube inserted?

30
Issues

• Does requiring tube feeding mean the same as
  being on life-support?

• What does the public understand from the term
  life-support? Does the term bias the result?

31
Issues

• Were results influenced by fact ABC poll was
  conducted in one evening?

• "Surveys conducted on one evening, or even over
  two days, have more sampling biases -- due to
  non-response and non-availability -- than surveys
  which are in the field for three, four or five
  days.
• National Council on Public Polls (NCPP)

32
Issues

• Were results influenced by whether the person
  involved was referred to as Schiavo or Terri?

33
Survey questions

• Be careful about social desirability

• Think about question sequence
• Use filter questions

34
Survey questions

• Dont write two questions as one
• What would No mean in answer to this question

• Do you support the Administrations decisions to
  increase class sizes and start classes earlier in
  the day?

35
Survey questions

• Do the people youre surveying have the
  information required to answer your questions?

• Check to see before you ask your questions

36
Survey questions

• Are you familiar with the Liberal Partys
  platform for this election?
• before

• In your view, will the Liberal Partys platform
  produce positive or negative effects on the
  economy?

37
Survey questions

• Is the meaning of your question clear?

• If you refer to young people do you mean 9 year
  olds? 25 year olds?

38
Survey questions

• Is the meaning of your question clear?

• Which newspaper do you read?
• Does this mean everyday? Sometimes?
• What if the respondent reads more than one?

39
Survey questions

• Is the meaning of your question clear?

• Be clear about the time frame of the behavior you
  are asking about
• E.g., CFQ asks about frequency of mental slips
  over the last six months

40
Sampling procedure

• Major task to select a representative sample

• Representative sample shares distribution of
  relevant characteristics with population

41
Sampling procedure

• Some technical terms you need to know

• Population
• Sampling frame
• Sample
• Element

42
Sampling frame
Sample
Population
Element
43
Types of sampling procedure

• Non-probability sampling

• Elements vary in probability of being chosen
• Those probabilities are not known
• Accidental samples
• Purposive samples

44
Types of sampling procedure

• Non-probability sampling
• Probability sampling

• Researcher knows the probability of inclusion for
  each element in population.
• Simple random sampling
• Stratified random sampling

45
Administration procedures

• Phone

• Cheap fast
• But who is home?
• Verbal instructions and response alternatives may
  be hard to remember

46
Administration procedures

• Phone
• Face-to-face

• Expensive
• But maximal control
• Interviewer can see whether respondent
  understands question
• Respondent may work harder

47
Administration procedures

• Phone
• Face-to-face
• Mail

• Inexpensive
• Significant problem with response bias

48
Analyze the Results

• Select a data analysis procedure before you
  collect data.

• Correlations
• Path analysis
• Factor analysis