Methods, Practice and Teaching of Survey Research

1
Methods, Practice and Teaching of Survey Research

• Don Haider-MarkelUniversity of Kansas
• May-June 2005

2
The Basics Why Conduct a Survey?
3
Sampling and Sample Design

• Types of Samples and Probability Theory
• Random Samples
• Reflecting a population
• Expense
• Non-Random Samples
• Stratified Sample
• Examples
• Quota Samples and Others

4
Implementing a Sample Design

• Issues of Cost
• Issues of response rates
• Biased sample reliability
• Reducing non-response rates
• Traditionally telephone was best (gt50)
• Increasingly mail response is the same as
  telephone sometimes better (25-35)
• Deciding on a data collection method
• Cost, response rate, population characteristics

5
Designing the Survey Instrument

• Defining Objectives
• What do you really want to know
• Several Questions Related to the issue
  triangulate
• Example Attitudes about the U.S. and the West

6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
Designing the Survey Instrument

• Question construction
• Reliability Wording
• Validity Are we measuring what we think we are
  measuring?
• Concept versus Measurement

11
Use of Terms and Question Wording
2005 May 2-5(sorted by "should") Homosexuals Gays and Lesbians Difference, in Pct. Pts.
   
Salesperson 90 93 3
Doctors 78 81 3
The armed forces 76 79 3
As a member of the president's cabinet 75 78 3
High school teachers 62 71 9
Elementary school teachers 54 64 10
Clergy 49 53 4
12
Designing the Survey Instrument

• Format and Layout
• Order Effects
• Appearance on Surveys that the Respondent is Able
  to view
• Example of scenarios

13
(No Transcript)
14
(No Transcript)
15

• Instrument Length
• Ideal is 10 to 15 minutes on telephone or mail
  survey
• Often not possible
• At about 30 minutes significant drop-off
• Translates into less than 75 non-complex questions

16
Non-Complex Question
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
Going Into the Field

• Ethical Issues and Consent
• Human Subjects Approval
• Funding issues
• Who is paying and notifying respondents
• Training interviewers
• Consistency
• Professional

21
Using Survey Data

• Returning to the Research Questions
• Recall what you wanted to learn

22
Using Survey Data
23
Using Survey Data

• Uncovering New Questions
• Data patterns are likely to reveal new issues

24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
Using Survey Data

• Bivariate versus Multivariate Analysis
• Bivariate allows simple way to show relationships
• Multivariate allows us to control for alternative
  explanations

29
Bivariate

• Republicans, independents, and Democrats have
  different ideas on the origins of homosexuality
  -- Democrats are more likely to believe it is
  something a person is born with Republicans
  believe it is due to upbringing and environment.
  
• In your view, is homosexuality something a
  person is born with, (or is homosexuality) due to
  factors such as upbringing and environment? 3
  pct. pt. margin of errorMay 2-5, 2005Sample
  size 1,005National adults

30
Multivariate

• Table 1. The Determinants of Causal Attributions
  about the Origins of Homosexuality Genetics as
  Cause.
• Independent Variables Estimate z
• Education .33 5.30
• (.06)
• Age .19 4.18
• (.04)
• Female .58 4.04
• (.14)
• Gay Friend .78 5.10
• (.15)
• Religiosity -.25 - 5.28
• (.04)
• Republican - .55 -
  3.40
• (.16)
• Liberal .36
  2.01
• (.18)
• Constant - 2.17
  - 6.88
• (.31)
• Pseduo R-square .13

31

• Explaining Margin of Error and Sampling Issues
• Need to provide clear methodology
• Exact Question Wording

32
(No Transcript)
33
Final Thoughts

• Just one research tool
• No better or worse than others
• Use depends on research questions
• Should always try to combine methods and analysis