Surveys

1
Surveys

• The most widely used data-gathering device in
  sociology

2
Surveys

• Survey research rests on sampling.
• A sample is simply the collection of people from
  whom you are gathering data.
• There are tons of ways to sample, but we want to
  be sure that we sample with generalizabilty in
  mind.
• Sampling will be discussed in detail later.

3
Surveys

• What type of sampling does the General Social
  Survey use???
• In 2000, it was a multistage cluster sample. It
  is a full probability sample of US households,
  meaning that each house had an equal chance of
  being selected.
• Persons in households were selected
  systematically to make demographic
  characteristics of the sample equivalent to the
  characteristics of the U.S. population.

4
Surveys

• What are the implications of the General Social
  Surveys sampling???
• The GSS is an adults-only survey of persons in
  households. Therefore, it underrepresents
• 18 24 year-olds (many not living in
  householdsmilitary, college, roaming)
• 65 and over (many not living in
  householdsvacations, RVs, assisted living)
• Persons who live in large households (only one
  person per household is interviewed)
• Homeless and some poor (not in official
  households, in shelters, on streets)

5
Surveys

• Sampling is a technique for selecting subjects.
  So what is a survey?
• A process of presenting a standard series of
  questions to a sample of persons.
• The purpose is to accurately reflect the beliefs,
  attitudes, and behaviors of the sample, so that
  one can generalize accurate information to some
  larger population.
• The survey is the most widely used technique in
  sociology because it is best suited for looking
  at the complex social world. To capture that
  world accurately, we have to measure it in
  situ. That means we must sample from it, and
  measure many different phenomena, where they are
  found naturally.

6
Surveys

• Types of Surveys
• The survey can be administered as a(n)
• Written document filled out by the samples
  respondents
• alone
• in groups
• or individually in group settings (group
  administered)
• Face-to-face interview, with questions read by an
  interviewer
• Telephone interview
• Electronic Questionnaire

7
Surveys

• Survey Delivery
• Written Questionnaires
• Mail
• Group Administered Questionnaires
• Drop-off Surveys
• Electronic Questionnaires
• Distributed as E-mail
• Web pagessampling can be done via normal
  methods, but often arent
• Public computers in high-traffic areas
• In the future
• Files sent via mail, replacing written surveys
• Mixed formats offered to subjects (choose
  pen-and-paper, web page, file on disk)

8
Surveys

• How is the General Social Survey Delivered and
  Administered???
• The GSS is a one-on-one interview. An
  interviewer asks subjects questions.
• In 2000, the interviewer used paper and pencil.
• In 2002 they switched to computer assisted
  personal interviewing (CAPI). A computer program
  provided questions and interviewers punched in
  answers, much like in telephone polling with CATI.

9
Surveys

• Before Choosing a type of survey, you should
  consider four (4) key issues
• Respondent Attitude Think like a subject. How
  will they react to the way they are approached?
• Nature of Questions Open-ended and scale
  questions would be difficult over the phone. You
  have to consider whether respondents can
  adequately answer your questions via your medium.

10
Surveys

• Before Choosing a type of survey, you should
  consider four (4) key issues
• Cost What can you afford? Target population,
  geographic distribution, and type of
  questionnaire all affect costs.
• Generalization needs If you need a broad
  sample, you cant do drop-off surveys. If you
  want to sample from the elderly, you should avoid
  electronic surveys as currently conceived.

11
Surveys

• So you have to think about the strengths and
  weaknesses of your survey delivery method.
• In a minute youll break into five groups to
  discuss strengths and weaknesses of
• Mailed
• Group Administered
• Face-to-Face Interviews
• Telephone
• Electronic
• Ill give you an example using the dropoff

12
Surveys

• Drop Off
• Strengths versus Weaknesses

13
SurveysDrop Off

• Weaknesses
• Time. Because of the personal contact, this uses
  more researcher time.
• Challenges to generalizability
• Hard to get a broad sample. Target population
  must be pretty specific because of the limited
  ability to make contact with different people.
• Lack of control over distribution.
• Lower response rates than face-to-face and
  telephone (possibly)
• InaccuracyBehavioral problems in groups,
  playful responses.

• Strengths
• Convenient for respondents, like mail survey.
• Potentially higher response rates than mailed
  survey because the researcher has a presence (if
  dropped off at residence) or organizational
  endorsement (if dropped off at an organization).

14
Surveys

• So you have to think about the strengths and
  weaknesses of your survey delivery method.
• Break into Five (5) groups.
• Mailed
• Group Administered
• Face-to-Face Interviews
• Telephone
• Electronic
• Spend twenty (20) minutes generating strengths
  and weaknesses of your groups survey delivery
  type.

15
Surveys

• You must weigh the strengths and weaknesses of
  the survey delivery methods to choose the one
  that is appropriate and feasible.
• The following slides consider the strengths and
  weaknesses of five delivery methods (the lists
  are not exhaustive).

16
Surveys

• Mailed
• Strengths versus Weaknesses

17
SurveysMailed

• Strengths
• Cost
• The greatest cost in most businesses is wages.
  Mailed surveys reduce labor compared with
  face-to-face or phone. However, human data entry
  is expensiveuse scan forms.
• Low cost for respondent when postage paid
  envelope is provided
• Convenience for participants
• Reduced Biasno interviewer
• Facilitates generalizabilityEasy to reach more
  people over large area
• Better Geographic (and demographic) targeting

• Weaknesses
• Motivation,
• People may not be motivated to respond or
  respond thoroughly when researcher is not present
• Lower response rates, compared with face-to-face
  or group administered
• Challenges to accurate response
• Assumes physical skills
• Literacy
• language abilities
• This likely biases results.
• Interpretations of Questions can be
  inaccuratebias from no interviewer to assist.
• Who fills it out?

18
Surveys

• Group Administered
• Strengths versus Weaknesses

19
SurveysGroup Administered

• Strengths
• Higher response rate, especially if a captive
  audience such as school.
• Permits complex questionsGreater specificity and
  variation in question types because theres
  someone to answer questions and explain.
• Good for educationally challenged and children
• Someone is there to help.
• Fast way to get sample.
• Inexpensive, especially if you dont hire many
  people. You get many cases at once. Data entry
  is still an issue.

• Weaknesses
• Challenges to generalizabilityHard to get a
  broad sample. Target population must be pretty
  specific.
• Scheduling. Requires a block of time convenient
  for all.
• Inaccuracybehavioral problems in groups may
  distract respondents or even lead to playful
  responses
• Privacysetting may allow others to view
  subjects responses

20
Surveys

• Face-to-Face Interviews
• Strengths versus Weaknesses

21
SurveysFace-to-Face

• Weaknesses
• Cost. Payroll, driving, and so forth are
  expensive compared with other forms.
• Increased Bias.
• The appearance and style of the interviewer can
  bias participation or answers.
• Imagine questions about drug use or sexuality.
• Interaction between interviewer and interviewee
  can lower reliability
• Oral format limits types of questions that you
  can ask, although visuals may be used.
• Interviewer safety is of concern and fear can
  lead to falsified questionnaires.

• Strengths
• Permits complex questionsGreater specificity and
  variation in question types because theres
  someone to answer questions, explain, and probe.
• Highest response rate. Personal appeal increases
  likelihood that respondents will participate.
• ConvenienceAllows rescheduling of visits (helps
  with response rate too).
• Reduces disability biasinterviewer may assist
  respondents
• CAPI
• If using computer entry of responses,
  interviewing doubles as data entry.

22
Surveys

• Telephone Interviews
• Strengths versus Weaknesses

23
SurveysTelephone

• Strengths
• Permits complex questionsGreater specificity and
  variation in question types because theres
  someone to answer questions, explain, and probe.
  Not as good as face-to-face.
• Higher response rate (maybe) Personal appeal
  increases likelihood that respondents will
  participate. Not as high as face-to-face.
• ConvenienceAllows rescheduling of contact (helps
  with response rate too).
• Fastsurveys can be quickly completed
• Facilitates generalizabilityEasy to reach more
  people over large area, cheaper than face-to-face
  but more expensive than mailed survey
• Environmentally friendly compared with other
  methods except electronic

• Weaknesses
• Cost. Payroll is still an issue, but is cheaper
  than face-to-face.
• Number of questions is limited by low tolerance
  for being on the phone for long.
• Biases
• The linguistic style of the interviewer can bias
  participation or answers.
• Imagine questions about drug use or sexuality.
• If using area codes for sampling frame in an era
  of cell phones.
• Type of household that answers the phone. Who
  has a phone?
• Limits on the types of questions that you can
  ask
• relies on short-term memory and quick recall
• visuals are not possible
• Falling response rates because of American
  attitude toward phone.
• Americans hang up on strangers or screen calls
• Phone interruption is an annoyance

24
Surveys

• Electronic Surveys
• Strengths versus Weaknesses

25
SurveysElectronic

• Strengths
• Cheap. Depending on type, data are automatically
  entered by respondents
• Ease of editing questionnaire and analyzing data
• FastDistribution and response time is reduced.
• Higher response rates (maybe) than other methods,
  sometimes.
• Reduced biasYou get more candid responses. One
  researcher found Number of persons having sex
  with same-sex individuals doubles compared with
  mail and face-to-face surveys.
• Some complexity permitted. Interaction can be
  available, either through faqs, e-mail, or call
  to researchers.
• Environmentally friendly compared with other
  methods except telephone
• Respondents can view tally of responses
• Can reach samples on a global scale

• Weaknesses
• Selection Bias Generalizability
• Must have computer access, but can be used to
  generalize to computer-access population.
• Lacks complete sampling frame
• Email contact may be treated as SPAM
• Construction of questionnaire can be
  challengingoften need web or computer expertise
• Challenges to accurate response
• Assumes physical skills
• Literacy and language abilities
• Instructions may get tedious and confuse
  respondents
• Require electronic literacy
• This likely biases results.
• Glitches can kill participation for some
• Long-term response rates may be no better than
  other survey forms.

26
SurveysElectronic

• Of the types of electronic survey, those that
  carry us toward the population coverage of mail
  surveys and the response rates of the
  face-to-face survey will be judged the best.
• Current forms ranked from worst to best
• Public computers in high-traffic areasas bad as
  man on the street convenience samples.
• Web pagessampling can be done via normal
  methods, but often arent
• Distributed as E-mail
• Hybrids will likely rule the electronic survey in
  the future.