Research Methods I

1
Research Methods I

• Survey Research

2
What is a survey?

• Tool for obtaining information from a sample of
  individuals
• Many purposes
• Many techniques
• Many different people studied
• Often not done correctly

3
What can a survey measure

• Behavior how do people act?
• Attitudes what do people believe and how do
  they feel about things
• Expectations whats going to happen
• Etc.

4
Steps to Survey Research

• Define problem and hypothesis recall scientific
  method
• Planning of survey
• What do you hope to answer?
• What type of survey?
• Writing of survey questions
• Layout and response choices
• Refinement of survey
• Who to sample?
• Analysis of results
• Reporting results

5

• Defining problem
• Perhaps the most important part
• Is a survey the appropriate research tool?
• Advantages
• often quick and easy
• Can assess many different variables at once
• Surveys can be either quantitative or qualitative
  (or both)

6

• Disadvantages
• It is descriptive research not causal
• Surveys are self-report.
• Attitude-behavior consistency people often
  behave differently than they think.
• E.g., helping behavior
• People often do not know why they act the way
  they do
• Nisbett Wilson (1976)
• Self-serving bias

7
Planning of Survey Types of surveys

• Questionnaires written questions
• Can be mailed or administered in person
• Convenient and cheap
• Subjects cant ask for clarification
• Generally, very poor response rates.
• Better for sensitive questions

8

• Telephone surveys / interviews
• Often are computer assisted - CATI
• Allows for clarification
• Inexpensive
• Very time consuming
• Difficult for some subjects to keep something in
  memory if not in front of them.
• People are annoyed by phone surveys
• Potential for experimenter bias.

9

• Face-to-face interviews oral questions
• Plethora of data can be gathered.
• Provides for flexibility in gathering responses
• Requires large amounts of time
• Potential for experimenter bias

10

• Focus Groups
• Advantages
• Exploratory gets people to talk about their
  attitudes and perceptions
• In-depth information
• Can use various resources
• Good for qualitative research
• Disadvantages
• Not efficient
• Only get data from small sample
• Must have a good moderator
• Sensitive topics are difficult

11
Layout of survey questions

• If your questions are bad, your whole research
  project is bad
• Garbage in Garbage out
• Most important rule Keep it simple
• Use plain, simple language

12

• 1. Avoid jargon (including technical terms),
  slang, and abbreviations
• Do you like your job?
• Do you experience satisfaction in accomplishing
  the daily activities involved with your
  employment?
• 2. Avoid ambiguity, confusion, and vagueness
• E.g., what is your income?

13

• 3. avoid emotional language and prestige bias
• Al Gore argues that global warming is a big
  problem on our planet. Do you agree?
• 4. Avoid double-barreled questions asking two
  questions but expecting only one response.
• Example Do you think students should be required
  to write research papers and take exams in every
  class?

14

• 5. Avoid leading questions
• Ex. Dont you think youre boss is a great guy
• Survey questions should never convey your
  personal opinions.
• Youre not trying to influence them.
• Ethics in surveys
• Push polls surveys designed to influence
  personal opinion, not simply to assess it.

15

• 6. Avoid asking questions that are beyond
  respondents capabilities
• Why do you think Rwandan tanks fire on the Congo?
• Depends on your audience.

16

• 7. Avoid false premises do not begin with a
  premise that the respondents may not agree with.
• Ex. Because the war in Iraq has been a disaster,
  should we pull out our troops?
• 8. Avoid asking about future intentions
• What will you do in the future?
• Very poor predictor of behavior, especially
  hypothetical situations.
• Attitude-behavior inconsistency

17

• 9. Avoid double negatives
• I am not unduly concerned about germs.
• True or False
• Avoid negatives in general
• People take longer to evaluate problems that
  contain negative information, and they make more
  errors.

18

• 10. Avoid overlapping or unbalanced response
  categories
• Mutually exclusive
• Please indicate your age
• i. 17 to 18 years old
• ii. 18 to 20 years old
• iii. 20 to 22 years old
• Exhaustive
• Please indicate your age
• i. 17 to 18 years old
• ii. 19 to 20 years old
• iii. 21 to 22 years old

19

• Other principles
• avoid response set the tendency for individuals
  to respond in a certain way.
• Change direction of the wording
• Ex. Heredity plays the major role in determining
  one's personality.
• It is one's experiences in life which determine
  what they're like.
• Keep the survey relatively short. If it is too
  long, participants lose interest
• Lose the data altogether
• Data becomes lousy near the end of the survey.

20
Types of Questions and Response Choices

• Can obtain multiple types of content
• Opinions
• Attitudes
• Behavior
• Knowledge
• Be careful about sensitive or threatening
  questions.

21

• Contingency questions multiple part question
• Ex. Do you like Research Methods I?
• Yes go to question 2
• No ____________
• A) Is it because of the book?
• B) is it because of the content?
• Make sure contingency questions are
  understandable.

22
Matrix questions
http//www.pollingpoint.com
23

• Rating scales format for answering questions
• Likert scale level of agreement
• Strongly agree
• Agree
• Neutral
• Disagree
• Strongly disagree
• 5 point Likert scale.

24
Open vs. closed questions

• Closed questions multiple-choice type questions
• Likert scale items
• Yes/no items
• Etc.
• Quantitative approach
• Open-ended questions No set way of responding
  is suggested.
• Why do you come to work everyday?
• Qualitative approach

25

• Pros and cons of each
• Closed
• Advantages
• Easier for participants to respond
• Allows for statistical analysis
• Less confusing answers
• Standardization
• Disadvantages
• Answer options can influence responses
• Many problems if questionnaire is not constructed
  properly
• Ex. Lack of exhaustiveness
• Ex. Confusing question

26

• Open
• Advantages
• Allows a much greater range of responses
• Allows for creativity
• May find unanticipated results
• Disadvantages
• Statistical analysis is very difficult
• Large variety of responses
• Takes much longer
• Interpretation of results is more difficult

27

• Partially open-ended questions - include other
  and specify other
• Choice of question type depends on design.

28

• Word choice can influence the answer that you
  receive.
• Framing How the question is asked strongly
  influences the decision.
• Kahneman and Tversky studies
• Effect of leading questions

29

• Imagine that the US is preparing for the outbreak
  of an unusual disease, which is expected to kill
  600 people. 2 alternative programs to combat the
  disease have been proposed. Assume that the
  exact scientific estimate of the consequences of
  the two programs is as follows
• If program A is adopted, 200 of these people will
  be saved.
• If Program B is adopted, there is a one-third
  probability that 600 people will be saved and a
  two-thirds probability that no people will be
  saved. Which of the 2 do you favor.

30

• If Program A is adopted, 400 people will die.
• If Program B is adopted, there is a 1/3
  probability that nobody will die and a 2/3
  probability that 600 people will die.

31
Who to sample?

• Population vs. sample
• Population every person in the group you are
  interested in.
• Sample a selection of individuals from within
  the population.
• Representative samples- the sample taken is
  representative of the population as a whole.
• Must have
• Results from the sample can be generalized to
  everyone.

32
Types of samples

• Convenience sampling use whoever is available.
• Generally considered bad especially marketing,
  opinion, etc.
• May not be a representative sample
• For many types of research, convenience sampling
  is alright

33

• Random sampling every member of the population
  has an equal chance of being selected for the
  study.
• If sample size is large enough, the sample should
  be a representative one.
• Choosing the random sample
• Stratified sampling try to make the sample
  representative for all sub-groups or strata.