Outline of Course

1
Outline of Course

• Basic principles of Survey Design and Analysis
• Theoretical underpinnings of the use of
  Questionnaires and Interviews as methodological
  tools for gathering data.
• Assignment Develop a questionnaire and short
  interview schedule and conduct a survey
• Quantitative and Qualitative analysis techniques
  will be covered using SPSS and nVivo software
  respectively. 

2
Learning Outcomes

• Understand the key principles informing survey
  design
• Have designed and administered a survey of your
  own
• Be able to critique the process of survey design,
  administration and analysis
• Have a basic understanding of the contribution of
  both quantitative and qualitative methods in
  conducting surveys

3
Course Outline

• Mondays Wednesdays 5 weeks
• Last week deferred for a week
• Theory Discussion
• Application
• Use of small groups and class group to design
  survey and develop questionnaire
• Assignment Critique of the process

4
Reference Book

• Mail and Internet Surveys, 2nd Edition, Don
  Dillman, Wiley, 2000
• Acknowledgements Myra ORegan, Aideen Keaney,
  TCD

5
Exercise

• What makes you answer a questionnaire?

6
The Process

• What is the research question?
• What did others do?
• How are you going to answer the question?
• How are you going to collect the data?
• How are you going to analyse the data?
• Writing up the results
• Dissemination

7
Methods of Data Collection

• Postal
• Self Completion
• Face to Face Interview
• Email
• Web
• Telephone

8
Criteria for evaluating methods

• Sample, facility to include questions, quality of
  data, administrative goals, respondents
  perspective

9
Sample

• Opportunity for inclusion is known
• Selection within sampling is controlled
• Respondents are likely to be located

10
Comparative modes
11
Questionnaire issues

• General flexibility of method
• Allowable length of questionnaire
• Ability to control question sequence
• Ability to ensure question completion

12
Facility to Include Question Types

• Complex questions
• Open-ended questions
• Screening questions
• Tedious boring questions
• Visual questions

13
Quality of data

• Odds of avoiding social desirability bias
• Odds of avoiding interviewer bias
• Odds of avoiding contamination by others

14
Criteria for evaluating methods Administrative
goals

• Odds of meeting personnel requirements
• Odds of meeting technology requirement
• Odds of keeping cost low

15
Respondent point of view

• Privacy
• Time to formulate answers
• Response rates
• Requirement for reading and writing
• Judgement of quality of response
• Importance of layout
• Ability to ask for clarification

16
Golden Rule

• Do unto your respondents as you would have them
  do unto you!

17
Telephone Surveys

• Advantages
• Computers can dial potential respondents
• Computers can structure questions and record
  responses
• Can contact ex-directory respondents
• Disadvantages
• Not good for visual material
• May be inappropriate for personal or sensitive
  questions
• Miss members of public without telephones

18
Mail surveys

• Advantages
• Least expensive method for lengthy Qs
• Need minimum staff and facilites
• Respondents have time to think
• Disadvantages
• Response rates tend to be lower
• Need address list
• Cannot be sure who completed Qe

19
Face to Face interviews

• Advantages
• Good for long and complicated questionnaires
• Good for specific materials
• Good for increasing co-operation
• Disadvantages
• Time consuming
• Expensive
• Interviewers needs training

20
Purpose of Questionnaire

• To obtain accurate information from all of a
  sample
• To convert the information needed into a set of
  specific questions that respondents can and will
  answer
• Must encourage the respondent to give the
  information
• Must minimize boredom, fatigue and non-response
• Provides a standard format on which all
  information can be recorded

21
Beginning the Process

• Clear research questions list them
• Identify your respondents
• Decide on method of administration
• Talk to interested parties

22
Start designing your survey

• Identify your research questions
• Identify your target sample
• Explore how to access your respondents
• Pencil paper questionnaire
• Electronic survey
• Discuss in small group

23
Tasks in constructing Questionnaire

• Sections list variables
• Tips Draw up tables for results to see what
  they will look like
• Comparisons with previous research search for
  existing questions/scales
• Draft new questions for each variable
• Type of question open vs closed

24
Tasks contd.

• Decide on question working
• Decide on form of response
• Decide on question sequence
• Precode the questionnaire if appropriate
• Decide on layout
• Write cover letter
• Test test test and administer

25
Challenge

• To obtain all the necessary information without
  obtaining unnecessary data

26
Criteria for assessing questions 1

• Does the question require an answer?
• To what extent do respondents have an accurate
  ready made answer?
• Can people accurately recall past behaviours?
• Is the respondent willing to reveal the requested
  information?
• Will the respondent feel motivated to answer each
  question?

27
Criteria for assessing questions 2

• Is the respondents understanding of response
  categories influenced by more than words?
• Is the survey information being collected by more
  than one mode?
• Is changing a question acceptable to survey
  sponsor?

28
Open Questions 1

• Completely Open
• Suggest improvements to service
• Word Association
• What is the first thing that comes to mind when
  you hear the following?
• BMW
• Ford
• Porsche
• Sentence Completion
• I would prefer to drive a Ford to a BMW because..

29
Open Questions 2

• Allows respondent to answer in own words, no
  influence
• Answers depend on how much respondents are
  willing to think
• Unanticipated responses
• Does not restrict to specific options
• Useful to give examples of answers in report

30
Open Questions 3

• When not possible to design closed
• Useful in pilot ot develop list of possible
  options for closed question
• More detailed info but takes more time for both
  respondent and you
• Depends on articulateness of respondent
• Depends on ability to record responses verbatim
• More difficult to code

31
Closed Questions

• Harder to develop have to ensure that all
  potentially important response alternatives are
  included
• Responses may be influenced by options provided
• Easier to code to record
• Less margin of error in recording

32
Open Vs Closed

• Open ended more costly, take longer to elicit,
  record code. May gather superfluous data
• Closed more economical, more risk of bias and
  restricted responses.
• Open closed can complement each other well
• Open to explore unknown issues Closed to test
  hypotheses

33
Exercise

• Closed Questions
• Devise 6 questions
• 3 questions 2 choices
• Yes/no
• Agree/disagree
• 3 questions 5 choices
• strongly agree/agree/not sure/agree/disagree/
  strongly disagree
• Ask your partner

• Open Questions
• Devise 3 questions
• Ask your partner
• Think about how to categorise and code responses