Questionnaire Design

1
Questionnaire Design
EPIET Introductory Course 2003 VERYIER DU LAC,
FRANCE
P McKeown
2
So far

• Introduction
• Outbreak or Incident
• Salmonella in Northern Ireland
• Steps in investigation
• Surveillance

? Questionnaire Design
Disease Detectives
? Looking for clues
? Building a case
3
Objectives

• To understand
• Why we use questionnaires
• The objectives of a questionnaire
• The advantages and disadvantages of
  questionnaires
• The different types of questions that can be used
  and the circumstances in which they would be used
• The way in which questionnaires can be designed
  and how to avoid some common problems and pitfalls

4
What is a questionnaire?

• An instrument (form) to collect answers to
  questions
• Collects factual data gathers information or
  measures
• A series of written questions / items in a fixed,
  rational order
• Every item is answered (unless instructed
  otherwise)
• Who completes it?
• The respondent public/professional
• Someone on behalf of the respondent
• The interviewer

5
Types of Questionnaire

• Prospective or retrospective
• Completed by participant or researcher
• Advantages of self-administered include
• Cheap and easy to administer
• Preserve confidentiality
• Can be completed at respondent's convenience
• Can be administered in a standard manner

6
Types of Questionnaire

• Advantages of interview-administered include
• participation by illiterate people
• clarification of ambiguity
• They can be administered face to face, via the
  telephone, by post or email

7
What makes an effective questionnaire?

• An effective questionnaire is highly structured
  to allow the same types of information to be
  simply collected from a large number of people in
  the same way so that the data can be analysed
  quantitatively and systematically

8
And why do we want one?

• A well designed questionnaire
• Will give you appropriate data which should allow
  you to answer your research question
• Will minimise potential sources of bias, thus
  increasing the validity of the questionnaire
• People are much more likely to complete a well
  designed questionnaire

9
Assumptions

• The use of questionnaires is based on certain
  assumptions
• That individuals will have had common
  understanding of particular issues and will share
  general beliefs
• (went to a wedding/ that we are trying to find
  out more about meningitis because this disease is
  a threat to our children and young people)
• That this understanding and these beliefs are
  shared and can be clearly expressed and quantified

10
Before starting..

• Questionnaire design should be an integral part
  of the study, not an add-on
• Therefore
• Decide on the goals of the study (for yourself
  and for others)
• Know the subject literature, experts
• Know the respondents - (SEG, occupation, special
  sensitivities, educational, ethnic)
• Know how much cost, time, sample size, response
  rate

11
Fundamental Principle

• An epidemiological investigation is a scientific
  loop that tests a hypothesis

Problem
Hypothesis
Question
Answer
Study
Results
Questionnaire
12
Objectives of a questionnaire

• To maximise the proportion of subjects answering
  the questionnaire as high a response rate as
  possible
• To ensure that the information obtained is as
  accurate and as relevant to our study as possible
• as simple and as focussed as possible

13
Advantages of questionnaires

• Can reach a large number of people relatively
  easily and economically (especially postal
  questionnaires)
• Provide quantifiable answers
• Relatively easy to analyse

14
Disadvantages of questionnaires

• May provide only limited insight into the
  problem
• Limited response allowed by questions
• You may not have asked the right questions eg
  Andorra, norovirus and ice-cubes
• Varying response
• they may not understand your question and may
  intrepret it differently
• Postal questionnaires ? low response rate
• Hard to chase after missing data need to get it
  right first time

15
Stages in designing a questionnaire

• Decide on aim of the study and the purpose of
  using a questionnaire
• Decide on content of the questions
• Decide on the format of the questions
• Decide on presentation and layout
• Formulate coding schedule (if appropriate)
• Pilot and refine questionnaire

16
Basic Rules

• KISS ? keep it short and simple
• Appearance is crucial and affects
• Response rate
• Ease of data summarisation and analysis
• Length of questionnaire shorter ? ?response
  rates
• Question order is important
• Easy ? difficult
• General ? particular
• Factual ? abstract
• Start with closed format questions.
• Start with questions relevant to the main
  subject.
• Try to avoid starting with demographic and
  personal questions

17
Basic Rules

• Number all items and pages
• Put an identifying mark on all pages
• Put the return address on the questionnaire
• Put the study title in bold on the first page
• Print directions in bold
• Remember the Unique Identifier

18
Basic Rules

• Group questions by topic or response options
• Dont put the most important item last
• Open with relevant non-threatening questions
• It is most likely to be completed if easy on the
  eye, relevant, logical and as short as possible.

19
Content of Questions

• Keep a clear focus on your research question
• Literature search what do we need to know in
  order to ask the most relevant and probing
  questions?
• Demographic information
• Contact information (if non-anonymised)

20
Format of Questions

• Ask only for the information you need
• Start with simple questions get them on your
  side
• Appropriate for responding audience eg middle
  class wedding guests vs prisoners
• Keep sentences simple and short

21
Format of Questions

• Ask for one piece of information at a time
• If you had not passed water for 6 hours but were
  watching a James Bond movie, and your wife
  brought you a sandwich would you get up to go to
  the toilet in the next 20 minutes? Y/N
• Remember dont know

22
Format of Questions

• Be precise Do you often have to get up at night
  to pass water? Y/N
• Be appropriate/sensitive bowel habits
• Aim always to minimise bias

23
Bias in Questions

• Information Bias systematic differences in the
  measurement of a response
• Recall bias eg cases being more likely to
  remember than controls
• Observer bias inter and intra a questionnaire
  is an important way to reduce this
• Non-response bias
• Those who respond are different from those who do
  not
• Reduce by ensuring high response rate or
• Correcting for during analysis (eg age, sex, SEG)

24
Format of questions

• Two main question formats
• Closed format ? give a fixed response
• Open format ? allow people to express their
  views in their own words
• What is your most distressing symptom? Please
  describe ________________________________________
  __________________________________________________
  ______

25
Open or Closed?

• Closed forced choice
• Simple and quick
• Reduces discrimination against those who are less
  literate
• Easy to code, record, and analyse results
  quantitatively
• Easy to report results
• Open format free text
• Allows exploration of issues
• Can be used even if a comprehensive range of
  alternative choices cannot be compiled

26
Closed Questions

• A number of types
• Straightforward response
• What is your age in years? _____ yrs
• How long have you worked for Company X? _____
• What is your sex (gender)? M ? F ?
• Did you stay in Hotel X on 23/6/02? Y ? N
  ?
• Did you eat dinner in the company canteen
• on 23/6/02? Y ? N ?

27
Closed Questions

• 2. Checklist
• Which of the following did you eat for dinner on
  23/6/02?
• Chicken ?
• Beef ?
• Ham ?
• Salad ?
• Egg mayonnaise ?

28
Closed Questions

• 3. Adjectival/adverbial responses
• Are you limited by hip pain during any of the
  following activities?
• Not at all Slightly Moderately
  Severely
• Walking ? ?
  ? ?
• Sitting ? ?
  ? ?
• Lying in bed ? ?
  ? ?

29
Closed Questions

• 4. Numerical rating scale
• How useful would you think a community methadone
  clinic in your area would be? (please circle)
• 1 2 3 4 5 6
  7
• Not at all useful Very useful

30
Closed Questions

• 5. Visual analogue scale
• At rest, how bad is your hip pain?
• (please mark where on the line below)

No pain
Worst pain imaginable
31
Closed Questions

• 6. Ranking scales
• Please rank these activities in order of the
  amount of pain they cause you. (1 would be most
  pain and 4 would be least)
• Sitting ?
• Running ?
• Lying in bed ?
• Walking ?

32
Closed Questions

• 7. Scales for measuring attitude eg Lickert
• A local methadone clinic would be useful for my
  community
• No, I strongly disagree ?
• No, I disagree quite a lot ?
• No, I disagree just a little ?
• Im not sure about this ?
• Yes, I agree just a little ?
• Yes, I agree quite a lot ?
• Yes, I strongly agree ?

33
Open Questions

• Used to provide fuller information
• Open, free-text form
• Often used in qualitative research, focus groups,
  at the preliminary and piloting stage of
  questionnaires
• What do you think is the reason you became ill?
  ________________________________________________
• Do you think service X would make your life
  easier? If so, in what way? If not, why not?
  __________________________________________________
  ______________________________________________

34
Problems and Pitfalls

• Avoid questions that ask two things at once - you
  wont know which bit people are answering
• Have you ever had neck pain and diarrhoea?
• Ambiguity.....
• e.g. Do you got to the toilet a lot?

35
Problems and Pitfalls

• Avoid jargon
• How often do you get up at night to PU? (pass
  urine)
• Should IVDUs be treated in the community?
• Avoid options that are not mutually exclusive
  when they need to be
• What age are you?
• 16-20 ?
• 20-25 ?
• 25-30 ?
• 35-40 ?

36
Problems and Pitfalls

• Avoid leading questions
• Do you think that the food in the hotel made you
  sick?
• Did the hotel staff seem unhygenic to you?
• Do you agree that the hospital staff were close
  to exhaustion?
• Avoid making the questionnaire too long
• Typographical / spelling errors
• Questionaire
• Diahurria
• Vommit
• Eppidemiology

37
Ensuring a High Response Rate

• Make the questionnaire relevant
• Ensure respondents see its potential and
  usefulness (we are trying to find out why you and
  your family/people in became ill while holidaying
  in this resort)

38
Ensuring a High Response Rate

• Include a brief covering letter
• Who you are
• Who you work for
• Why you are investigating/researching
• Where you obtained the respondents name
• How and where you can be contacted
• Absolute guarantee of confidentiality
• Self-addressed envelope!!

39
Questionnaire Validation

• Many existing questionnaires can be used or
  adapted eg SF-36
• Validated
• Others e.g. for an outbreak have not been
  validated and will need to be tested
• Piloting is a common way of doing this

40
Piloting and Evaluation

• Pilot with a similar group of people to your
  intended subjects
• Piloting a essential
• Highlights problems before starting
• Often concentrates initially on effects of
  alternative wording
• Then will concentrate on respondents and
  interviewers overall impressions
• With a final polishing after several amendments
  to ensure response rate

41
Presentation and layout

• Decide on the order of the items / questions
• Where to place sensitive questions?
• Make the layout clear, consistent and easy on the
  eye
• Try to get on overall simple feel
• Ensure there is adequate space to answer
• Choose a large enough font size
• Avoid flippancy
• Avoid fancy logos/printed on recycled paper/is an
  equal opportunity employer etc
• Do not slip to another page

42
Presentation and layout

• Using colour or printing questionnaire on
  coloured paper may help
• Use filter questions, if necessary
• Give clear instructions about how to answer the
  questions

43
Coding Schedule

• Questionnaire can be pre-coded
• Quicker and easier to hands responses prior to
  data entry eh Epi-Info or SPSS
• Examples
• M1 F2 DK3
• Ill1 Not ill2 DK3
• Single1 Married2 Separated3 Divorced4 DK5
• No medical intervention1 Saw GP2 Went OPD3
  Admitted Hospital4 Admitted ICU5 Died6 DK7

44
Summary

• A well designed questionnaire
• Will give you appropriate data which should allow
  you to answer your research question
• Will minimise potential sources of bias, thus
  increasing the validity of the questionnaire
• People are much more likely to complete a well
  designed questionnaire

45
FINALLY, keep your questionnaire short and the
questions simple, focussed and appropriate