Field test options

1
Field test options instrument, instructions,
non-response/ refusals, and Interviewer
debriefing Washington Group Regional Training
Workshop Rio de Janeiro 19 20 September
2005 Margie Schneider HSRC, South Africa
2
Recap

• What we have covered so far
• Purpose of the questions and where they came from
• Core set of questions and extended set and Q X Q
  specifications
• Objectives of the test
• Translation protocol
• What we still need to go through
• How to design the test and related issues
• Enumerator training
• Plan for analysis and report writing

3
Different components of the testing process

• Pre-testing and expert review
• Testing internal validity on people with known
  disability status (quota sampling and linked to
  cognitive testing)
• Pilot testing as for full Census or survey
• Full field test
• Different sampling approaches will be discussed
  in relation to different testing protocols

4
Field Test options

• 2 questions to be answered
• What set of questions should be used in field
  test?
• What field test approach should be used?
• Consider your own context, need, funding, skills,
  etc. in deciding

5
Question sets

• Core set 4 2 (6 domains of functioning)
  Compulsory
• Core extended set on core domains highly
  recommended
• Core extended set further additional
  questions nice to have for analysis
• 2 additional domains (learning and interpersonal
  interactions)
• Psychological distress
• Country specific question set nice to have for
  comparison with prior data

6
What approach to testing?

• What factors to consider in deciding
• Country needs
• Capacity to undertake test
• Funds available
• Time available
• Planned activities, e.g. other surveys
• ?

7
Census or national survey

• Use of question set in planned Census or national
  survey
• Advantages
• Large sample (5000 10 000 or whole pop)
• Minimal additional cost for disability questions
• Disadvantages
• Not sufficient space for extended set
• Need to administer extended set and cognitive
  test to sub-sample (costs of time and additional
  training)

8
Special study small sample

• Select sample on basis of known disability
  status
• Allows for construction of 2X2 table for
  sensitivity and specificity calculations and
  identifying true and false positives and
  negatives use of gold standard
• Suggest around 200 true positives and 200 true
  negatives
• Interviewer does not know status of respondent
  (unless visible and obvious)

9
Special study (contd.)

• Advantages
• Larger set of questions
• Close observation of interview
• Cost is not exorbitant
• Disadvantages
• Might not get full population representation
• Cost of establishing true status is high
• What counts as the true positive? (beyond the
  easily observable)
• What is the gold standard?

10
Special study large sample

• Uses population based sample
• Sample size based on expected prevalence of the
  different types of activity limitations
• Prevalence rates are usually low and so need
  large sample to yield enough disabled
  respondents

11
Special study large sample (contd)

• Advantages
• Large data set
• Detailed set of questions - more than extended
  set
• Provides an indication of prevalence using core
  and extended sets
• Disadvantages
• Costly and time consuming
• Requires capacity to run and analyse

12
Field test and cognitive test

• All respondents have core set
• One sub sample of respondents have cognitive test
  (core extended are embedded)
• A second sub-sample of respondents do
• Core extended
• Core extended further Qs
• Core extended country specific questions
• Some do all ?

13
Non-responses and refusals

• How do you define these?
• How do you deal with these?
• Aim to
• Reduce non-response and refusals
• Manage them when they do arise

14
Reducing NR and refusals

• Use up to date sampling frame
• Clearly defined reasons for data collection
• Preparing the way prior contact, letters, etc.
• Adequate interviewer training
• Allow budget for call-backs
• Plan for follow up of NR and refusals
• Separate refusals, part refusals, non-contacts
  and sample loss (e.g. vacant dwellings)

15
Effect of each on analysis

• Sample loss does not create bias but reduces
  sample size need large enough initial sample to
  take these into account
• Refusals and non-contact bias where these
  respondents might be different to those
  respondents reached
• Need to understand reasons for refusals and
  non-contacts during pre-testing and pilot stage

16
Interviewer debriefing

• Interviewers are well placed to evaluate process
  and content at an early stage
• Interviewers should note comments in margins of
  questionnaires
• Different approaches
• Group discussion (focus group technique)
• Interviewer rating forms
• Standardised interviewer questionnaires
• Combination of all three

17
Enumerator training

• Washington Group Regional Training Workshop
• Rio de Janeiro
• 19 20 September 2005
• Margie Schneider
• HSRC, South Africa

18
General points

• Select some disabled interviewers
• Importance of all interviewers having a good
  understanding of what disability is and is not
  and how it relates to the questions asked
• Avoid using term disabled or with
  disabilities
• All interviews are face to face
• Translation training

19
General points (contd.)

• Confidentiality and understanding what this means
• Getting informed consent (ethics clearance?)
• Interviewers must be able to explain purpose of
  survey
• Read questions in set order and with set wording
• Editing in field of completed questionnaires
• Submitting of completed interviews to head office
  

20
Types of interviews

• Direct respondent answers for him or herself
• Interpreted an interpreter translates and
  respondent answers directly
• Facilitated a third party assists in explaining
  (e.g. intellectually disabled person)
• Proxy a person responds for another (e.g. child)
  

21
Interviewing disabled people

• Show respect and treat the person like anyone
  else
• Dont use first names unless permitted
• Address the person directly (not their attendant)
• Ask how you can adapt your presentation to make
  it easier (no need to ask what is wrong with
  person)

22
Hearing difficulties

• Lip reading
• Lighting
• Face the person
• Get persons attention before speaking
• Reduce background noise
• Set the context especially when changing topics
• Use written communication (literate)

23
Physical difficulties

• Accessibility of building where conducting
  interviews
• Presence of attendant and confidentiality issues
• Get to same level (e.g. sitting for person using
  wheelchair)
• Person to be seated comfortably
• Address person directly
• Pointing may be difficult
• Person may need breaks to move around

24
Visual difficulties

• Large print and small print for cue cards
• Braille versions of cue cards
• Good contrast printing for pictures and print
  (black on white or yellow)
• Identify yourself and others in the room verbally

25
Communication difficulties

• Clarify preferred mode of communication
• Repeat what you think was said to clarify unclear
  speech
• Limit to yes / no questions

26
Specific learning difficulties

• Manage problems in spatial orientation, hand-eye
  coordination
• Limit auditory, visual and tactile distractions
• Avoid written text
• Explain carefully (if verbal language skills are
  affected)

27
Intellectual difficulties

• Be careful with informed consent
• Explain terms simply
• Listen carefully
• Have familiar person (friend or relative) close
  by
• Use pictures or role play with little human or
  animal figures

28
Emotional or mental health difficulties

• Side effects of medication
• Break up interview if too fatigued
• Give encouragement and support
• Manage expressions of frustration
• Manage stress

29
Hidden difficulties

• Might not come forward with information because
  of fear of stigma
• Effect of medication
• May need to break up interview