Ennsuring the quality of field work

1
Assuring good field work
Juan Muñoz
2
What happens when fieldwork is poor?

• A long and frustrating process of data cleaning
  becomes unavoidableThe data loose their
  policy-making relevance
• Data quality is not guaranteedThe process
  converges (at best) to databases that are
  internally consistent
• The process entails a myriad of decisions,
  generally undocumentedUsers mistrust the data

3
Key factors

• Manage the survey as an integrated project
• Implement the team concept in the organization of
  field operations
• Integrate computer-based quality controls to
  field operations
• Establish strong supervision procedures
• Ensure sufficient training
• Work with a reduced staff over an extended period
  of data collection

4
Management levels

• Core staff
• Survey manager
• Field operations manager
• Data manager
• Tactical options for the organization of field
  teams
• Mobile teams with fixed data entry
• Mobile teams with integrated data entry
• Sometime in the future the paperless interview

5
Mobile teams with fixed data entry

• Cote dIvoire (1984)
• Peru (1985)
• Ghana
• Pakistan
• Guinea-Conakry
• Mozambique
• Iraq (2006)

6
Composition of a field team
7
The team and its tools
Anthropo- metrist
Data entry operator
Supervisor
Interviewers
8
Two PSUs visited in a four-week period
Alama
Bamako
Regional Office
9
First week
Alama
Bamako
Regional Office
Operator remains in Regional Office
They complete first half of questionnaires in all
selected households
Rest of the team travels to Alama
10
Second week
Alama
Bamako
Regional Office
Operator enters first week data from Alama
They complete first half of questionnaires in all
selected households
Rest of the team travels to Bamako
11
Second week
Alama
Bamako
Regional Office
Supervisor gives Bamako questionnaires to DEO.
DEO gives back Alama questionnaires with flagged
inconsistencies
Rest of the team travels to Bamako and back
12
Third week
Alama
Bamako
Regional Office
Operator enters first week data from Bamako
Team completes second half of questionnaires.
They correct inconsistencies from first half
13
Fourth week
Alama
Bamako
Regional Office
Operator enters second week data from Alama.
Corrects inconsistencies from first round
Team completes second half of questionnaires.
They correct inconsistencies from first half
14
Fourth week
The result is a clean data set on diskette, ready
for analysis immediately after data collection
Regional Office
15
Mobile teams with integrated data entry

• Nepal (1992 and 2001)
• Argentina
• Paraguay
• Bangladesh (2000)

16
Mobile teams with integrated data entry
Bamako
Cocody
Alama
Team works with portable computers and printers
Regional Office
17
Mobile teams with integrated data entry
Bamako
Cocody
Alama
Operator travels with the rest of the field team
Regional Office
18
Mobile teams with integrated data entry
Bamako
Cocody
Alama
Data entry and validation almost immediate
Regional Office
19
Mobile teams with integrated data entry
Bamako
Cocody
Alama
Reduced trips to and from Regional Office to
selected PSUs
Regional Office
20
Mobile teams with integrated data entry
Bamako
Cocody
Alama
Regional Office
21
Benefits of integration

• Provides reliable and timely databases
• Provides immediate feedback on the performance of
  the field staff, allowing early detection of
  inadequate behaviors
• Ensures that all field staff applies uniform
  criteria throughout the full period of data
  collection
• Solves inconsistencies through direct
  verification of households reality, rather that
  through office guessing
• Is consistent with the total quality culture

22
Selecting and training field staff

• Why is it important
• How long does it take
• How is it organized

23
Example Day 2 of interviewer training

• Definition of household (and dwelling, family,
  etc.)
• Pictorial of a sample household
• Slide with an empty roster (explain case
  conventions, encodings, skip patterns, etc.)
• Fill the roster for the sample household (need
  for legible handwriting, recording of ages, use
  of a calendar of events, etc.)
• Role playing (trainer as a respondent, simulating
  borderline cases)
• Role playing (trainees interview each other)

24
The role of the team supervisor

• Manager/administrator ("traditional" role)
• Monitor completion of work
• Collecting and accounting for all of the
  questionnaires
• Paying interviewers/managing the fuel budget
• Administrative functions
• Sometime interviewer
• Quality control
• Continuous training of interviewers
• Random quality checks in the field

25
Supervision tasks

• Verification of questionnaires for completeness
• Completion of household roster ID of members
• Completion of all sections for all individuals
• Limited internal consistency
• Random re-interviews of households
• Observation of interviews
• Observation of anthropometrics
• Supervision of data entry

26
The paperless interview (CAPI)

• The option of the future
• Is used successfully by some statistical agencies
  for simple surveys (LFS and CPI price collection)
• Recent experiments have shown that
• Technology is already available(Lightweight
  notebooks and software development platform
  both Windows based)
• Can be cost-effective
• No negative serious externalities
• We still need to solve
• Questionnaire design
• Ergonomic aspects of the interview
• Interviewer training
• Development of supervision procedures adapted to
  the new technology (voice recording, use of
  GPSs, etc.)

27
Case study The IHSES Iraq Household
Socio-Economic Survey Presenter Shwan J Fatah
Sulaimania, KRG Stat Office

• Each cluster (6 households) visited by one
  interviewer in a 20-day period (a wave)
• Each household records food expenses in a diary
  for 10 days
• The interviewer visits each household seven
  times, before during and after the 10-day diary
  recording period
• During these visits, the interviewer
• Helps with diary recording
• Asks different questionnaire modules (education,
  heath, labor, etc.)
• Checks for inconsistencies in the data collected
  in previous visits

28
Case study The IHSES Iraq Household
Socio-Economic Survey (continued)

• This was possible by organizing the field workers
  into teams, composed of
• One supervisor
• Three interviewers
• One data entry operator
• Data was entered and checked in between
  interviewer visits
• Fieldwork concluded in January 2008
• A database is already available
• Preliminary outputs expected in March 2008