Overview of Census Evaluation and Selected Methods Pres. 2

1
Overview of Census Evaluationand Selected
MethodsPres. 2
2
Why evaluate ?

• Because the census is a huge operation (size,
  number of persons involved) prone to errors
• To provide users with some measures of quality of
  census data to help them interpret the results
• To identify types and sources of error in order
  to assist the planning of future censuses
• To serve as a basis for constructing a best
  estimate of census aggregates, such as total
  population, or to provide adjustment of census
  results
• Butnot to criticize the census takers !!

3
What errors in a census?

• Coverage errors
• Errors in the count of persons or housing units
  resulting from cases having been missed or
  counted erroneously
• Content errors
• Errors in the recorded characteristics of persons
  or housing units enumerated in the census (e.g.
  wrong age...)

4
Coverage errors

• Omissions
• Missing housing units, households and/or persons
  during census enumeration
• If the whole housing unit is missed, all
  households and persons living in the housing
  unit will also be missed
• Major causes of omissions are failure to cover
  whole land area of a country in creating EAs
• Mistakes made by enumerators in canvassing
  assigned areas
• Ambiguous definitions of EAs, unclear
  boundaries of EAs, faulty maps or coverage error
  during the pre-census listing exercise

5
Coverage errors

• Omissions contd.
• In addition, omissions within EAs can result
  because all or some of the members of the
  household were not present at the time of
  enumeration
• Proxy respondents can inadvertently or
  deliberately omit some members of a household

6
Coverage errors

• Duplications
• Occur when persons, households or housing units
  are counted more than once
• Reasons for duplications include
• Overlapping of enumerators assignments owing to
  errors done during pre-census listing and
  delineation
• Failure by enumerators to clearly identify
  boundaries
• In practice, the number of omissions usually
  exceeds the number of duplicates (net
  under-counts)

7
Coverage errors

• Erroneous Inclusions
• This includes
• Housing units, households and persons enumerated
  while they should have not been (e.g. babies born
  after the census reference date)
• Or enumerated in a wrong place

8
Coverage errors

• Gross error
• This is the sum of duplications, erroneous
  inclusions and omissions
• Net error
• This is the difference between over-counts and
  under-counts
• Net census under-count exists when number of
  omissions exceeds the number of duplicates and
  erroneous enumerations
• Net census over-count is the converse

9
Methods for evaluation of census errors

• Single Source of Data
• Demographic analysis of the census
• Interpenetration studies
• Multiple Sources of Data
• Non-matching studies
• Demographic analysis using previous censuses
• Comparison with administrative sources or
  existing surveys
• Matching studies
• Post Enumeration Surveys
• Record checks

10
Single Source of Data

• Demographic Analysis of the Census
• Average number of persons per household
• Sex- and age- ratios
• Tabulations...
• For an overall assessment of quality
• an age pyramid is a standard method
• stable population analysis can be undertaken as
  long as assumptions pertaining to constant
  fertility and mortality and no migration are met,
  for countries with declining mortality a
  quasi-stable model may be appropriate

11
Single Source of Data

• Interpenetrating studies
• Method involves drawing subsamples, selected in
  an identical manner, from the census frame
• Each subsample should be capable of providing
  valid estimates of population parameters
• Assignment of personnel (i.e. enumerators,
  coders, data entry staff, etc.) is done randomly
• The method helps to provide an appraisal of the
  quality of census information and procedures

12
Multiple Sources of Data Non matching studies

• Demographic Analysis
• Results from a census may be compared with data
  from other demographic systems such as vital
  registration systems
• For example, the cohort component method of
  demographic analysis uses successive censuses
  including
• life-table survival rates
• age-specific rates
• age-specific fertility rates and
• estimates of international migration

13
Multiple Sources of Data Non matching studies

• Demographic Analysis contd.
• Population can be projected forward to the
  reference date of the second census based on
  estimated levels of and age schedules of
  fertility, mortality
• The expected population is then compared to the
  enumerated population in the current census
• Yet another method is the comparison of age
  distributions of successive censuses

14
Multiple Sources of Data Non matching studies

• Demographic Analysis contd.
• Also the cohort survival method which is a
  regression method can be used, thus, population
  counts by age from two censuses and deaths by
  age during the inter-censal period are used to
  estimate coverage rate

15
Multiple Sources of Data Non matching studies

• Comparison with existing household surveys
• In theory any probability sample of households or
  persons can be used to evaluate coverage and
  content error in a census if
• They have identical items with same concepts and
  definitions
• They are independent from the census
• Must have been conducted close to the census date
  
• There should be sufficient identification
  information to facilitate accurate matching
  

16
Multiple Sources of Data Matching studies

• Record checks
• Census records are matched with a sample of
  records from identification systems such as the
  vital registration system
• The relevant respondents to the census
  questionnaire are traced to the time
  synchronized with the census
• Sources include
• Previous census
• Birth registrations
• School enrolment
• Citizen registration card
• Immigration registers etc.

17
Multiple Sources of Data Matching studies

• Record checks contd.
• Both coverage and content errors could be
  measured through the above comparisons
• To evaluate coverage efficiently the following
  preconditions are essential
• A large proportion of census population should be
  covered in record system
• The census and record system should be
  independent from each other
• There should be sufficient information in records

18
Multiple Sources of Data Matching studies

• To evaluate content efficiently the following
  preconditions are essential
• The record system should contain some relevant
  items covered in the census such as age, sex,
  education, relationship, marital status etc.
• Definitions of items should be identical between
  the census and the record system
• Countries that have used record checks include
  Demark, Finland, Norway, Sweden, Taiwan and Canada

19
Multiple Sources of Data Matching studies

• Post Enumeration Survey (PES)
• Consists of two separate coverage studies
• A survey conducted using a sample frame
  independent of the census. Persons from this
  survey are matched to the census to estimate the
  number of persons missed in the census
• A survey conducted using a sample drawn from
  persons enumerated in the census. This sample is
  re-enumerated to determine if the sample person
  or unit was erroneously enumerated (inc.
  erroneously located)

20
Multiple Sources of Data Matching studies

• Post Enumeration Survey (PES) contd.
• Results can be used to evaluate the reliability
  of some characteristics such as sex, age, marital
  status, relationship to reference person or head
  of the household.
• For some countries the results of PES can be used
  to adjust some census results
• Facilitates better interpretation of census
  results
• More discussion of PES is the focus of this
  workshop

21
Strengths and weaknesses of evaluation methods

• Single source
• Methods that depend on a single data source
  provide less insight into the magnitude and types
  of errors in the census data
• The merit is that the methods using such sources
  do not require additional data to be collected
• No need for sophisticated matching although this
  is also a limitation
• They provide a general impression of quality of
  the census data

22
Strengths and weaknesses of evaluation methods
(contd.)

• Single Source - Interpenetrating studies
• Gives good idea of different contribution of
  component errors to total census error
• Helps to identify operational stages that
  contribute to census error, thus identifying
  procedural limitations in a census
• Demerits include
• That it is an expensive operation demanding many
  field staff, intensive training and close
  supervision
• Relatively complex in designing and
  implementation

23
Strengths and weaknesses of evaluation methods
(contd.)

• Multiple sources - Non-matching studies
• Review census results at aggregate rather than
  unit level i.e. provides only estimates of net
  census error
• Evaluates very limited characteristics such as
  sex and age distributions
• Merit
• They are relatively cheap compared to matching
  studies

24
Strengths and weaknesses of evaluation methods
(contd.)

• Non matching methods - Demographic Analysis
• Advantage no additional data is needed to be
  collected to perform the analysis
• Less costly
• In statistical offices with sufficient numbers of
  demographers there is no need for additional
  staff to do the technical analysis
• On the negative side these methods provide less
  insight into the different contributions of
  component errors to total error in the census
• Quality of sources (Vital Statistics)

25
Strengths and weaknesses of evaluation methods
(contd.)

• Matching methods
• It provide separate estimates of coverage and
  content error
• Prospects of evaluating more characteristics
  compared to what can be done with non-matching
  studies
• Challenges
• Calls for high level technical skills including
  managerial
• Matching is expensive

26
Strengths and weaknesses of evaluation methods
(contd.)

• Post enumeration survey
• Merits
• Its results can be used to independently evaluate
  census coverage and content error, including
  reliability of selected characteristics
  collected in a census
• Incorporates matching of individuals or units
  between the census and PES
• Its results are generally more reliable than
  those of the census i.e. it justification for
  evaluation

27
Strengths and weaknesses of evaluation methods
(contd.)

• Post enumeration survey
• Challenges
• Requires highly skilled field and professional
  staff
• Matching is complex
• As it is supposed to be carried out immediately
  after the census at times there is lack of
  adequate funds to implement the PES exercise

28

• Thank You!