Overview of Census Evaluation and Selected Methods Pres' 1

1
Overview of Census Evaluationand Selected
MethodsPres. 1
2
Census Errors

• Omissions missing housing units, households
  and/or persons during census enumeration
• When you miss the whole housing unit, it means
  all households and persons living in the housing
  unit will also be missed
• Major causes of omissions are failure to cover
  whole land area of a country in creating EAs
• Mistakes made by enumerators in canvassing
  assigned areas

3
Census Errors (contd.)

• Omissions contd.
• The latter problem may be due to unclear
  boundaries of EAs, faulty maps or coverage error
  during the pre-census listing exercise
• Canvassing errors can be caused by ambiguous
  definitions of EAs, faulty maps, and laxity of
  some enumerators
• In addition, omissions within EAs can result
  because all or some of the members of the
  household were not present at the time of
  enumeration

4
Census Errors (contd.)

• Omission contd.
• Proxy respondents can inadvertently or
  deliberately omit some members of a household
• Duplications
• Occur when persons households or housing units
  are counted more than once
• Reasons for duplications include
• Overlapping of enumerators assignments owing to
  errors done during pre-census listing and
  delineation
• Failure by enumerators to clearly identify
  boundaries

5
Census Errors (contd.)

• Duplication contd.
• In practice, the number of omissions usually
  exceeds the number of duplicates
• This implies that it is common, for countries to
  have net under-counts
• Erroneous Inclusions
• This includes housing units, households and
  persons canvassed in a census while they should
  have not been or were canvassed in a wrong place

6
Census Errors (contd.)

• Erroneous Inclusions contd.
• An example, is entering in census forms
  people who died before the census date
• Gross error
• This is the sum of duplications, erroneous
  inclusions and omissions
• Net error
• This is the difference between over-counts and
  under-counts

7
Census Errors (contd.)

• Net error contd.
• In measuring net coverage error we note that
  omissions result in underestimating the
  population while duplications and erroneous
  inclusions result in overstatement of the
  population
• Thus net census under-count exists when number of
  omissions exceeds the number of duplicates and
  erroneous enumerations
• Net census over-count is the converse

8
Why evaluate ?

• To provide users with some measures of quality of
  census data to help them interpret the results
• To identify types and sources of error in order
  to assist the planning of future censuses
• To serve as a basis for constructing a best
  estimate of census aggregates, such as total
  population, or to provide census results
  adjusted
• Butnot to criticize the census takers !!

9
Data Sources for Census Evaluation

• Single source of data
• An example is one census whose results can be
  used for demographic analysis
• Interpenetration studies ( see definition below)
  which are used to evaluate the implementation
  procedures of a particular census
• Multiple sources of data
• In order to do matching studies, there is need
  to compare two independent sources of data e.g.
  Census and PES

10
Data Sources for Census Evaluation (contd.)

• Record checks
• Census records are matched with a sample of
  records e.g. from a vital registration system
• The relevant respondents to the census
  questionnaire are traced to the time
  synchronized with the census
• Comparison with other surveys
• In theory any probability sample of households
  or individuals can be used to measure coverage
  and content error in a census if they have
  identical items using same concepts and
  definitions

11
Data Sources for Census Evaluation (contd.)

• Non-matching studies
• Demographic analysis based on two censuses
• Census results compared with administrative
  records and
• Census results compared to existing survey
  results

12
Census Evaluation Methods

• Demographic Analysis
• Results from a census may be compared with data
  from other demographic systems such as vital
  registration systems
• For example, the cohort component method of
  demographic analysis uses successive censuses
  including
• life-table survival rates
• age-specific rates
• age-specific fertility rates and
• estimates of international migration

13
Census Evaluation Methods (contd.)

• Demographic Analysis contd.
• Population can be projected forward to the
  reference date of the second census based on
  estimated levels of and age schedules of
  fertility, mortality
• The expected population is then compared to the
  enumerated population in the current census
• Yet another method is the comparison of age
  distributions of successive censuses

14
Census Evaluation Methods (contd.)

• Demographic Analysis contd.
• Also the cohort survival method which is a
  regression method can be used, thus, population
  counts by age from two censuses and deaths by
  age during the inter-censal period are used to
  estimate coverage rate
• For an overall assessment of quality
• an age pyramid is a standard method
• stable population analysis can be undertaken as
  long as assumptions pertaining to constant
  fertility and mortality and no migration are met,
  for countries with declining mortality a
  quasi-stable model may be appropriate

15
Census Evaluation Methods (contd.)

• Interpenetrating studies
• Method involves drawing subsamples, selected in
  an identical manner, from the census frame
• Each subsample should be capable of providing
  valid estimates of population parameters
• Assignment of personnel (i.e. enumerators,
  coders, data entry staff, etc.) is done randomly
• The method helps to provide an appraisal of the
  quality of census information and procedures

16
Census Evaluation Methods (contd.)

• Record checks
• Census records are matched with a sample of
  records from identification systems such as the
  vital registration system
• Sources include
• Previous census
• Birth registrations
• School enrolment
• Citizen registration card
• Immigration registers etc.

17
Census Evaluation Methods (contd.)

• Record checks contd.
• Both coverage and content errors could be
  measured through the above comparisons
• To evaluate coverage efficiently the following
  preconditions are essential
• A large proportion of census population should be
  covered in record system
• The census and record system should be
  independent from each other
• There should be sufficient information in records

18
Census Evaluation Methods (contd.)

• To evaluate content efficiently the following
  preconditions are essential
• The record system should contain some relevant
  items covered in the census such as age, sex,
  education, relationship, marital status etc.
• Definitions of items should be identical between
  the census and the record system
• Countries that have used record checks include
  Demark, Finland, Norway, Sweden, Taiwan and Canada

19
Census Evaluation Methods (contd.)

• Comparison with existing household surveys
• In theory any probability sample of households or
  persons can be used to evaluate coverage and
  content error in a census if
• They have identical items with same concepts and
  definitions
• They are independent from the census
• Must have been conducted close to the census date
  
• There should be sufficient identification
  information to facilitate accurate matching
  

20
Census Evaluation Methods (contd.)

• Overview of Post Enumeration Survey (PES)
• This is a complete re-enumeration of a
  representative sample of census population and
  matching each individual enumerated in a PES with
  information from the census
• An independent probability household survey whose
  broad objective is to determine the magnitude of
  coverage and content error
• Coverage error refers to people missed in the
  census or erroneously included

21
Census Evaluation Methods (contd.)

• Overview of Post Enumeration Survey (PES) contd.
• Content error evaluates the response quality of
  selected questions in a census
• Its results can also be used to evaluate the
  reliability of some characteristics reported in
  the census
• For some countries the results of PES can be used
  to adjust some census results
• Facilitates better interpretation of census
  results
• More discussion of PES is the focus of this
  workshop

22
Strengths and weaknesses of evaluation methods

• Single source
• Methods that depend on a single source data
  provide less insight into the magnitude and types
  of errors in the census data
• The merit is that the methods using such sources
  do not require additional data to be collected
• No need for sophisticated matching although this
  is also a limitation
• They provide a general impression of quality of
  the census data

23
Strengths and weaknesses of evaluation methods
(contd.)

• Demographic Analysis
• Advantage no additional data is needed to be
  collected to perform the analysis
• Less costly
• In statistical offices with sufficient numbers of
  demographers there is no need for additional
  staff to do the technical analysis
• On the negative side these methods provide less
  insight into the different contributions of
  component errors to total error in the census
• Quality of sources (Vital Statistics)

24
Strengths and weaknesses of evaluation methods
(contd.)

• Matching methods
• It provide separate estimates of coverage and
  content error
• Prospects of evaluating more characteristics
  compared to what can be done with non-matching
  studies
• Challenges
• Calls for high level technical skills including
  managerial
• Matching is expensive

25
Strengths and weaknesses of evaluation methods
(contd.)

• Non-matching studies
• Review census results at aggregate rather than
  unit level i.e. provides only estimates of net
  census error
• Evaluates very limited characteristics such as
  sex and age distributions
• Merit
• They are relatively cheap compared to matching
  studies

26
Strengths and weaknesses of evaluation methods
(contd.)

• Interpenetrating studies
• Gives good idea of different contribution of
  component errors to total census error
• Helps to identify operational stages that
  contribute to census error, thus identifying
  procedural limitation in a census
• Demerits include
• That it is an expensive operation demanding many
  field staff, intensive training and close
  supervision
• Relatively complex in designing and
  implementation

27
Strengths and weaknesses of evaluation methods
(contd.)

• Post enumeration survey
• Merits
• Its results can be used to independently evaluate
  census coverage and content error, including
  reliability of selected characteristics
  collected in a census
• Incorporates matching of individuals or units
  between the census and PES
• Its results are generally more reliable than
  those of the census i.e. it justification for
  evaluation

28
Strengths and weaknesses of evaluation methods
(contd.)

• Post enumeration survey
• Challenges
• Requires highly skilled field and professional
  staff
• Matching is complex
• As it is supposed to be carried out immediately
  after the census at times there is lack of
  adequate funds to implement the PES exercise

29

• Thank You!