Impact Evaluation for Land Property Rights Reform

1
Impact Evaluation for Land Property Rights Reform

• Jonathan Conning
• Hunter College and The Graduate CenterCity
  University of New York

2
(No Transcript)
3
A brief polemic to make some obvious points

• Randomized controlled trials are not the gold
  standard
• Best ideal tool in the toolkit for measuring ATE
  but does not trump other evidence.
• The purpose is not to understand whether
  projects work but rather why and in what
  contexts projects can be expected to work
  (Deaton,2009).
• Average treatment effect from a RCT by itself may
  be of limited value in guiding policy on specific
  subgroups.
• Example ATE finds that project to build rural
  land registries had no impact.
• should we never again build a land registry?
• are land registries in specific localities (say a
  land-scarce region) are not worthwhile?
• Have we learned enough about why project failed
  to re-design and try again?
• We can only make such judgments by bringing
  additional evidence, and our understanding of the
  mechanisms at work, to play.

4
The purpose

• is to improve understanding of the mechanisms
  that make projects work or fail to work.
• ATEs measured in impact evaluation are useful and
  important, but..
• more broadly it is the process of building impact
  assessment into projects that generates knowledge
  for framing and focusing better decision making.
• The dialogue with policy makers and stakeholders
  about intended impacts and mechanisms and
  potential winners and losers, the challenges of
  data collection .
• Especially in land property rights reforms
  where interventions are deeper and more
  complicated (than, say, giving out iodine
  tablets) and success or failure often depends on
  context.
• Collect data to understand context and the
  mechanisms.
• Dont throw away data b/c cant analyze it with
  sufficient purity.

5
Why do impact evaluation?

• If built into program implementation and it
  engages stakeholders and local governance
  structures it can offer useful framing and
  feedback for better understanding of mechanisms
  for better policy design.
• Scaling up and replication
• Accountability, open debate and good PR

6
Types of Intervention

• Land Administration, titling and other tenure
  reforms
• Area and community-wide cadastral surveys,
  improved registries and adjudication.
  Village-level boundaries and governance.
• Individual regularization, registration,
  issuance of titles.
• Redistributive Land Reforms
• Imposed
• Negotiated
• via Restitution

7
Few Impact Studies

• Very few carefully identified impact studies on
  land issues.
• Very few No purposefully designed impact studies
  with baseline.
• Most existing studies had to make up for lack of
  data on comparison group and no baseline survey.
  
• Evidence to date from natural experiments and
  observational data
• Feder et al in Thailand - QR, HH FE
• Galiani et al in Argentina -QR, ITT
• Field et al in Peru QR, pipeline
• Carter and Olinto in Paraguay structural model
  of selection
• Akee on Native-American lands in South-west, QR
• Data collection should be built into program
  design.
• Much has been learned from evidence collected,
  concerns over biases in the estimates of ATE may
  remain in doubt.

8
Property Rights and Property Reforms

• Property rights established via
• Individual actions (e.g. investments to
  establish possession
• Community sanction (e.g. customary tenure,
  neighbors acceptance)
• State Enforcement (e.g. enforcement of legal
  title by courts or police)
• Claims can conflict and overlap. Many disputes
  are settled privately, others politically.
• Land administration and reform programs aim to
  facilitate process and/or to politically
  re-assign rights.
• Implication for Impact Evaluation
• Program placement and timing non-random
  (politically determined)
• Self-selection of beneficiaries
• Differentiated impacts

9
Dar es Salaam, Tanzania
Source Google Earth
10
Chogo village, Handeni, Tanzania
11
Expected impacts of interventions

• Household level
• Wider society/economy

12
Possible Positive Impacts

• Investment Demand Effect
• Credit Supply Response
• Gains-to-trade effects
• A few others mentioned in literature
• Attitudes and outlook, civic participation,
  willingness to pay taxes, household labor supply,
  investment in education, area crime.

13
From Pagiola (1999) adapted from Feder and
Nishio (1999).
14
The endogeneity Problem. How did the potential
beneficiary get to the front of the line?
15
Negative Impacts?

• Badly designed top-down intervention could
  disrupt existing informal property arrangements
  (built upon Coasian bargains and balanced
  overlapping claims.
• Costly conflicts, rent-seeking
• Land grabs and inefficient rush to privatize
  common areas.
• Strengthen elites while leading to loss of
  security for weakest stakeholders
• World Bank programs are supposed to be designed
  to avoid the above, but the road to hell is
  paved with good intentions.
• All the more reason to study and use impact
  assessments, as well as case studies and
  participatory appraisals to understand mechanisms
  in small pilots before scaling up and to embed
  impact assessment in policy design process.
• Even in the best designed programs, rarely is the
  intervention win-win. Should identify losers as
  well as winners.

16
Household Level Impacts

• Income, Production, Consumption, Safety Net
• Investments (under- or over-)
• Health, Education
• Market participation
• Land Values, sale and lease activity
• Labor supply
• Credit Access
• Intra-household bargaining
• Attitudes and outlook
• Measurement issues

17
Wider Impacts

• Reduction of conflict and transaction costs
• Benefits or costs spillover to non-target groups
• More active land markets
• Labor markets
• Financial Markets
• Public finance and public goods
• Environment conservation
• Spillover effects may complicate evaluation

18
Data Collection(See also Deininger (2004))

• Baseline
• Build on standard LSMS-type surveys with added
  land module
• Collect information on
• .
• land transaction histories, info on partners.
• plot level data
• intra-HH distribution of rights and
  transferability
• property-related transaction costs, HH knowledge
  of law
• HH expected impact of reforms (e.g. expect gain
  in land value).
• HH attitudes
• .
• Data from communities and reform programs.
• existing property rights arrangements, governance
  structures
• variables explaining program placement and
  beneficiary selection
• program rules and implementation features
• measures of community-level property rights
  insecurity

19
Impact EvaluationChallenges

• Measuring treatment and outcome
• Finding right comparison group
• Interpreting findings

20
Finding a Comparison Group

• Comparison simple if treatment (e.g title)
  randomly assigned, but
• Programs are targeted to specific groups and/or
  demand driven
• e.g. most politically organized communities
• Participants self-select
• e.g. occupants have already made costly
  investments/taken chances, first in line value
  title more highly

21
Attribution Problem

• Participants might have had higher outcome even
  in absence of treatment.
• Might falsely attribute outcome to treatment when
  (fully/partly) due to failure to control for
  observable and unobservable differences between
  treatment and comparison groups
• e.g. Participants more risk-taking, higher
  entrepreneurial drive, better political
  connections, better access to credit, etc.

22
Solutions

• Collect good baseline data
• Data before and after intervention.
• Good outcome measures and proxies and data that
  may account for participation.
• Build comparison groups carefully
• Via various randomization devices where possible
• By controlling for observable differences
• Not difficult or costly if built into program
  design.

23
Solutions

• Compare different outcome of treated to that of
  comparison using
• D Difference
• land value of titled before and after
  intervention
• land value of titled vs. non-titled after
  intervention
• DD Double Difference
• increase in land value of titled vs. non-titled
• Extensions Differentiated/non-additive
  impacts, fixed effects
  (household, community)

24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
Solutions

• Compare different outcome of treated to that of
  comparison using instrumental variables method
• Compare land value of titled vs. non-titled
  after intervention
• Use program eligibility rules as instrument for
  selection into treatment
• Extensions Differentiated/non-additive
  impacts, fixed effects

29
Data Collection(See also Deininger (2004))

• Baseline
• Build on standard LSMS-type surveys with added
  land module
• Collect information on
• .
• land transaction histories, info on partners.
• plot level data
• intra-HH distribution of rights and
  transferability
• property-related transaction costs, HH knowledge
  of law
• HH expected impact of reforms (e.g. expect gain
  in land value).
• .
• Data from communities and reform programs.
• existing property rights arrangements, governance
  structures
• variables explaining program placement and
  beneficiary selection
• program rules and implementation features
• measures of community-level property rights
  insecurity

30
(No Transcript)
31
Methods for building Comparison Groups

• Experimental Randomization
• Pseudo-Experimental
• Controlling for observable differences

32
Experimental Randomization

• Ideal
• Often not feasible

33
Pseudo-experimental Discontinuity

• Pipeline
• e.g. comparison group from those still on
  waitlist (if order is not affected by groups
  characteristics)
• Geography
• e.g. treated if on one side of arbitrary
  boundary and unexpected
• Program Eligibility Criteria Rule
• e.g. arbitrary eligibility cutoffs

34
(No Transcript)
35
Controlling for Observable Differences

• Regression methods
• Propensity score methods