Systemic impact evaluation

1
Systemic impact evaluation

• Evaluating impacts of agricultural projects

The example of SOGUIPAH in Guinea
AFD - J. Delarue
2
What do we want ?

• Rigorously estimating impact (counterfactual
  approach)
• Accounting for impact heterogeneity
• Evaluating impacts on the lifetime of the
  projects outputs

3
What is SOGUIPAH?

• An agroindustrial company in Guinea SOGUIPAH
  (1987), produces rubber (exports) and palm oil
  (national market)
• Land allocation 23 000 ha
• Farmers deprived of land
• SOGUIPAH helps other farmers develop perennial
  cropping systems
• A model farmer 1 ha oil palm, 2 ha rubber and
  0.5 irrigated rice

4
SOGUIPAH
5
Evaluative questions

• What are the impacts of SOGUIPAH on the farmers
  income on the lifetime of the plantations?
• Impacts of land allocation
• Impacts of out-grower scheme
• Who is impacted by the project?
• directly
• Indirectly
• Why is there such an impact?

Evaluation context and methodological
challenges An evaluation 15 years after the
project started Very little baseline and
follow-up data
6
Estimating impacts a counterfactual approach

• Counterfactual at village level
• Counterfactuals for the different types of farms

Matching on observables their combination in
a system
7
Agrarian system analysis matching villages
Region
8
Farming system analysis matching farms
Baseline situation
WITH SOGUIPAH
WITHOUT SOGUIPAH
RESULTS 5 farming systems before SOGUIPAH 11
trajectories with SOGUIPAH 7 trajectories without
SOGUIPAH
9
Agrarian system analysis activities and results
ACTIVITIES 1. In depth interviews (focus
groups, farms) 2. Landscape observations 3.
Secondary data and triangulation
RESULTS 2 comparable villages
Understanding interactions of SOGUIPAH with the
agrarian system (land market, labour market,
prices) Indentification of farming systems
10
Farming system analysis activities and results
ACTIVITIES 1. In depth interviews (recall
method at farm level) 2. Field observations 3.
Productivity analysis nowadays and
triangulation
RESULTS Comparable farms before SOGUIPAH
Trajectories of farming systems with and without
SOGUIPAH Economic results with and without
SOGUIPAH for a variety of cropping systems
11
What is the impact we want to estimate ??
Impact ?

• A cross-section-based measure of average impact
  on the population is not what were looking for !

12
Modeling economic results with and without
SOGUIPAH for one farming system
2005
Translating trajectories into a simple economic
model based on financial analysis
Assessing future impact based on structural
trends (yields) and a few assumptions (prices)
Impact on the lifetime of the plantations
13
Accounting for heterogeneity
Impacts are highly differentiated ! Average
200 /y ranging from 200 /y to 1530 /y
14
Conclusions about the systemic impact evaluation

• A qualitative approach which allows the
  quantification of impacts
• A very simple economic model based on high
  quality data
• It assesses the impacts along the life time of
  the projects outputs
• It takes into account the complexity of real
  world
• It provides an in-depth understanding of the
  project effects and allows formulating
  recommandations