Assessing Success in Agricultural Development: What will it take to do better

1
Assessing Success in Agricultural
DevelopmentWhat will it take to do better?

• Lawrence Haddad
• Institute of Development Studies
• AFREA-NoNIE-3ie Conference

2
Outline

• 9.00 - 10.00 Introduction (Lawrence Haddad)
• Introduce goals of workshop
• Discuss our assumptions around the diagnosis of,
  drivers of and responses to current ME in
  Agriculture
• 10.00 - 10.45 Planning for and learning about
  impact? How can we be intentional (Peter Taylor)
  
• 10.45 - 11.15 Break
• 11.15 - 12.00 Peter Taylor, contd.
• 12.00 - 1.15 Aggregating impact can it be done?
  Should it? (Edoardo Masset)
• 1.15 - 2.15 Lunch
• 2.15 - 3.45 Constituency voice how can
  constituents strengthen dialogue with agencies
  about what works? (Alex Jacobs)
• 3.45 - 4.00 Break
• 4.00 - 5.00 Round up/ways forward and evaluation
  (Lawrence Haddad)

3
Goals of Workshop

• 1. Improved diagnosis of the challenges in Ag ME
• 2. Sharing and testing assumptions about drivers
  of challenges and responses
• 3. Sharing approaches that we think might be
  important parts of the response
• Impact Planning and Learning why are we doing
  evaluations and for who? (Session 1)
• Ways to better understand when evaluations do and
  do not lead to learning (Session 2)
• Developing indicators and mechanisms for
  aggregation of impacts across projects (Session
  3)
• Mechanisms for supporting and hearing
  constituency voice (Session 4)
• Introduce the ALINe initiative and explore
  potential linkages with related activities (wrap
  up)

4
The state of ME in agriculture

• Weak, especially with respect to welfare links
• Evidence
• Funding declines in agriculture
• Standing Panel on Impact Assessment--number of
  impact studies from CGIAR and its partners using
  income as an impact indicator 2008 (none), 2007
  (1), 2006 (4all from WARDA), 2005 (none), 2004
  (4)
• MIT Poverty Lab searches in their project
  database -- 25 health, 38 education, only 5
  agriculture (and these are all in Kenya)
• IDS/Keystone Critical friend review
• IDS/Keystone Review of grantees proposal

5
What did the critical friends (n20) tell us?

• About Agricultural ME
• lack of rigour
• failure to engage project staff in ME
• failure to involve farmers in ME
• inability to internalise gender sensitivity

6
What did the analysis of agriculture projects
tell us?
Percent of Ag projects (n35) that..

7
Our assumptions about why this is

• General
• Weak demand for ME in international development
  no colocation of funders, implementers and those
  directly affected
• Agriculture is particularly challenging ME
  terrain
• context specificity
• cultural and institutional embeddedness
• less consensus on outcomes sought
• longer and more indirect causal chain - theory of
  change is complex
• Benefits of ME in Agriculture
• full range not well understood, demonstrated or
  communicated
• do not accrue in practice
• incentives--commercial, organisational,
  individual
• capacity low, training often mechanical
• Costs of ME too high or perceived as too high
• not enough understanding of how to align
  incentives
• cost of developing tools to capture full range
  of benefits

8
To begin to address this...

• Focus on the Theory of Change, not the Practice
  of Routine
• Understand the multiple potential benefits from
  good ME and select main purposes - Impact
  Planning and Learning (IPL)
• Understand organisational and individual
  incentives to use and learn from ME or IPL data
• Strengthen accountability to those directly
  affected
• Ability to aggregate impacts
• Work hard to influence the wider field of
  agricultural ME

9
Agriculture is challenging terrain for ME
--spend a lot of time on developing the
interventions Theory of Change

• Make explicit vision of success
• Identify requirements for success
• Identify interventions that contribute to
  requirements
• Articulation of assumptions about why these
  requirements and interventions are important for
  vision of success
• Assess risk of assumptions not holding

10
Be clear on purposes of Agricultural ME
Make Projects Work Better
Demonstrate Impact
Generate Global Public Goods
Improve Human Welfare Transform Agriculture
Build Sustainability and Maintain Legitimacy
Influence Strategy
11
Questions for you to discuss

• From your own work, do you think ME in
  agricultural development is weak? If so in which
  areas?
• Why do you think this is the case?
• In your own work, what would help you to use ME
  data?