Focusing on Development Outcomes in Projects

1
Focusing on Development Outcomesin Projects

• SARAR ME Workshop
• 20-21 June 2006
• New Delhi, India

2
Outline

• Management for results as an emerging practice
• Processes to manage quality
• Methodology to manage for results outcome
  focused design and supervision

3
1 - Management for Results is an emerging
practice within the development community

• Compared to Financial Management and Procurement,
  Management for Results is an emerging practice
  (within the last 10 years), now rooted in a
  broader agenda of the development community
  (Paris Declaration)
• Management for Results has been practiced in
  recent years under several titles (ME,
  management by objectives, balanced scorecard,
  budget rationalization)
• The emphasis on Management for Results grows out
  of a need to improve development effectiveness.
  It will help the Bank and its development
  partners to tell their results story in a more
  convincing way

4
2 - Quality Processes

• Processes to manage quality
• Country level
• Portfolio level
• Project level

5
The Banks view of what needs to be monitored now
includes the entire results hierarchy
Results Hierarchy

• Impacts
• Outcomes
• Outputs
• Activities
• Inputs

Longer-term benefits
Effectiveness
Results
Deliverables
Efficiency
Procurement Disbursements
6
Key Events in the Bank related to Results
Management

• Jan 96 - Performance indicators become mandatory
  in all new projects
• Feb 97 - Strategic Compact commits the Bank to
  100 quality-at-entry by FY2001
• June 97 - Revised supervision reporting system
  introduced, including indicators monitoring
• Aug. 97 - Revised project documentation
  introduced for new operations (including the
  Logical Framework)
• 2002 - Results Secretariat unit formed within the
  Bank
• 2003 - Logical Framework simplified to the
  Results Framework
• 2004 - Task Force on Organizational
  Effectiveness created
• 2005 Paris Declaration on Aide effectiveness

7
Magnitude of the Results Management Task Bank-wide

• 6 Bank regions (AFR, EAP, ECA, LAC, MNA, SAR)
• 1282 new projects under preparation
• 2372 existing projects under supervision
• US 20-25 billion in new lending per year the
  Bank also manages US 9 billion in trust funds

8
Country level Results Cycle
National Development Strategy Poverty Reduction
Strategy
9
SAR Portfolio level quality standard

• Defines portfolio management as a management
  function that seeks to maximize the overall
  quality of the portfolio by (i) carefully
  screening new projects prior to entry, and (ii)
  identifying projects that are performing poorly
  for early closure or restructuring.

10
Project level standards for SAR work

• a. Standard for Quality-at-Entry (Design)
• Project Concept Note (PCN)
• Project Appraisal Document (PAD)
• b. Standard for Project Supervision
• Project status report (PSR)
• Project completion report (PCR)

11
a. Quality-at-Entry Standard

• A fully satisfactory project contains a well
  conceived intervention model (i.e., storyline)
  which
• is designed to bring about specific, worthwhile
  and economic outcomes
• embodies a focused and realistic strategy
• can be expected to document and achieve results
  (outputs and outcomes) within the time frame and
  resources allocated to the project.

12
Typical Problems, Quality-at-Entry

• Bringing about specific, worthwhile, and economic
  outcomes Many projects have difficulty bringing
  about identifiable outcomes because they are
  focused on other things
• (i) the PDO is too high (i.e., CAS-level,
  long-term, sweeping, non-specific,
  non-attributable, unattainable), or
• (ii) the PDO is too low (i.e., output level,
  supply-driven, internally controlled by the
  project, or trivial).
• Embodying a focused and realistic strategy Many
  projects lack sufficient focus, realism and
  clarity with respect to strategy
• Documenting and achieving results within the time
  and resources available Many projects do not
  document results adequately, and then have
  difficulty convincing observers that the intended
  results were achieved.

13
b. Quality in Supervision Standard

• For existing projects in the portfolio, high
  quality supervision is characterized by
• proactive management of emerging problems, and
• consistent, honest reporting focused on
  documenting and achieving the expected
  development outcomes.
• High quality supervision is accomplished without
  extension of the time frame or expansion of the
  funding available to the project.
• Quality supervision also meets or exceeds the
  requirements of the Banks current guidelines and
  procedures for projects under supervision.

14
Typical problems in Supervision

• Supervision plans often lack a strategic vision
  and a necessary focus on outcomes
• Many projects fail to use key performance
  indicators as a management tool during
  supervision
• Too many projects lack a focus on outcomes in
  supervision reporting (ISR, MTR, ICR)

15
3 Focus and methodology

• It is important to focus on outcomes
• What is a focus on outcomes?

16
Terminology

• Results
• Impacts
• Outcomes
• Outputs

17
It is important to focus on outcomes
18
What is a focus on outcomes?

• Provides a convincing storyline (outcome level)
• Provides a clear and meaningful vision of success
  (objective statement)
• Answers the question What visible changes in
  behavior can be expected among end users as a
  result of the project, thus validating the causal
  chain?

19
Outcomes appear on the demand side of the
project, validating the causal chain
Project implementation environment
Impacts
(External)
Longer-term goals
-- ICR
Outcomes
Demand
Project Objective
Effectiveness
Outputs
Supply
Project Deliverables
(Internal)
Activities
Efficiency
Project Components
Inputs
Project Funding ()
Implementing Agency
20
The previous Logical Framework has been
simplified to the Result Framework
Key Performance Indicators (KPIs)
Hierarchy of Objectives
Level
Means of Verification
Critical Assumptions
Impact IM
Result Framework
PDO
Outcome OC
External
Internal
Output OP
Input IP
21
Outcome-focused projects utilize participatory
design
Output focus
Outcome Focus
22
Using a storyline to structure a design concept
Crossing the River of Uncertainties
Results
Future Vision of Success
Present Unsatisfactory Situation
23
The parts of a storyline
(PDO)
Problem
Strategy
Vision of Success
End-of-project KPIs
Baseline KPIs
Present unsatisfactory situation
Future vision of success
24
A dysfunctional storyline fails to provide a
consistent approach for the delivery of results
Results
Present Unsatisfactory Situation
Future Satisfactory Situation
River of Uncertainties
25
A good storyline bridges key areas of uncertainty
with a outcome-level approach
Specify indicators to verify success
Determine Strategy
Specify Vision of Success OC
Identify Problem
Results
Process Environment
Reporting Environment
Sector Context
Country Context
Present Unsatisfactory Situation
Future Satisfactory Situation
Project Storyline of PCN/PAD Crossing the River
of Uncertainty
26
AppendixProject examples

• Rural water
• Rural health
• Transport sector

27
Rural water example (causal chain)

• Increase in rural productivity and decrease in
  incidents of poor health caused by water-related
  diseases
• Significant increase in the adoption and use of
  improved water supply, sanitation and hygiene
  practices by participating households (objective
  achieved)
• Participating households gain access to completed
  water supply and sanitation facilities (output
  delivered)
• Target population selects among available options
  for water supply and sanitation services to meet
  their needs at affordable prices
• Awareness raising activities to promote new
  integrated approach
• Integrated water supply, sanitation and hygiene
  services are made available at the county level
• Capacity building for integrated delivery of
  water supply, sanitation and hygiene services at
  the county level

Demand
Supply
Problem Poor water supply, sanitation and
hygiene practices limit the productivity of rural
households and often adversely affect health
28
Rural water example storyline
29
Rural health example storyline
30
Transport sector example storyline
31
The End

• Focusing on Development Outcomes
• in Projects
• presented by Charles G. Chandler
• Assumption Analysis, Inc.
• www.AssumptionAnalysis.com