Dissemination and Use of Results from OVC Program Evaluations

1
Dissemination and Use of Results from OVC
Program Evaluations

• Florence Nyangara, PhD
• MEASURE Evaluation/Futures Group
• Dissemination Meeting, September 3rd, 2009
• Washington, DC

2
Primary Objective of OVC Program Evaluations

• Provide evidence to guide program decisions such
  as
• Scaling-up of best practices (models,
  strategies), and
• Modify improve interventions - to make them
  effective
• Therefore, needed to collect quality and relevant
  data, analyze, and use results to guide OVC
  programs
• Monitoring and Evaluation to Assess and Use
  Results (MEASURE) Evaluation project

3
Overall Data Use Strategy

• Employed a comprehensive data use strategy
• Diverse stakeholders were involved throughout the
  study
• Ensured only relevant and useful data was
  collected by continuous consultations between
  researchers practitioners
• Data is packaged to meet the needs of target
  audiences
• Results are used to improve programs

4
Comprehensive Data Use Strategy

• Stakeholder engagement To ensure - support,
  ownership, relevance and sustainability.
• Diverse stakeholders were involved (e.g.
  beneficiaries, program)
• Capture different perspectives and information
  needs
• Stakeholders were involved throughout the study
• Get buy-in and promote ownership (consultation
  meetings)
• Continuous communication between researchers and
  various stakeholders for updates (feedback).

5
Comprehensive Data Use Strategy

• Ensured that only relevant data was collected- by
  holding consultation meetings with donor, program
  implementers, community, and beneficiaries. This
  helped to
• Identify key OVC program issues and information
  needs for service, program, and policy decisions
• Identify program models for evaluation
• Inform questionnaire development

6
Examples

• Example 1 - Dissemination of Case Studies
• The 1st feedback to share information on
  program descriptions, implementation challenges,
  opportunities
• Involved - program staff and in-country
  stakeholders
• Discussed and identified issues to consider for
  outcome evaluations
• Example 2 - Dissemination of preliminary outcome
  evaluation results
• Consultations with each program key in-country
  stakeholders to validate preliminary findings
• Presentations at international conferences and
  other forums helped interpret findings

7
Packaging Data for and Reaching Various Audiences

• Packaging information in various formats for
  diverse audiences
• Publications http//www.cpc.unc.edu/measure/ovc
• Six case study reports for each program
  evaluated1.
• Five briefing papers specific to each of the
  program evaluated
• Two summary papers - four CORE-funded programs
• Overarching paper on key findings
• Cost-effectiveness analysis paper
• One summary paper of key findings from the three
  studies in Tanzania
• Program-specific summaries of key findings
  (1-page) TSA
• Dissemination meetings
• Program sites
• In-country national level
• International level

8
Use of Results Workshops Tanzania Example

• Facilitated two workshops with OVC stakeholders
  in TZ
• TSA program staff (field-staff and managers)
• National OVC stakeholders (Implementing partners,
  government, donors, bilateral agencies, etc)

9
Objectives of Results Use Workshops

• Present and discuss the key findings
• Develop actionable recommendations based on the
  results
• Develop a data use action plan to implement each
  of the recommendations
• Develop and agree on a mechanism to monitor the
  data use action plan
• Follow-up plan

10
Use of Results Program Staff

• Findings were presented to the Salvation Army
  Mama Mkubwa program staff (field-supervisors,
  program managers, ME staff) from all regions
• Discussions of how TSA findings could be used to
  inform program improvement and the well-being of
  OVC.
• Program-specific recommendations
• Developed a data use action plan
• Developed and agreed on how to monitor the plan

11
Use of Results National Stakeholders

• Results were presented to National OVC
  stakeholders service providers, policy- makers,
  donors in TZ
• Discussions of how findings from three program
  evaluations could be used for decision-making.
• National OVC program actionable recommendations
• Developed a data use action plan
• Developed and agreed on how to monitor the plan.

12
Example of a Recommendation

• Researchers proposed recommendations were
  challenged participants came-up with their
  own, e.g.
• Researchers Need to review restructure Kids
  Clubs and home-visit activities to make them more
  effective.
• TSA staff volunteer motivation through
  incentives.
• National stakeholders more government
  involvement to develop guidelines that will allow
  volunteer growth, recognition, and ensure
  sustainability.

13
Information Use Bulletin

• Contains
• Reactions to the findings (surprises)
• Data use actionable recommendations plan (see
  National Tanzania)
• Responsibilities are assigned ( joint plans)
  which shows who will do what - implicated people
  do something
• Follow-up plans to assess if the actionable
  recommendations are implemented according to plan
• This formalizes the follow-up plans which is
  often forgotten after disseminations
• Qualitatively (January/February, 2010)
• Process Effect Increased demand for more data
  for decision-making - more programs want to
  conduct simple evaluations of these nature to
  find out if their key program components (IGA)
  are making a difference.

14

• MEASURE Evaluation is funded by the U.S. Agency
  for
• International Development through Cooperative
  Agreement
• GHA-A-00-08-00003-00 and is implemented by the
  Carolina
• Population Center at the University of North
  Carolina at
• Chapel Hill, in partnership with Futures Group
  International,
• ICF Macro, John Snow, Inc., Management Sciences
  for
• Health, and Tulane University. The views
  expressed in this
• presentation do not necessarily reflect the views
  of USAID or the United States government.
• Visit us online at http//www.cpc.unc.edu/measure.