Establishing a Monitoring and Evaluation System

1
Establishing a Monitoring and Evaluation System

• Joel Turkewitz
• World Bank
• April 2003

2
Key Issues

• Tendency to jump What should be monitored.
• ME involves more than just measuring a few
  variables

3
Core questions for us to Answer

• What is going to be monitored?
• What methodologies are going to be used?
• Who is going to monitor?
• How will this data be evaluated?
• What will be done with the evaluation?

4
1. WHAT IS GOING TO BE MONITORED

• What are our objectives?
• Types of monitoring
• Monitor Inputs
• Monitor process
• Monitor outputs
• Monitor Outcomes
• Traditional vs. Performance Monitoring

5
What is Going to be Monitored (cont)

• What resources exist to monitor and evaluate?
• Data
• Financial
• Human capacity
• Are we prepared for the results?

6
2. WHAT METHODOLOGIES ARE GOING TO BE USED?

• Selection of Indicators
• Clear precise and unambiguous
• Relevant Appropriate to the Subject at Hand
• Economical Available at reasonable cost
• Adequate Must provide a sufficient basis to
  assess performance.
• Monitorable Must be amenable to independent
  validation
• Salvatore Schiavo-Campo, 2000

7
Data Collection Methods
Key informantinterviews
Census
Conversationwith concernedindividuals
Questionnaire
Participant Observation
Householdbudgetsurveys
Direct observation
Reviewsof officialrecords
Communityinterviews
Fieldexperiments
Focusgroupinterviews
Fieldvisits
Informal/Less Structured Methods
More Structured/Formal Methods
8
Data Collection Issues that Need to Be Defined

• What data do we need?
• When does it need to be collected?
• How will it be collected?
• Who collects the Data?
• Who reports on the Data?

9
3. WHO IS GOING TO MONITOR

• National vs. Regional/International
• Central Agency v. Line Ministries
• Central Government v. Sub-national Governments
• Government v. Non-Government

10
Questions to ask about who will monitor

• Where is the information that is needed found?
• Where will the analysis take place?
• Who will act on the information that is created
  through monitoring?

11
4. HOW WILL INFORMATION BE EVALUATED

• Who will evaluate the monitoring data?
• What will be the unit of analysis?
• Raw numbers
• Percentages
• Organizational units
• Geographical
• Demographics

12
Evaluation on Info (cont)

• How frequently will the data be evaluated?
• What type of reports will be created?
• How will the reports be distributed?

13
5. WHAT WILL BE DONE WITH THE EVALUATION

• Will it be made public?
• How will it impact on
• Policy
• Individuals/programs
• Critical question Timing of evaluation reports
  if findings are to have impact

14
FINAL DIMENSION

• Establishing a realistic time framework.
• Movement to performance orientation is a medium
  to long-term reform
• Establishing an effective ME system will also
  take time.
• ME system needs to be put in place incrementally