Frame work for Monitoring

1
Workshop for Effective Implementation of IWMP
Frame work for Monitoring Evaluation under IWMP
DOLR Govt. of India 7th 8th January 2014
2
Components to be Monitored under ME

1. Concurrent Process Monitoring
2. Input Output Monitoring
3. Impact Evaluation
4. Thematic/Special Studies
5. Sustainability Assessment
6. Case Studies/ Success Stories
7. Feedback and Dissemination Mechanism for Learning
8. Learnings and Adoptability
9. Envisaged Deliverables/Outputs

Learning Documentation
ME is an integral part of the projects day to
day operations rather than a periodic offline
activity
3
1. Concurrent Process Monitoring
Major Processes to be Monitored under IWMP

• Entry point activity (EPA)
• Sensitization and awareness
• CBO formation functioning
• Capacity building
• Transparency
• Action plan preparation (Net plan Complete
  action plan)
• Action plan implementation (Activity
  implementation)
• Flow of funds and procurement procedures
• Income Generating Activities (Livestock / Skill
  based Micro enterprises activities)
• Institutional functioning- NGOs, CBOs
  Government dept. or any other agencies.
• Demonstrations if any
• Withdrawal strategy
• Convergence (with other projects)
• Post project OM strategy
• Sustainability

Separate indicators (Both qualitative and
quantitative) to be developed for each of above
processes
4
Project Stages and Process Monitoring
Process Monitoring Cont.
A

• Indicators
• Prioritize
• Identify new
• Continue or discontinue old
• Make changes (if needed)
• Operationalise

CBO Ist Six months Formation Meetings Attendance Savings IInd six months Capacitation Savings Meetings Lending IIIrd six months Financial transaction Book keeping IVth six months IGA Value additions Profit earned Sustainability employment
SHG (Indicators)
Indicators to be redefined as per the stage of
the project Weightage to be shifted as per
project time chart
5
Process Monitoring carried out in Five-Stages
Collection of Data via Physical Surveys

Process Data and GIS analysis as applicable Ex graphs / tables

Interpret Data using MIS. And other wise Ex Trends Graphs / Patterns/ GIS outputs

Use Findings for Learnings and Corrective Measures Ex To change practice, inform policy, etc.

Documentation of best practices and strategies for up scaling
In An Effective ME the five stages are carried
sequentially
6
Process Monitoring Cont.
A Format for Process Monitoring (Observation)
Report - IWMP
Issues observed Situational Status (What is happening in the field) Reasons (Why it is happening) Suggestion / Scope for improvement (What is desirable) Stakeholders to be involved
CBOs Establishment Functioning
IWMP Implementation
Transparency
Capacity Building
IGA Micro Enterprises
7
Eg Monthly Report Extract
Taluk Bidar BIDAR DISTRICT
Sub-Watershed Bagdal IWMP 1/2009-10

NGO Santeshwari
Sl. No. Activities / Processes Situational Status Brief Analysis of the Status Observed
01 CBO training S1, S2, S3, U1, U2, U3, E1, E2 E3 trainings are completed in Bagdal sub watershed. All the trainings are completed. Overall participation level in S1, S2 S3 trainings was about 86.45. Overall participation level in U1, U2 U3 trainings was about 69.37. Overall participation level in E1, E2 E3 trainings was about 58.33. Details pertaining to the trainings have been provided in the previous reports.
02 Activity wise Documentation NGO level documentation is made. Photo documents of Grama sabha, Jatha Street play have been maintained. S1, S2, S3, U1, U2, U3, E1, E2 E3 trainings documents are maintained. Register book and photo documentation is also maintained.
03 Baseline survey Baseline survey of 2390 households is completed. Entire 2390 households data has been entered into the system.
04 EC functioning EC office establishment There are 4 ECs in Bagdal watershed. Offices have been opened for all these 4 ECs in Bagdal sub watershed. Office related logistics were provided in the month of August to 4 EC offices viz Bagdal, Khashempura, Bairanahalli Honnadi in this sub watershed. Books are not maintained in the EC office. Documents are not maintained in the EC.
05 S WC Work S WC Work has been carried out in this Sub-Watershed Visited Bagdal sub watershed on 19/01/2013 Visited Sy. No. 65 (Lakshmana) where 300 rmt of bunding work has been done instead of 600 rmt as per net plan. Bund dressing has not been carried out. Bund carried out is not as per specification.
8
Process Monitoring Cont.
Process Monitoring Methods Reports

• Reports on Monthly/ Quarterly basis
• Reflect the actual ongoing field status.
• All the process and activities will be monitored
  at random basis (at least 30 of Project site in
  a quarter)
• Watershed wise reports will be consolidated at
  the district level
• Report would also identify the key issues
  requiring immediate attention
• Flag-off issues to be taken up as special studies
  or thematic studies.
• Highlight the particular processes observed, the
  situational status, the reasons for the same and
  the institutions or individual associated in it.
• Also suggest field based remedial measures to
  improve upon.

9
2.INPUT-OUTPUT MONITORING
10

• Under the concurrent input-output monitoring, the
  financial and physical progress of the project
  would be monitored.
• Aspects such as fund release from the PIA to
  grass root level, actual condition/status of
  physical achievements, awareness creation, CBOs
  capacity building, etc will be monitored using
  specific indicators.
• This would be done through the MIS deployed and
  operationalised by the project.
• ME unit would verify the MIS data integrity
  through field verification on sample basis
  through process monitoring.
• Project will ensure flow of MIS data to ME for
  analysis.
• The MIS data thus received would be analyzed and
  the report would be made available to the top
  management once in six months.

11
(No Transcript)
12
3. Impact Evaluation
Address the overall impact on Natural Resources
Socio -Economic aspects

Household
level Short term Impact

Community / Village

Long term

Micro / Sub Watershed
APPROACH

• Before (Baseline)
• During (Midterm)
• End of the Project
• Post Project

With Without
Before After
Continues.
13
Impact Evaluation Cont.
Continues.
14
Sampling Approach
Impact Evaluation Cont.
Micro-Level (Micro watershed)
Sub watershed
State
Village-Level (Household Survey)
15
Impact Evaluation Cont.
Baseline Study

• At the outset, as a part of impact assessment,
  Baseline Survey would be conducted in the sampled
  watersheds.
• This analysis will provide a bench mark enabling
  a comparative analysis at the end of the project.
  
• The degree of project goals and objectives
  achieved would be assessed as against this bench
  mark.
• For a better comparison, control samples
  outside the watershed (i.e. untreated area with
  the similar topographic and socio-economic
  condition) also be analyzed.

Continues.
16
Impact Evaluation Cont.
End Project Impact Study

• At the End of the Project, ME will also carry
  out the impact assessment to establish the net
  impact of the project on the socio-economic and
  natural resource indicators.
• Data would be collected with respect to the same
  samples collected during the baseline.
• A comprehensive comparative analysis to the
  baseline status would be made in this report.

• The major perceivable impacts would be-
• Hydrological Ground Surface water, Drinking
  water status, etc.
• Soil Status Runoff, infiltration and moisture
  content, nutrient capacity, etc.
• Agriculture and allied Cropping pattern,
  cropping intensity, land reclamation,
  productivity and diversity
• Institutional Aspects CBOs, PRIs, any other
• Socio-economic Income level, Employment status,
  Migration, Women Empowerment, Educational status,
  Living condition, Livelihood options, etc.
• Environmental Land use land cover change, a
  forestation, biomass and carbon sequestration
  and
• Spillovers / Externalities (if any) Political
  involvement, linkages with other projects,
  financial linkages, etc.

17
Impact Evaluation Cont.
Sampling Strategy for Impact Assessment

• Sampling will be Based on Ridge, Middle, Valley
  Concept
• 25 of the projects will be studied
• Households will be selected using PPS Method
• Detailed questionnaire schedule covering relevant
  indicators will be used for Baseline, Midterm and
  End of the project enabling comparative analysis
• Data will be collected by the ME field staff
• Satellite imageries will be used for spatial
  analysis

18
Impact Evaluation Cont.
Indicators for Impact Assessment
Natural Resources
Socio Economic

• Surface Runoff
• Water Resource Development
• Ground Water level/Yield
• Change in Irrigated Area
• Crop Diversity
• Crop Yield
• Crop intensity
• Fodder Availability
• Afforestation
• Climate Change Biodiversity
• Land Use Change

• Sustainability of Structures /CBOs/IGA
• Micro Enterprises
• Employment Opportunity
• Migration Status
• Economic potential of Household Income
• BPL Family
• Animal Husbandry
• Impact on Milk yield

19
4. Thematic/Special Studies

• ME will also conduct certain thematic studies on
  key issues as identified.
• Some of the thematic areas include Capacity
  building, Investment pattern CBO participation,
  women empowerment, gender issue, level of
  participation, Income generating activities
  Run-off and land cover transformation, success
  stories, etc.

• The basic principle governing the thematic
  studies are that
• it should answer specific questions in the short
  run and help in policy derivations for next
  phase.
• should reflect both positive and negative aspects
  of the project.
• should be demand driven and locally identified
  (local specific) by the PIA.

It is proposed that about one or two of thematic
studies to be conducted per year.
Continues.
20
Thematic/Special Studies cont..
A few possible Thematic / Special Studies

• Investment Pattern
• Social Inclusion
• Women Empowerment
• Capacity Building
• Equity Issues
• Common Property Resources
• Income Generating Activities
• Live stock
• Short term Impacts
• Sustainability

• Run off Estimation
• Land use / Land cover changes
• Ground Water recharge
• Soil Fertility
• Water Quality

21
Thematic/Special Studies cont.
A Few New Thematic Areas (Suggestive)
Community Based Activities
Soil Water Conservation Activities

• Poverty Mapping
• Sustainability Micro Enterprises
• Input Output estimate for a given Micro
  watershed

• Soil erosion and Run-off estimation
• Crop Water Budgeting

Hydrology
Natural Resources

• Groundwater level and quality

• Reclamation of degraded land
• Agriculture Productivity
• Biomass Estimation
• CDM supportive studies (Assessment of Carbon
  Credits)

• Water Harvesting Structures Potential of Water
  availability and its utility.

Environmental Impact Assessment and Climatic
Change Studies

• Watershed impact on overall ecosystem and climate

22
5. Sustainability Assessment

• Post project sustainability of assets generated
  and also the institutions created is very
  essential.
• The level of involvement various institutions
  particularly the CBOs are crucial for the
  sustainability.
• In the long run the maintenance of the assets has
  to be taken up by the community on their own.
• Particularly the Common Property Resources (CPRs)
  needs to be maintained by the community.
• It requires a support system built in by the
  project during the project period.
• The withdrawal strategy needs to be evolved on
  these lines.
• The ME studies need to focus on the successful
  and effective functioning institutional
  arrangements for replication at a larger scale
  over time.
• The withdrawal strategy adopted in the IWMP-I
  will be assessed for evolving a strategy in the
  next phase of the project.
• Selected 10 per cent of the institutions from
  SHGs, CIGs, and AGs will be studied assessing
  their sustainability after completion of IWMP I
  projects.

23
6. Case Studies/ Success Stories

• Case studies to identify the specific project
  impacts would be carried out from time to time.
• These case studies will reflect the immediate
  outcomes and outputs of the project with respect
  to specific area or sector.
• The case studies reflect both positive and
  negative aspects.
• This will help to consolidate on the success
  stories of the project and to improve upon the
  weak areas if any.
• At least one case study will be prepared per
  project per year.

7.Feedback and Dissemination Mechanism for
Learning

• Along with respective State departments
  identified ME agencies evolve suitable
  dissemination mechanism for learning at different
  levels.
• It will assist department in reviewing the ME
  observations /findings, evolve compliance
  mechanism and provide feedback for
  decision-making.
• Dissemination of the findings through learning
  events is proposed to ensure that available and
  accumulated knowledge is fed back into an
  operational learning process that provides
  appropriate and timely information to relevant
  stakeholders, including management, to allow
  informed participation and on-going
  corrections/adjustments.

24
8. Learnings and Adoptability

• The regular feedback of Process Monitoring
  Reports will provide online corrections in the
  course of project implementation itself.
• The regular dissemination of information both
  from Process Monitoring and Impact Assessment if
  any would enhance the corrective options.
• The thematic studies carried out at a larger
  scale discretely would support the policy
  corrective measures to be adopted as a future
  strategy.
• Annual learning workshop would be organized at
  each district level and at the project level
  (State Level) to exchange the ME findings
  arrived at through various monitoring and
  assessment mechanisms.
• The End of Project reports would lead to a State
  level Workshop disseminating the major findings
  which would guide in developing new policy
  requirements if any.
• Based on the experience of various States a
  National level workshop (Annual/Once in two
  years) can be planned.

25
9. Envisaged Deliverables/Outputs

• The following outputs / reports will be envisaged
  during the project period, from the ME Sector.
• Inception report
• Process monitoring reports (Monthly/Quarterly,
  Half yearly/ Annually)
• Baseline Impact report (for the sampled
  watersheds) - Preparatory phase Evaluation
  report, Work phase Evaluation report and Final
  Evaluation report.
• Input Output reports (Annually)
• Thematic reports (Discrete)
• Case studies/Success stories

26
Monitoring Evaluation (In Nutshell )
Discrete monitoring
Concurrent monitoring
Impact Assessment
Process Monitoring
Self Evaluation/ Assessment
Input Output Monitoring
Participatory Monitoring
Independent ML
By CBOs (SWS-EC, AG SHG) facilitated by NGO
(Frequency)
Community and household level
Various levels through MIS Data
Continuous Monitoring
Yearly
Quarterly
Pre treatment Baseline
Mid term impact
In addition Thematic Studies, Case Studies and
other related studies (Special) if any will be
prepared as directed by VWDA
End of project
Post Project
27
Conclusion

• ME is a part of the project not an outsider
• ME is a facilitator of learning and is not for
  policing
• ME is an effective management tool
• ME ensures very high level of transparency

MEL D is the eyes and ears of the project

28
How an M E can be successful?

• Good Integration Co operation between ME
  Project Management
• ME Information should be unbiased, relevant,
  timely accurate
• ME documentation should address or resolve
  problems
• ME staff should have good skills and experience
  
• Monitoring and Evaluation should be integrated
  Single agency
• Project Evaluations and Assessments at frequent
  intervals
• Information Dissemination Upward Downward
• M E also plays a facilitators role for all
  stakeholders
• ME should be an integral part of the project

ME is not - cannot be - a substitute for
good management. For ME to be success it
needs to be driven by managers need for
information, their use of information and their
desire to create a learning environment
29
Road ahead

• Put in place an independent ME agencyFeb-2014
• (ToR and Bid documents uploaded on DoLR Web
  site)
• Ensure the critical elements of Concurrent
  Process monitoring in real time
• Arrange ME workshop at SLNA level (April 2014
• Realise importance of learning and policy
  linkages
• DoLR to contract directly Regional Independent
  ME Agencies

Note the Change in the receptive level and Body
language of DoLR- Reciprocate
30
Organizational Structure
Proposed MELD Structure
Project Director (ME)
HEAD QUARTER

• Natural Resource Scientist
• Hydrologist
• Social Scientist
• Statistics Scientist
• Remote sensing/GIS/IT Scientist
• Documentation Specialist

AT PROJECT LEVEL
District Coordinator (1 for 3 districts) Natural
Sciences / Social Sciences
Field Staff (2 for each District) Social
Sciences Natural Sciences
A senior person among the scientists will be
act as Project Director
31
ANNEXURE III
Multi-faceted Indicators of Impact at Different
Spatial Scales (In Case of Soil Conserving
Technologies)
Indicator Level Level Level
Indicator Farm Household Watershed
Biophysical Rate of erosion Soil fertility status Vegetation cover Crop yields Areas abandoned due to high erosion Food produced Access to water and fuel Quality of drinking water Quantity of drinking water Slopes stabilized Rainfall data Rate of siltation Quantity of water in reservoir Area under tree cover
Social -- Awareness on environmental aspects Rate of immigration Conflict for access to land and water Income redistribution Access to natural resources
Economic Fertilizer use Rate of profits Level of risk Level of diversification Income level Level of food security Level of assets Infrastructure network Biodiversity level Siltation cost
32
ANNEXURE IV
A Format for Process Monitoring (Observation)
Report - IWMP
Issues observed Situational Status (What is happening in the field) Reasons (Why it is happening) Suggestion / Scope for improvement (What is desirable) Stakeholders to be involved
CBOs Establishment Functioning
IWMP Implementation
Transparency
Capacity Building
IGA Micro Enterprises
33
ANNEXURE V
EXPECTED / ESTIMATED OUTCOME
List of Expected/Estimated Outcome to be compiled district wise List of Expected/Estimated Outcome to be compiled district wise List of Expected/Estimated Outcome to be compiled district wise List of Expected/Estimated Outcome to be compiled district wise List of Expected/Estimated Outcome to be compiled district wise List of Expected/Estimated Outcome to be compiled district wise
Sl. no Particulars Unit of measurement Pre-project status (Baseline) Expected Post project status Actual Outcome
1 Status of water table Depth in ft   Increased water table (Specify in feet)  
2 Ground water structures repaired/ rejuvenated Numbers   Increased  
3 Quality of drinking water Hardness   Reduced  
4 Availability of drinking water Distance from the house to source (m) Duration (months)   Reduced   Increased      
5 Increase in irrigation potential Pumping hours Yield (gallon/hr)   Increased pumping hrs Increased yield (gallon/hr)        
6 Change in cropping/land use pattern Area in ha (Crop wise Shift from - to Wasteland reclaimed   Shift to Agro-forestry/Horticulture Wasteland reduced (area in ha)      
7 Area under agriculture crop Area in ha   Increase (10-15)  
8 i. Area under single crop Ha   Increase (18-20)  
9 ii. Area under double crop Ha   Increase (10-15)  
10 iii. Area under multiple crop Ha   Increase (10-15)  
11 Net increase in crop production area Ha   Increase (20-25)  
12 Increase in area under vegetation Ha   Increase (10-15)  
13 Increase in area under horticulture Ha   Increase (15-20)  
14 Increase in area under fuel fodder Ha   Increase (10-15)  
15 Increase in milk production Number/Yield   Increase (20)  
16 No. of SHGs Number Savings (Rs)   Increase Increase (25-30)    
17 Increase in no. of livelihoods Number Diversification   Increase (10-15) Increased    
18 Increase in Income Rs   Overall by 30-35 increase    
19 Migration Numbers   Reduced  
20 SHG federations formed Numbers Savings (Rs)   Increase Increase (20-30)    
21 Credit linkage with banks No. of institutions Amount Rs.   Increase  
22 Resource use agreements Number of MOUs Executed   Increase  
23 Collection management Number of institutions   Regular  
24 Summary of lessons learnt        
34
ANNEXURE - VI LOG MATRIX FOR KEY INDICATORS TO BE
ASSESSED UNDER IWMP This is only an indicative
format for operationalization of project
assessment through MEL activities. However, this
is not a final document it can be used as guide
to conduct various activities under MEL. An
improvement on this matrix can be made as per the
actual field requirements. (For Neeranchal
project additional indicators will be added and
details would be worked out in the format after
inputs from various stakeholders)
Project Goals Objectives Key Performance Indicators Level of data collection Methods Frequency
Improve the productive potential of watersheds Changes in household income - Area under Agri. - Agri. Income / Ha Household Baseline (once) Survey, Periodic Assessment Annual
Improve the productive potential of watersheds Increment in crop yield Household/ micro watershed Baseline (once) Survey, Periodic Assessment Crop Seasons
Improve the productive potential of watersheds Ground water recharges 1. Change in GW level yield 2. Drinking water availability 3. Change in cropping intensity 4. Change in irrigated area Household/ micro watershed PRA Survey, Measurement Baseline and End of the Project
Improve the productive potential of watersheds Soil Erosion Soil loss Run-off Household/ micro watershed Field observation PRA Annual
Improve the productive potential of watersheds Change in Cropping Pattern Household/ micro watershed PRA / RRA, Survey Annual
Improve the productive potential of watersheds Changes in milk, fuel wood fodder production Household/ micro watershed Household Survey, FGD, PRA Survey Annual
Improve the productive potential of watersheds Treatment of non-arable land - changes in forest cover, forest produce, output from revenue land Micro watershed Sample Survey RSGIS Annual
35
When You Have Measurement, You Have
PERFORMANCE When You Have NO Measurement, You
Have EXCUSES What Gets Measured, Gets Managed Well
36
(No Transcript)