Evaluating the Common Agricultural Policy

1
Evaluating the Common Agricultural Policy
Christiane Canenbley AGRI G.4 - Evaluation
of Measures applicable to Agriculture Studies
2
What is Evaluation?
3
Scope of Evaluation
4
Main Evaluation Criteria

• Effectiveness
• The extent to which objectives pursued by an
  intervention are achieved
• Efficiency
• Best relationship between resources employed and
  results achieved in pursuing a given objective
  through an intervention
• Relevance
• The extent to which an interventions objectives
  are pertinent to needs, problems and issues

5
What are the formal requirements?
6
General Legal Obligations for Evaluation

• Financial Regulation (EC) 1605/2002, Article 27.4
• In order to improve decision-making,
  institutions shall undertake both ex ante and ex
  post evaluations in line with guidance provided
  by the Commission. Such evaluations shall be
  applied to all programmes and activities which
  entail significant spending and evaluations
  results disseminated to spending, legislative and
  budgetary authorities
• Implementing Rules, Commission Regulation
  2342/2002, Article 21(2)b
• Activities financed on an annual basis shall have
  their results evaluated at least every six years

7
Evaluation Standards andGood PracticesSEC(2002)
5267

• Profile, role, tasks and resources of the
  evaluation function
• Establishment of evaluation functions to support
  policy formulation
• Appropriate resources
• Management of evaluation activities
• Multiannual evaluation programme and an annual
  evaluation plan
• Appropriate internal feedback mechanisms,
  reporting and publication
• Evaluation Process
• Specification of scope, key questions,
  information sources and timing
• Independency of evaluators
• Quality of reports
• Good practice of evaluation
• Conclusions and recommendations based on findings

8
Legal Obligation for Evaluating Rural Development
Programmes

• Rural Development Regulation (EC) 1698/2005
• Art. 84.1 Rural development programmes shall
  be subject to ex ante, mid-term and ex-post
  evaluation
• Art. 86.1 Member States shall establish a
  system of ongoing evaluation for each rural
  development programme
• Art. 86.4 A summary of mid-term evaluation
  reports shall be undertaken on the initiative of
  the Commission
• Art. 87.1 A summary of ex-post evaluations
  shall be made, under the responsibility of the
  Commission, in cooperation with Member States and
  the Managing Authority

9
Steps of Evaluation
Period 1
Policy Application
Mid-term Evaluation
Ex-ante Evaluation
Ex-post Evaluation
SWOT
Programming Period 2
Policy Application
Mid-term Evaluation
Ex-ante Evaluation
SWOT
10
How do we do Evaluation?
11
Evaluation Tasks

• Structuring
• Establish intervention logic
• Describe measures and create typology
• Define key terms of evaluation questions and
  indicators
• Observing
• Identify information sources and scope of case
  studies
• Create information tools (questionnaires, data
  bases etc.)
• Collect information
• Analysing
• Carry out quantitative and qualitative assessment
• Answer all evaluation questions
• Judging
• Draft conclusions and recommendations

12
The Intervention Logic
Hierarchy of Effects
Needs
Hierarchy of Objectives
Impacts
Overall objectives
Results
Specific objectives
Outputs
Operational objectives
Inputs
13
Standard-Model of an Intervention Logic
14
Indicators reflecting the Intervention Logic an
Example
15
Evaluating impact on the environment
16
Environmental Integration Requirement of the
Treaty

• Treaty of Amsterdam
• Article 3c
• Environmental protection requirements must be
  integrated into the definition and implementation
  of the Community policies and activities referred
  to in Article 3, in particular with a view to
  promoting sustainable development.

17
Evaluating impacts on the environment

• Applicable for all CAP measures not only the ones
  with an environmental objective
• Distinguish between effects and impacts of the
  sector and the policy
• Evaluate the impact of a certain policy on
  behaviour and management practices of farmers
• Identify the impact on the environment based on
  the changed farming practices

18
Methodological Challenges

• Difficulty to establish a clear intervention
  logic for environmental impacts
• Difficulties regarding the data as
• Impacts are often depending on site-specific
  circumstances (soil, temperature, rainfall etc.)
• Impacts might take long time to emerge
• Impacts might be depending on other intervening
  factors (national/ regional policies,
  implementation mechanism etc.)
• Data collection is complex and time consuming

19
Evaluations undertaken or ongoing

• Related to the environment
• Agri-environment measures
• Cross Compliance
• Environmental impact of the CMOs for arable crops
• Environmental impact of the CMOs for beef and
  milk
• Environmental impact of the milk quota system
• Environmental impact of the CMO for cotton

20
Achievements of Evaluation

• Assessment of quality, efficiency, and
  effectiveness of measures
• Identification and appraisal of needs for change
• Assessment of strategic orientation for policies
  and programmes
• Better focus of programming on needs
• Compliance with legal requirements to evaluate
• Accountancy of public money