Monitoring

1
Monitoring Evaluation in Police Capacity
Building Operations
Jodie Curth Canberra, July 2008
2
All models are wrong, some models are
useful George Box Quoted in P Morgan (2006)
3
Introduction What are we doing?

• The intention of this research is to identify
  appropriate approaches to the monitoring and
  evaluation (ME) of international police capacity
  building operations of the Australian Federal
  Police, International Deployment Group.
• The intention of this presentation is to
  highlight some of the basic criticisms of
  Conventional Monitoring Evaluation (CME) in
  the interests of generating a discussion with you
  about a way forward.

Jodie Curth
4
Whats out there?

• After conducting a preliminary literature review
  it is clear that the area of ME is extremely
  underdeveloped particularly in the case of
• Security Sector Reform (SSR)
• Police Reform and
• Police Capacity Building.
• ME in capacity building generally however has
  received significantly more attention in recent
  years.
• Initial focus on lessons learnt within the
  development community.

Jodie Curth
5
The major issues - so far

• Capacity Building operations are notoriously
  difficult to measure.
• A crucial factor in the sustainability of police
  capacity building is in fact the ME process
  itself.
• Poorly designed indicators can potentially
  undermine the process of capacity building -
  reduced local ownership, negatively impact upon
  sustainable capacity, distort behaviour and
  divert resources this is particularly a problem
  in the building of sustainable peace and
  capacity.
• A lot of the problems associated with defining
  success in capacity building stem from the
  problems associated with defining capacity in the
  first place.

Jodie Curth
6
Capacity

• We have identified two broad understandings of
  capacity
• Technical Capacity
• Institutional / Organisational Capacity

Jodie Curth
7
Organisational Capacity

• Capacity development is taken to mean the
  growth of formal organizational sic
  relationships and abilities ie those changes in
  organizational behaviour, values, skills and
  relationships that lead to the improved abilities
  of groups and organizations to carry out
  functions and achieve desired outcomes over time
  Capacity development can thus refer to either
  process or outcomes ie those efforts to improve
  organizational performance and/or results of
  those efforts in terms of capacities developed.
• Peter Morgan
• The European Centre for Development Policy
  Management (ECDPM) further demonstrates the
  complexities in stating
• Capacity is that emergent combination of
  attributes, assets, capabilities and
  relationships that enables human systems to
  perform, survive and self-renew.

Jodie Curth
8
Results Based Management (RBM)

• RBM is a corporate based method of strategic
  performance management. It is focused on
  streamlining every tangible aspect of capacity
  building. There is a heavy emphasis on
  accountability. In this approach all aspects of
  resource allocation and funding, process and
  action, reporting and accountability should
  directly relate to the desired outcomes as
  defined by the donor. In reality, however, the
  process is only accountable to the output
  stage.
• Preoccupied with demonstrating success.

Jodie Curth
9
Example RBM table
Jodie Curth
10
Criticisms of RBM

• Reduced local ownership
• Reliance of specific problem diagnosis
• Divert resources
• Distort behaviour
• Short term goal orientation
• Preoccupation with hard indicators
• Documentary realities
• Linguistic building blocks
• Reduced institutional knowledge and continuity

Jodie Curth
11
Reduced local ownership

• The interventionist model of external diagnosis,
  gap filling and accountability reduces the
  sense of ownership for local participants - due
  to a lack of comprehensive local input from the
  outset.
• This is also considered a problem in the building
  of capacity itself - participatory learning is
  reduced.
• Lack of mutual understanding and limited
  agreement on what success might look like.

Jodie Curth
12
Short term goal orientation

• In the interests of demonstrating success
  resources are diverted toward areas of quick
  realisation with quantifiable outcomes.
• The result is a reduced focus on soft less
  tangible aspects of capacity building.
• The artificial construct of time-blocks and
  reporting periods can also motivate short-term
  goal orientation.
• In country partner personnel may be sidelined.

Jodie Curth
13
Utilising hard indicators

• Preoccupation with hard indicators (micro
  aspects of the organisation, rather than the
  whole)
• This relates to criticisms of compartmentalising
  the organisation rather than treating it as a
  whole.
• The organisation broken down into discrete units
  and treated in isolation, potentially
  undermining the whole.
• The treatment of the organisation as something
  separate from the broader society.

Jodie Curth
14
Documentary realities Linguistic building
blocks

• Documentary realities (Atkinson Coffey)
• Do not reflect whats happening on the ground
• Reporting creates a paper chain documents refer
  to each other
• Linguistic building blocks (Atkinson Coffey)
• Management jargon creating a barrier to
  understanding prompting resistance failure to
  report

Jodie Curth
15
Reduced institutional knowledge

• Availability of external experts causes
  considerable problems in SSR ME processes
• Interruption of ME disjointed and ad hoc
• If locals participate in conducting the ME this
  increases institutional knowledge and provides a
  continuity

Jodie Curth
16
What are we trying to achieve?

• Overcome the problems inherent in CME practices
  by incorporating critical systems thinking and
  participatory approaches
• Achieve a practical framework appropriate in all
  types of police capacity building environments
  with a particular focus on the complex issue of
  organisational capacity building.

Jodie Curth
17
Where to from here?

• Further research
• Gain a comprehensive understanding of the IDGs
  ME practices
• Identify the problems with existing frameworks in
  consultation with IDG practitioners
• Comparative analysis between IDG and various
  international practices
• Identify expectations and priorities of key
  partners ie. AusAID
• Identify reporting requirements including PBS,
  Annual Reports etc.
• Develop draft approaches to ME in police
  capacity building and apply these findings to the
  work of the IDG regional focus (eg. Solomon
  Islands, Timor)

Jodie Curth