Criminal justice evaluation in the UK - PowerPoint PPT Presentation

About This Presentation
Title:

Criminal justice evaluation in the UK

Description:

Chris Fox Dan Ellingworth * * * * * * * * Modern policy making Forward looking: Defining policy outcomes and taking a long term view Outward looking: Take ... – PowerPoint PPT presentation

Number of Views:198
Avg rating:3.0/5.0
Slides: 25
Provided by: DanElli
Category:

less

Transcript and Presenter's Notes

Title: Criminal justice evaluation in the UK


1
Criminal justice evaluation in the UK
  • Chris Fox
  • Dan Ellingworth

2
Evidence-based policy
3
Modern policy making
  • Forward looking Defining policy outcomes and
    taking a long term view
  • Outward looking Take account of national,
    European and international situation learning
    from experience of other countries recognising
    regional variations.
  • Innovative, flexible Questioning established
    ways of dealing with things, encouraging new and
    creative ideas, identifying and managing risk.
  • Joining up Joining up the work of different
    government departments ensuring that
    implementation is part of the policy process.
  • Inclusive Consulting those responsible for
    implementation and those affected by the policy
    carrying out an impact assessment
  • Evidence based Basing policy decisions and
    advice upon the best available evidence ensuring
    evidence is available in an accessible and
    meaningful form.
  • Evaluated Systematic evaluation of the
    effectiveness of policy.
  • Reviews Policy constantly reviewed to ensure it
    is dealing with problems it was designed to
    solve.
  • Lessons learned Learning from experience of what
    works and what does not
  • Cabinet Office (1999) Professional policy making
    for the twenty-first century London Cabinet
    Office

4
What is evidence-based policy?
  • Evidence-based policy (EBP). An approach that
  • helps people make well informed decisions about
    policies, programmes and projects by putting the
    best available evidence from research at the
    heart of policy development and implementation
    (Davies 1999 quoted in Davies 2004)
  • Contrasted with opinion-based policy which
  • relies heavily on either the selective use of
    evidence (e.g. on single studies irrespective of
    quality) or on the untested views of individuals
    or groups, often inspired by ideological
    standpoints, prejudices, or speculative
    conjecture. (Davies 2004)

5
Opinion-based policy
Taken from Chalmers, I. (2003) Campbell and
Cochrane the need for generosity of spirit and
mutual support Jerry Lee Lecture 3rd Annual
Campbell Colloquiem
6
Opinion based policy
Taken from Chalmers, I. (2003) Campbell and
Cochrane the need for generosity of spirit and
mutual support Jerry Lee Lecture 3rd Annual
Campbell Colloquiem
7
What factors influence policy?
Taken from Davies, P. (2004) Is evidence-based
government possible Jerry Lee Lecture 2004
8
How to do an evaluation
9
An evaluation framework
  • Should it work? (Theory of change)
  • What is the underlying theory of change which
    explains why the intervention will make an
    impact?
  • Can it work? (Implementation evaluation)
  • Has the project been properly implemented? What
    were the challenges to implementation and how
    were they overcome?
  • Does it work? (Impact evaluation)
  • What is the impact of the intervention?
  • Is it worth it? (Economic evaluation)
  • What are the resource implications of
    implementing the intervention and what benefits
    will it deliver?

Adapted from Haynes, B., 1999. BMJ 319652-653 (
11 September )
10
Overview of possible methods
Review of project literature Review of international research literature Interviews with key stakeholders Staff interviews Observation of key processes Capture / analysis of monitoring data Capture / analysis of case files or other administrative data Interviews with service users Surveys of those affected Economic analysis
Should it work?
Can it work?
Does it work?
Is it worth it?
11
Should it work? Theories of change
  • Sometimes programmes and projects
  • dont have clear aims or change their aims over
    time
  • are shoe horned into a set of funding criteria
    to access that funding
  • dont have clearly defined processes and
    structures
  • are implemented in a multi-agency context where
    different agencies will have different ideas
    about what the project is trying to achieve

12
Theories of change
  • What is the conceptual link from an
    intervention's inputs to the production of its
    outputs and, subsequently, to its impacts on
    society in terms of results and outcomes?
  • (United Kingdom Evaluation Society Glossary of
    evaluation terms http//www.evaluation.org.uk/Pub_
    library/Glossary.htm)

13
Scared straight
  • Started as US programme in 1970s
  • Target group At risk or delinquent young people
  • Prison visits including tour and confrontational
    meeting with serving prisoners
  • Graphic depiction of life in prison
  • TV documentary in US extols its virtues
  • Also used in US

14
Two possible theories of change
Visit to a Prison by juveniles First Hand Experience of Prison Life Exposure to Prison Life and Prisoners as Negative Role Models Frightens or Scares Juveniles Away from Crime Reduces Crime and Offending
or . . .
Visit to a Prison by juveniles First Hand Experience of Prison Life Exposure to Prison Life and Prisoners as Positive Role Models Stimulates or Attracts Juveniles Towards Crime Increases Crime and Offending
15
Can it work? Process evaluation
  • Identify whether the intervention has been
    implemented as intended
  • Explore how a service or policy is delivered and
    experienced in practice
  • Identify the mechanisms by which it can produce
    the desired effects, potential barriers and
    facilitators
  • Identify circumstances under which successful
    operation might be replicated

16
Scurvy
  • James Lind (1716 1794) Ships Surgeon
  • Lind selected 12 men, all suffering from scurvy
  • Divided them into six pairs, giving each group
    different additions to their basic diet.
  • Cider
  • Seawater
  • A mixture of garlic, mustard and horseradish
  • Spoonfuls of vinegar
  • Oranges
  • Lemons.
  • Those fed citrus fruits experienced a remarkable
    recovery
  • In 1753, he published 'A Treatise of the Scurvy
  • http//www.bbc.co.uk/history/historic_figures/lind
    _james.shtml

17
Does it work? Impact evaluation
Sherman et al. (1998) Preventing Crime. What
works, what doesnt, whats promising. National
Institute of Justice.
18
Measuring impact
  • Choice of impact measures is often limited by
    availability of data or resources to collect
    primary data.
  • In the UK, re-offending data can be accessed via
    the Police National Computer (PNC) or the
    Offender Index
  • For less tangible outcomes such as family
    reconciliation or increased self-confidence the
    only option is often expensive face-to-face
    interviews/surveys.

19
Finding a comparator
  • A comparator group or area may not be readily
    available.
  • While a programme or pilot area may be willing to
    cooperate with an evaluation an organisation that
    is not part of the evaluation but is being asked
    to provide data for a comparator area may be hard
    to persuade to cooperate.
  • A regional or national roll-out may limit the
    number of potential comparator areas

20
Throughputs
  • Programmes and projects often struggle to
    generate sufficient throughputs to allow for
    robust statistical analysis.
  • Set-up times are often under-estimated
  • The need for inter-agency cooperation is often
    under-estimated

21
Timescales
  • Often evaluation timescales preclude appropriate
    follow-up periods to identify the long-term
    impact of interventions.
  • The Home Office recommends that reconviction
    studies use a 2 year follow-up period.
  • By the time a cohort to study has been generated
    this will often require an evaluation period of 3
    4 years.

22
Systematic Reviews
  • Systematic reviews are overviews of the existing
    research literature on a topic
  • A comprehensive search of print, electronic, and
    unpublished sources is made.
  • Sources identified are screened to see if they
    are relevant.
  • The quality of the sources (the strength of the
    evidence) is then assessed.
  • Sometimes the findings from individual studies
    are pooled in a process called meta-analysis

23
Scared straight systematic review
  • The systematic review shows that
  • The program increases the percentage of the
    treatment group committing new offences anywere
    from 1 to 30. (365) (emphasis added)
  • Petrosino, A., Turpin-Petrosino, C., and
    Finckenauer, J. (2000) Well-Meaning Programs Can
    Have Harmful Effects! Lessons from Experiments of
    Programs Such as Scared Straight Crime
    Delinquency 46 354

24
Interested?
  • If so, Chris and Dan run a Third Year Module
    called What works in social and criminal policy
    that looks in more detail at how policy is made
    and the role of evaluation in supporting the
    policy-making process.
Write a Comment
User Comments (0)
About PowerShow.com