Public Service Productivity Measurement:

1
Public Service Productivity Measurement Macro or
Micro? Aileen Simkins, Department of
Health Co-Director of the Atkinson Review
2
What I will Talk About

• Some concepts
• Macro productivity measurement the Atkinson
  Review (and Gershon)
• Micro productivity measurement some NHS examples
• Challenges for the future

3
Productivity What do we mean?
Technical efficiency allocative efficiency value
for money effectiveness economy cuts
Increase in output divided by increase in
volume of input
Getting more for the same, or the same for less
Improving services while money is tight
4
Measuring Productivity Who By, What for?
Do you get what you pay for? Accountability to
the tax payer
The XX industry is inefficient by international
standards Evidence of use of resources
Best of breed have 3 times more throughput than
bottom quartile Benchmarking, competitive
pressures
Just try harder Performance management
Cant we get another one in? - Local drivers to
expand/ improve
5
Some key questions

• Can Government measures of public service
  productivity be valid?
• Is measuring productivity the same as measuring
  performance?
• Improving productivity matters is measurement a
  side alley?
• Does measurement of efficiency create
  distortions?

6
The Atkinson Review

• How to use National Accounts
• to measure public service productivity
• and
• How to do it better

7
How National Accounts Measure Public Services

• Traditionally, output input
• From 1997, System for National Accounts changed
  measure outputs via activities
• UK early implementer health (cost weighted
  activity index), schools (pupil days and quality
  adjustment), social services, social security
  admin
• ONS began publishing productivity articles

8
ONS Output Measure, 2003
9
Better Measures for the National Accounts

• Queries on basis of public service measures
• National Statistician set up Review Dec 2003,
  chaired Sir Tony Atkinson
• Measurement of Government Output and
  Productivity for the National Accounts

10
Atkinson Report Jan 2005

• 9 principles
• output should be measured in a way that is
  adjusted for quality, taking account of the
  attributable incremental contribution of the
  service to the outcome
• Specific recommendations for improvement to
  measures used in NA
• Encouraged ONS productivity articles
• UK Centre for Measurement of Government Activity

11
ONS Health Productivity Oct 2004
12
DH Press Release Oct 04

• John Reid (Secretary of State for Health) says
• it is absurd to measure NHS output without
  taking account of quality

13
Quality as part of NHS Output

• How many domains of quality?
• Health gain
• Patient experience
• What can we measure?
• How can we link quality measures to the NHS
  output index?
• How should we weight different aspects of
  quality?
• How valid is a partial story?

14
Accounting for Quality Change

• Average over last 5 years
• Value of health 1.5
• Value weight for statins 0.81
• York/NIESR adjustment 0.17
• Patient experience 0.07
• Blood pressure control 0.05
• Heart attack survival 0.01
• Total 2.68
• Quality adjusted output growth 6.29

15
ONS Health Productivity 2006
16
Education and Other Areas

• DfES action following Atkinson new use of pupil
  attainment tests and GCSE grades
• DfES article, ONS education productivity article
• Controversial issues grade drift, real
  earnings effect
• Also productivity articles for adult social care,
  social security administration

17
(No Transcript)
18
Value and Validity of Atkinson-based Productivity
Measures

• Designed to compare total outputs with total
  inputs
• Focus on attributable impact on outcomes and
  quality change
• Data incomplete biased towards areas of
  attention / improvement analysis by Depts
• Major developmental issues on techniques
• ONS consultation document Sept 2006

19
Performance Productivity?

• A perfect Atkinson measure captures all
  activities, all contributions to outcomes
• Is this the same as measuring organisation
  performance?
• If not choices about key priorities, value
  judgements
• Should those values inform the aggregate
  productivity measure?

20
Another macro measure Gershon

• Gershon includes savings from better procurement,
  Atkinson doesnt
• Specific changes in use of inputs valued as m
• Better use of productive time evidence from
  outputs and outcomes?

21
Gershon and Atkinson

• Gershon efficiency improvements are managed, not
  just measured
• Strong delivery support for Gershon, changes
  evident
• Measured to deliver target? NAO scrutiny
• Will measured productivity gains match? HMT,
  OGC and UKCeMGA should be able to explain

22
Micro Measures NHS

• NHS Institute for Innovation and Improvement,
  with DH
• Developing and issuing key metrics for health
  authorities, PCTs, Trusts, FTs
• Systematic focus on key areas for improvement
  volume, costs, variation
• Clinical engagement, benchmarking

23
First Metrics 22 Sept 2006

• Pdfs and web based tool
• Finance, workforce, procurement
• Clinical productivity
• Potential bed days saved through reduced length
  of stay
• Day case rate
• Reduce pre-operative bed days

24
Reduce wasted bed days
390,000 bed days
100m
25
Benchmarking

• Evidence based assessment of performance
• Addressing variations and helping outliers
• Providing a tool kit
• Opportunity to test performance against peers
• Stimulus to learn how to improve practice
• Recognition of success and challenge to do better

26
XXX NHS Trust
27
Value and Power of Micro Productivity Indicators

• Practical engagement in what to change
• Service based, clinical focus
• Small steps on long journey
• Risks that change happens, savings dont
• Check for unintended perverse incentives
• Tipping points for cutting overheads

28
Impact of Micro Changes on Macro Measure

• NHS III says
• 1 Reduce Avoidable Emergency Admissions
• 2 Reduce Unnecessary Outpatient Appointments,
  Follows-up, and DNAs
• 3 Avoid Unnecessary Procedures
• 4 Improve Day Case Performance
• 5 Reduce Wasted Bed Days
• 6 Accurate Clinical Coding
• 7 Reduce Variation in Length of Stay
• 8 Improve Staff Productivity
• 9 Actively Manage Staff Costs

29
If NHS follows NHS III advice..

• Lower growth in some hospital activities
• Shifts in unit costs
• Shifts in NHS spending to primary care,
  community, public health (harder to count)
• Probably, slower growth in output
• Measurable change in quality, outcomes?
• Productivity measure?

30
Measures, not Targets

• Macro and micro productivity measures both need
  careful interpretation
• Micro are more useful for managers, front line,
  actionability
• Macro should help with do you get what you pay
  for but may mislead
• Use of any efficiency measures changes behaviours
  intended, maybe also unintended

31
Lessons of History

• For the first decade of its existence
  (1982-92) the HCHS efficiency index received
  little adverse comment either from the NHS or
  policy analysts. However, once it was converted
  from a retrospective analytical exercise into a
  management tool it attracted a barrage of
  criticism.attaching rewards to counts of
  particular activities led to reporting drift,
  reclassification and at worst statistical
  dishonesty
• Clive Smee Speaking Truth to Power 2005

32
Dilemma Measuring with Right Impact

• Productivity is Govt business accountability
• Many drivers in public service to improve
  productivity
• Accountability needs metrics
• Metrics are almost always partial, may mislead or
  have unintended incentives
• Choice of metrics affects what is delivered
• Moral use mix of macro and micro indicators,
  with intelligence transparency