Results and Performance Accountabilty, Decision-making and Budgeting

1
Outcome BasedAccountability
The Fiscal Policy Studies Institute Santa Fe, New
Mexico
Websites raguide.orgresultsaccountability.com
Book - DVD Ordersamazon.comresultsleadership.org
2
SIMPLE COMMON SENSE PLAIN LANGUAGE MINIMUM
PAPER USEFUL
3
Outcome Based Accountabilityis made up of two
parts
4
Results and Performance Accountability
COMMON LANGUAGE COMMON SENSE COMMON GROUND
5
THE LANGUAGE TRAPToo many terms. Too few
definitions. Too little discipline
Benchmark
Result
Outcome
Modifiers Measurable Core
Urgent Qualitative Priority
Programmatic Targeted
Performance Incremental Strategic
Systemic
Indicator
Goal
Objective
Measure
Target
Core qualitative strategic objectives
Your made up jargon here
Measurable urgent systemic indicators
Lewis Carroll Center for Language Disorders
6
DEFINITIONS
Being Healthy - Staying Safe - Enjoying and
achievingMaking a Positive Contribution -
Economic Well-being
Safe Communities - Clean Environment
Rate of low-birthweight babies - Rate of
confirmed child abuse Percent 16 19 yr. olds
with 5 A-C GCSEs - Rate of volunteering Rate
of child poverty
Crime rate - Air quality index
1. How much did we do? 2. How well did
we do it? 3. Is anyone better off?
Customer Outcome
7
From Ends to Means
From Talk to Action
From Talk to Action
OUTCOME
ENDS
INDICATOR
PERFORMANCEMEASURE
MEANS
8
IS IT A OUTCOME, INDICATOR OR PERFORMANCE MEASURE?
1. Safe Community 2. Crime Rate 3. Average
Police Dept response time 4. An educated
workforce 5. Adult literacy rate 6. People have
living wage jobs and income 7. of people with
living wage jobs and income 8. of participants
in job training who get living wage jobs
OUTCOME
INDICATOR
PERF. MEASURE
OUTCOME
INDICATOR
OUTCOME
INDICATOR
PERF. MEASURE
9
Results Indicators Performance Measures in
Amharic, Cambodian, Laotian, Somali, Spanish,
Tigrigna, Vietnamese
10
(No Transcript)
11
Translation Guide/Rosetta StoneNot the Language
Police
Ideas
Group 1
Group 2
Group 3 etc.
1. A condition of well-being for children,
adults, families communities 2. 3. etc.
OUTCOME
RESULT
GOAL
12
POPULATIONACCOUNTABILITY

• For Whole Populationsin a Geographic Area

Fiscal Policy Studies Institute Santa Fe, New
Mexicowww.resultsaccountability.comwww.raguide.o
rg
13
Community Outcomesfor Christchurch, NZ

• 1. A Safe City
• 2. A City of Inclusive and Diverse Communities
• 3. A City of People who Value and Protect the
  Natural Environment
• 4. A Well-Governed City
• 5. A Prosperous City
• 6. A Healthy City
• 7. A City for Recreation, Fun and Creativity
• 8. City of Lifelong Learning
• 9. An Attractive and Well-Designed City

14
Outcomes forChildren, Families and CommunitiesA
Working List

• ? Healthy Births
• ? Healthy Children and Adults
• ? Children Ready for School
• ? Children Succeeding in School
• ? Young People Staying Out of Trouble
• ? Stable Families
• ? Families with Adequate Income
• ? Safe and Supportive Communities

15
Georgia Policy Councilfor Children and
Families RESULTS

• ? Healthy Children
• ? Children Ready for School
• ? Children Succeeding in School
• ? Strong Families
• ? Self Sufficient Families

16
Every Child Matters Children ActOutcomes for
Children and Young People
Being Healthy enjoying good physical and mental
health and living a healthy lifestyle. Staying
Safe being protected from harm and neglect and
growing up able to look after themselves. Enjoying
and Achieving getting the most out of life and
developing broad skills for adulthood. Making a
Positive Contribution to the community and to
society and not engaging in anti-social or
offending behaviour. Economic Well-being
overcoming socio-economic disadvantages to
achieve their full potential in life.
17
Our Health, Our Care, Our Say White
PaperOutcomes for Adults

• 1. Health Emotional Well-being
• 2. Quality of life
• 3. Making a positive contribution
• 4. Exercising choice control
• 5. Freedom from discrimination harassment
• 6. Economic well being
• 7. Personal dignity respect

Note that 8. Effective leadership and (9.
Effective commissioning are means and not ends
Source A New Outcomes Framework for Performance
Assessment of Adult Social Care 2006 - 07
18
REPORT CARDS
Georgia
Santa Cruz, CA
Dayton, OH
Lehigh Valley, PA
19
Country
Neighborhood
Kruidenbuurt Tilburg, Netherlands
City
New Zealand

Portsmouth, UK
20
(No Transcript)
21
Leaking Roof(Outcomes thinking in everyday life)
Experience
Inches of WaterBASELINE
Not OK
Measure
Turning the Curve
Story behind the baseline (causes)
Partners
What Works
Action Plan 2
Action Plan
22
Outcomes
23
(No Transcript)
24
Criteria forChoosing Indicatorsas Primary vs.
Secondary Measures
Communication Power
Does the indicator communicate to a broad range
of audiences?
Proxy Power
Does the indicator say something of central
importance about the outcome?
Does the indicator bring along the data HERD?
Data Power
Quality data available on a timely basis.
25
Choosing IndicatorsWorksheet
Safe Community
Outcome _______________________________
ProxyPower
DataPower
CommunicationPower
Candidate Indicators
Measure 1 Measure 2 Measure 3 Measure 4 Measure
5 Measure 6 Measure 7 Measure 8
H M L
H M L
H M L
H
H
H
H
L
H
DataDevelopmentAgenda
26
Three Part Indicator List for each Outcome
Part 1 Primary Indicators
? 2 or 3 or 4 Headline Indicators? What this
outcome means to the community? Meets the
Public Square Test
Part 2 Secondary Indicators
? Everything else thats any good (Nothing is
wasted.)? Used later in the Story behind the
Curve
Part 3 Data Development Agenda
? New data? Data in need of repair
(quality,timeliness etc.)
27
The Matter of Baselines
OK?
Point to Point
Turning the Curve
History
Forecast
Baselines have two parts history and forecast
28
The Cost of Bad Outcomes
The costs of remediating problems after they occur
300 billion
InvestmentTrack
Cost
Invest in prevention to reduce or avoid out-year
costs.
29
COST OF BAD OUTCOMESUnited States 1970 to 2010
30
Newcastle, UK
Source Connexions Tyne and Wear, UK
Revised 9 Nov 2007
31
Paulsgrove, Portsmouth, UKYouth Nuisance

32
(No Transcript)
33
Rebound
34
(No Transcript)
35
(No Transcript)
36
Alternative to Traditional Evaluation
MethodsDEMONSTRATING a CONTRIBUTIONto complex
change efforts requires 3 elements
Given a Curve to Turn
c FPSI
37
If I include you,you will be my partner. If I
exclude you,you will be my judge.
- Rosell
38
Performance Accountability

• For Services, Agencies and Service Systems

Fiscal Policy Studies Institute Santa Fe, New
Mexicowww.resultsaccountability.comwww.raguide.o
rg
39
All Performance Measures that have ever
existed for any service in the history of the
universe involve answering two sets
of interlocking questions.
40
Program Performance Measures
Quantity
Quality
HowMuch did we do? ( )
HowWell did we do it? ( )
41
Performance Measures
Effort How hard did we try?
Effect Is anyone better off?
42
Program Performance Measures
43
Program Performance Measures
Quantity
Quality
How much service did we deliver?
How welldid we deliver it?
Effect Effort
Output Input
How much change / effect did we produce?
What quality of change / effect did we produce?
44
Program Performance Measures
Quantity
Quality
How much did we do?
How welldid we do it?
Effect Effort
Is anyonebetter off?


45
Education
Quantity
Quality
How much did we do?
How well did we do it?
Student-teacherratio
Number ofstudents
Effect
Effort
Is anyone better off?
Number of 16 olds with 5 A to CGCSEs Number of
fixedterm exclusions
Percent of 16 yr olds with 5 A to CGCSEs Rate
of fixedterm exclusions
46
Education
Quantity
Quality
How much did we do?
How well did we do it?
Student-teacherratio
Number ofstudents
Effect
Effort
Is anyone better off?
Number of 16 olds with 5 A to CGCSEs Number
with goodschool attendance
Percent of 16 yr olds with 5 A to
CGCSEs Percent with goodschool attendance
47
Pediatric Practice
Quantity
Quality
How much did we do?
How well did we do it?
Percent ofpatients treatedin less than1 hour
Number ofpatientstreated
Effect
Effort
Is anyone better off?
48
Drug/Alcohol Treatment Program
Quantity
Quality
How much did we do?
How well did we do it?
Percent ofstaff withtraining/certification
Number ofpersonstreated
Effect
Effort
Is anyone better off?
Number of clientsoff of alcohol drugs - at
exit - 12 months after exit
Percent of clientsoff of alcohol drugs -
at exit - 12 months after exit
49
Home Care Services
How much did we do?
How well did we do it?
visits per week Ave length of visit service
users who say the service is reliable, flexible,
respectful, friendly (SPRU))
of persons served visits
Is anyone better off?
Rate entry into hospital, residential or nursing
care carers report the service helped the
service user remain at home
entry into hospital, residential or nursing
care carers report the service helped the
service user remain at home
50
Fire Department
Quantity
Quality
How much did we do?
How well did we do it?
ResponseTime
Number ofresponses
Effect
Effort
Is anyone better off?
of fireskept toroom of origin
of fireskept toroom of origin
51
General Motors
Quantity
Quality
How much did we do?
How well did we do it?
Employees pervehicleproduced
of production hrs tons of steel
Effect
Effort
Is anyone better off?
of cars sold Amount of Profit Car value
after 2 years
Market share Profit per share Car value after
2 years
Source USA Today 9/28/98
52
Not All Performance Measures Are Created Equal
Quality
Quantity
Quality
How much did we do?
How well did we do it?
LeastImportant
Least
AlsoVery Important
Effect
Effort
Is anyone better off?
MostImportant
Most
53
RBA Categories Account for All Performance
Measures(in the history of the universe)
TQM
Efficiency, Admin overhead, Unit costStaffing
ratios, Staff turnoverStaff morale, Access,
Waiting time, Waiting lists, Worker safety
Efficiency
Cost
Quantity
Quality
Process Input
Effort
Customer Satisfaction(quality service delivery
customer benefit)
Product Output Impact
Effect
Cost / Benefit ratioReturn on investment Client
results or client outcomes
Benefit value
EffectivenessValue addedProductivity
Effectiveness
54
RBA Categories Account for All Performance
Measures(in the history of the universe)
TQM
Efficiency, Admin overhead, Unit costStaffing
ratios, Staff turnoverStaff morale, Access,
Waiting time, Waiting lists, Worker safety
Cost
Quantity
Quality
Process Input
Effort
Customer Satisfaction(quality service delivery
customer benefit)
Product Output Impact
Effect
Cost / Benefit ratioReturn on investment Client
results or client outcomes
Benefit value
EffectivenessValue addedProductivity
55
RBA Categories Account for All Performance
Measures(in the history of the universe)
TQM
Efficiency, Admin overhead, Unit costStaffing
ratios, Staff turnoverStaff morale, Access,
Waiting time, Waiting lists, Worker safety
Cost
Quantity
Quality
Process Input
Effort

Customer Satisfaction(quality service delivery
customer benefit)
Product Output Impact
Effect
Cost / Benefit ratioReturn on investmentClient
results or client outcomes
Benefit value
EffectivenessValue addedProductivity
56
RBA Categories Account for All Performance
Measures(in the history of the universe)
TQM
Efficiency, Admin overhead, Unit costStaffing
ratios, Staff turnoverStaff morale, Access,
Waiting time, Waiting lists, Worker safety
Cost
Quantity
Quality
Process Input
Effort
Customer Satisfaction(quality service delivery
customer benefit)
Product Output Impact
Effect
Cost / Benefit ratioReturn on investment Client
results or client outcomes
Benefit value
EffectivenessValue addedProductivity
57
The Matter of Control
Quality
Quantity
How much did we do?
How well did we do it?
MostControl
Effect
Effort
Is anyone better off?
LeastControl
PARTNERSHIPS
58
The Matter of Use

1. First Purpose is to Improve Performance as
   a contribution to improving results

2. Avoid the Performance Measurement Equals
Punishment Trap ? Acknowledge the experience
as real. ? Work to create a healthy
organizational environment ? Start small. ?
Build bottom-up and top-down simultaneously.
59
Comparing Performance
1. To Ourselves First Can we do better than
our own history?
2. To Others When it is a fair
apples/apples comparison.
3. To Standards When we know what
good performance is.
60
Comparing Performance
1. To Ourselves First Can we do better than
our own history?
2. To Others When it is a fair
apples/apples comparison.
3. To Standards When we know what
good performance is.
61
Comparing Performance
1. To Ourselves First Can we do better than
our own history?
2. To Others When it is a fair
apples/apples comparison.
3. To Standards When we know what
good performance is.
62
Comparing Performance
1. To Ourselves First Can we do better than
our own history?
2. To Others When it is a fair
apples/apples comparison.
3. To Standards When we know what
good performance is.
63
The Matter of Standards
1. Quality of Effort Standards aresometimes WELL
ESTABLISHED
? Child care staffing ratios? Application
processing time? Handicap accessibility? Child
abuse response time
Quantity
Effort
BUT
Effect
2. Quality of Effect Standards arealmost always
EXPERIMENTAL
? Hospital recovery rates ? Employment placement
and retention rates? Recidivism rates
AND
3. Both require aLEVEL PLAYING FIELD and an
ESTABLISHED RECORDof what good performance is.
64
Advanced Baseline Display
Create targets only when they areFAIR USEFUL
Goal (line)
Target or Standard
Avoid publicly declaringtargets by year if
possible.
Your Baseline
InsteadCount anything better than baseline as
progress.
Comparison Baseline
65
Program Performance Measures
Quantity
Quality
How much did we do?
How welldid we do it?
Effect Effort
Is anyonebetter off?


66
All Data have two Incarnations
Lay Definition
Technical Definition
Graduation Rate
enrolled June 1 who graduate June 15
enrolled Sept 30 who graduate June 15
enrolled 9th grade who graduate in 12th grade
67
Separating the Wheat from the ChaffTypes of
Measures Found in Each Quadrant
How much did we do?
How well did we do it?
Clients/customers served
Activities (by type of activity)
Is anyone better off?
68
Choosing Headline Measures and the Data
Development Agenda
Quantity
Quality
How much did we do?
How well did we do it?
3 DDA
Measure 1 ----------------------------
Measure 2 ----------------------------
Measure 3 ----------------------------
Measure 4 ----------------------------
Measure 5 ----------------------------
Measure 6 ----------------------------
Measure 7 ----------------------------
Measure 8 ----------------------------
Measure 9 -----------------------------
Measure 10 ---------------------------
Measure 11 ---------------------------
Measure 12 ---------------------------
Measure 13 ---------------------------
Measure 14 ---------------------------
2 Headline
Effect
Effort
Is anyone better off?
2 DDA
Measure 15 ----------------------------
Measure 16 ----------------------------
Measure 17 ----------------------------
Measure 18 ----------------------------
Measure 19 ----------------------------
Measure 20 ----------------------------
Measure 21 ----------------------------
Measure 15 ----------------------------
Measure 16 ----------------------------
Measure 17 ----------------------------
Measure 18 ----------------------------
Measure 19 ----------------------------
Measure 20 ----------------------------
Measure 21 ----------------------------
3 Headline
1 Headline
1 DDA
69
Select 3 to 5 Performance Measuresat each level
of the organization
3 - 5
3 - 5
3 - 5
3 - 5
?
Be disciplined about whats most important. Dont
get distracted.
Get over it!
Pick the 3 5 most important of the 9 15
measures or create composites.
3 - 5
3 - 5
3 - 5
70
Customer Outcomes
71
(No Transcript)
72
(No Transcript)
73
(No Transcript)
74
How Population Performance AccountabilityFIT
TOGETHER
75
THE LINKAGE Between POPULATION and PERFORMANCE
POPULATIONOUTCOMES
Contributionrelationship
Alignmentof measures
Child Protection Service
ofinvestigationscompleted
initiatedwithin 24 hrsof report
Appropriateresponsibility
repeatAbuse/Neglect
repeatAbuse/Neglect
CUSTOMEROUTCOMES
76
THE LINKAGE Between POPULATION and PERFORMANCE
POPULATIONOUTCOMES
Contributionrelationship
Alignmentof measures
Job Training Program
Unit costper persontrained
personsreceivingtraining
Appropriateresponsibility
who getliving wage jobs
who getliving wage jobs
CUSTOMEROUTCOMES
77
(No Transcript)
78
Population Accountability
Outcome to which you contribute to most directly.
Indicators
Story
Every timeyou make apresentation, use
atwo-partapproach
Partners
What would it take?
Your Role as part of a larger strategy.
Your Role
Performance Accountability
Service
Performance measures
Story
Partners
Action plan to get better
79
Population Accountability
Result to which you contribute to most directly.
Indicators
Story
Every timeyou make apresentation, use
atwo-partformat
Partners
What would it take?
Your Role within the larger strategy.
Your Role
Performance Accountability
Service
Performance measures
Story
Partners
Action plan to get better
80
Different Kinds of Progress
1. Data a. Population indicators Actual
turned curves movement for the
better away from the baseline.
b. Service performance measures
customer progress and better service
How much did we do?
How well did we do it? Is
anyone better off?
2. Accomplishments Positive activities, not
included above.
3. Anecdotes Stories behind the statistics that
show how individuals are better off.
81
(No Transcript)
82
(No Transcript)
83
(No Transcript)
84
Logic Model
85
Board of Directors MeetingAGENDA
1. New data
1. New data
2. New story behind the curves
2. New story behind the curves
3. New partners
3. New partners
4. New information on what works.
4. New information on what works.
5. New information on financing
5. New information on financing
6. Changes to action plan and budget
6. Changes to action plan and budget
7. Adjourn
7. Adjourn
86
Whats Next?A Basic Action Plan for Outcome
Based Accountability
TRACK 1 POPULATION ACCOUNTABILITY
? Establish outcomes? Establish indicators,
baselines and charts on the wall? Create an
indicators report card? Set tables (action
groups) to turn curves
TRACK 2 PERFORMANCE ACCOUNTABILITY
? Performance measures, and charts on the wall
for programs, agencies and service systems?
Use 7 Questions program by program in management,
budgeting and strategic planning
87
IN CLOSING
88
If you do what you always did,you will get
what you always got.
Kenneth W. JenkinsPresident, Yonkers NY NAACP
89
THANK YOU !
Websites raguide.orgresultsaccountability.com
------Book - DVD Orders------amazon.comresultsle
adership.org
90
EXERCISES
Fiscal Policy Studies Institute Santa Fe, New
Mexicowww.resultsaccountability.comwww.raguide.o
rg
91
(No Transcript)
92
(No Transcript)
93
(No Transcript)
94
(No Transcript)
95
(No Transcript)
96
Turn the Curve Exercise Population Well-being
5 min Starting Points - timekeeper and
reporter - geographic area - two hats (yours
plus partners)
10 min Baseline - pick an outcome, and an
indicator curve to turn - forecast OK or not
OK?
15 min Story behind the baseline -
causes/forces at work - information research
agenda part 1 - causes
Two pointers to action
15 min What works? (What would it take?) -
what could work to do better? - each partners
contribution - no-cost / low-cost ideas -
information research agenda part 2 what works
10 min Report convert notes to one page
97
4. --------- Off the Wall
ONE PAGE Turn the Curve Report Population
Outcome _______________
Indicator(Lay Definition)
IndicatorBaseline
Story behind the baseline
---------------------------
--------------------------- (List as many as
needed)
Partners ---------------------------
--------------------------- (List as many
as needed)
Three Best Ideas What Works 1.
--------------------------- 2.
--------------------------- 3.
---------No-cost / low-cost
SharpEdges
4. --------- Off the Wall
98
The first step in performance accountability is
toDRAW A FENCEAround something that has
ORGANIZATIONAL OR FUNCTIONAL IDENTITY
99
Creating a Working Baselinefrom Group Knowledge
Graduation Rate
Not OK?
Forecasting
Backcasting
100
Turn the Curve Exercise Program Performance
5 min Starting Points - timekeeper and
reporter - identify a program to work on -
two hats (yours plus partners)
10 min Performance measure baseline - choose 1
measure to work on from the lower right
quadrant - forecast OK or not OK?
15 min Story behind the baseline -
causes/forces at work - information research
agenda part 1 - causes
Two pointers to action
15 min What works? (What would it take?) -
what could work to do better? - each partners
contribution - no-cost / low-cost ideas -
information research agenda part 2 what works
10 min Report Convert notes to one page
101
4. --------- Off the Wall
ONE PAGE Turn the Curve Report Performance
Service _______________
Performance Measure (Lay definition)
PerformanceMeasureBaseline
Story behind the baseline
---------------------------
--------------------------- (List as many as
needed)
Partners ---------------------------
--------------------------- (List as many
as needed)
Three Best Ideas What Works 1.
--------------------------- 2.
--------------------------- 3.
---------No-cost / low-cost
SharpEdges
4. --------- Off the Wall
102
THANK YOU !
103
20 MinuteExercise
2 customers
1 measure
1 measure
Discuss how were doing
2 partners
2 what works ideas (1 nc/lc)
Discuss how to implement
40
104
20 MinuteExercise
POPULATION ACCOUNTABILITY
1. What population are we concerned about?
(e.g. U.S. population)
1 population
2. What condition(s) do we want for this
population? (Americans are free of cancer.)
1 result
1 indicator
3. How could we measure these conditions? (
(e.g. cancer rates)
Discuss how were doing
4. How are we doing on the most important of
these measures? (baseline history)
5. Who are the partners with a role to play
in doing better?
2 partners
2 what works ideas (1 nc/lc)
6. What works what would it take to do
better? (What is our role?)
7. What do we propose to do?
Discuss how to implement
105
Some datafor the Turn the CurveExercises
106
(No Transcript)
107
(No Transcript)
108
(No Transcript)
109
(No Transcript)
110
(No Transcript)