ResultsBased Accountability

1
Results-Based Accountability

• Discovery Communities TA Institute
• April 1, 2008

2
Results Accountability Decision-makingand
Strategic Planning
Fiscal Policy Studies Institute Santa Fe, New
Mexico
WEBSITES www.resultsaccountability.com
www.raguide.org www.charteroakgroup.com
BOOK ORDERS www.trafford.com
www.amazon.com
3

• Why Are We Here?

3
4
SIMPLE COMMON SENSE PLAIN LANGUAGE MINIMUM
PAPER USEFUL
5
Concepts to Take Away Today

• Population accountability v. performance
  accountability
• Ends v. means
• How to choose indicators and measures
• The importance of a data development agenda
• How to move from talk to action

6
Results Accountabilityis made up of two parts
7
Results and Performance Accountability
COMMON LANGUAGE COMMON SENSE COMMON GROUND
7
8
THE LANGUAGE TRAPToo many terms. Too few
definitions. Too little discipline
Benchmark
Result
Outcome
Modifiers Measurable Core
Urgent Qualitative Priority
Programmatic Targeted
Performance Incremental Strategic
Systemic
Indicator
Goal
Objective
Measure
Target
Lewis Carroll Center for Language Disorders
9
The Humpty Dumpty Approach to Language

• When I use a word, Humpty Dumpty said, in
  rather a scornful tone, it means just what I
  choose it to meanneither more nor less. The
  question is, said Alice, whether you can make
  words mean so many different things. The
  question is, said Humpty Dumpty, which is to be
  masterthats all.
• LEWIS CARROLL, Through the Looking-Glass, chapter
  6, p. 205 (1934). First published in 1872.

10
DEFINITIONS
RESULT
A condition of well-being for children, adults,
families or communities.
Children born healthy, Children succeeding in
school, Safe communities, Clean Environment,
Prosperous Economy
INDICATOR
A measure which helps quantify the achievement
of a result.
Rate of low-birthweight babies, Rate of high
school graduation, crime rate, air quality
index, unemployment rate
PERFORMANCE MEASURE
A measure of how well a program, agency or
service system is working. Three
types
1. How much did we do? 2. How well did
we do it? 3. Is anyone better off?
Customer Outcomes
11
From Ends to Means
From Talk to Action
RESULT
ENDS
INDICATOR
PERFORMANCEMEASURE
MEANS
12
Connecticut Glossary of RBA Terms

• The Appropriations Committee standardized the
  terms we use in Connecticut. You have a copy of
  the glossary.
• We could have chosen other terms consistent
  with Friedmans RBA approach so this list is
  somewhat arbitrary.
• Its virtue is that everyone in Connecticut
  doing this work both executive branch and
  legislative branch is using the same vocabulary
  and meaning the same thing by it. At least we
  should be able to understand each other.

13
Some Connecticut Early Childhood Indicators

• Infants born at low birth weight
• Births to mothers without a high school degree
• of kindergartners with all or most
  pre-literacy and personal skills
• 4th grade reading scores at mastery or above

14
IS IT A RESULT, INDICATOR OR PERFORMANCE MEASURE?

• 1. Safe Community
• 2. Crime Rate
• 3. Average Police Dept response time
• 4. A community without graffiti
• 5. of surveyed buildings without graffiti
• 6. People have living wage jobs and income
• 7. of people with living wage jobs and income
• 8. of participants in job training who get
  living wage jobs

15
IS IT A RESULT, INDICATOR OR PERFORMANCE MEASURE?

• 9. HS graduates enrolling in college
• 10. Traffic-related death rate
• 11. Clean environment
• 12. Air pollutants in parts per million
• 13. participating cities fixing treatment
  plants
• 14. All children eat healthy
• 15. students eligible for free lunch
  participating in free lunch

16
POPULATIONACCOUNTABILITY

• For Whole Populationsin a Geographic Area

17
MarylandResults for Child Well-Being

• ? Babies born healthy
• ? Healthy children
• ? Children enter school ready to learn
• ? Children successful in school
• ? Children safe in their families and communities
• ? Stable and economically independent families
• ? Communities that support family life

17
18
Connecticut Results Statements

• A clean and healthy Long Island Sound
• All children healthy and ready to learn by
  age 5
• All children ready by five and fine by nine

18
19
Leaking Roof(Results thinking in everyday life)
Experience
Not OK
Inches of Water
Measure
Turning the Curve
Story behind the baseline (causes)
Partners
What Works
Action Plan
20
Seven Population Accountability Questions

• What are the quality of life conditions we want
  for the children, adults and families who live in
  our community?
• What would these conditions look like if we could
  see them?
• How can we measure these conditions?
• How are we doing on the most important of these
  measures?
• Who are the partners that have a role to play in
  doing better?
• What works to do better, including no-cost and
  low-cost ideas?
• What do we propose to do?

20
21
We haventgot the money, so weve gotto
think. Lord Rutherford1871 - 1937
22
Connecticut Population Template
22
23
Criteria forChoosing Indicators as Primary vs.
Secondary Measures
Communication Power
Does the indicator communicate to a broad range
of audiences?
Proxy Power
Does the indicator say something of central
importance about the result?
Does the indicator bring along the data HERD?
Data Power
Quality data available on a timely basis.
24
Choosing IndicatorsWorksheet
Outcome or Result_______________________
Safe Community
ProxyPower
DataPower
CommunicationPower
Candidate Indicators
Measure 1 Measure 2 Measure 3 Measure 4 Measure
5 Measure 6 Measure 7 Measure 8
H M L
H M L
H M L
H
H
H
H
L
H
DataDevelopmentAgenda
25
Three Part Indicator List for each Result
Part 1 Primary Indicators
? 2 or 3 or 4 Headline Indicators? What this
result means to the community? Meets the
Public Square Test
Part 2 Secondary Indicators
? Everything else thats any good (Nothing is
wasted.)? Used later in the story behind the
baseline
Part 3 Data Development Agenda
? New data? Data in need of repair
(quality,timeliness etc.)
26
The Matter of Baselines
OK?
Point to Point
Turning the Curve
Forecast
History
Baselines have two parts history and forecast
27
Performance Accountability

• For Programs, Agencies and Service Systems

28
Results Accountabilityis made up of two parts
29
All Performance Measures that have ever
existed for any program in the history of the
universe involve answering two sets
of interlocking questions.
30
Program Performance Measures
Quantity
Quality
HowMuch did we do? ( )
HowWell did we do it? ( )
31
Program Performance Measures
Effort How hard did we try?
Effect Is anyone better off?
32
Program Performance Measures
33
Program Performance Measures
Quantity
Quality
How much service did we deliver?
How welldid we deliver it?
Effect Effort
Output Input
How much change / effect did we produce?
What quality of change / effect did we produce?
34
Program Performance Measures
Quality
Quantity
How much did we do?
How welldid we do it?
Effect Effort
Is anyonebetter off?


35
Education
Quantity
Quality
How much did we do?
How well did we do it?
Student-teacherratio
Number ofstudents
Effect
Effort
Is anyone better off?
Percent ofhigh schoolgraduates
Number ofhigh schoolgraduates
36
Education
Quantity
Quality
How much did we do?
How well did we do it?
Student-teacherratio
Number ofstudents
Effect
Effort
Is anyone better off?
37
Health Practice
Quality
Quantity
How well did we do it?
How much did we do?
Percent ofpatients treatedin less than1 hour
Number ofpatientstreated
Effect
Effort
Is anyone better off?
Incidence ofpreventabledisease(in the practice)
Rate ofpreventabledisease(in the practice)
38
General Motors
Quantity
Quality
How much did we do?
How well did we do it?
Employees pervehicleproduced
of production hrs tons of steel of cars
produced
Effect
Effort
Is anyone better off?
of cars sold amount of Profit car value
after 2 years
market share Profit per share car value after
2 years
Source USA Today 9/28/98
39
What Kind of PERFORMANCE MEASURE?

• ? of people served
• ? participants who got jobs
• ? staff turnover rate
• ? participants who got jobs
• ? of children reading at grade level
• ? cost per unit of service
• ? applications processed
• ? patients who fully recover

Upper Left
Lower Right
Upper Right
Lower Left
Lower Right
Upper Right
Upper Left
Lower Right
39
40
RBA Categories Account for All Performance
Measures(in the history of the universe)
TQM
Efficiency, Admin overhead, Unit costStaffing
ratios, Staff turnoverStaff morale, Access,
Waiting time, Waiting lists, Worker safety
Efficiency
Cost
Quantity
Quality
Process Input
Effort
Customer Satisfaction(quality service delivery
customer benefit)
Product Output Impact
Effect
Cost / Benefit ratioReturn on investment Client
results or client outcomes
Benefit value
EffectivenessValue-addedProductivity
Effectiveness
41
(No Transcript)
42
The Matter of Control
Quality
Quantity
How much did we do?
How well did we do it?
MostControl
Effect
Effort
Is anyone better off?
LeastControl
PARTNERSHIPS
43
The Matter of Use

• Fundamental Purpose is to Improve Performance
  as a contribution to improving results

2. Avoid the Performance Measurement Equals
Punishment Trap ? Acknowledge the experience
as real. ? Work to create a healthy
organizational environment ? Start small. ?
Build bottom-up and top-down simultaneously.
44
Program Performance Measures
Quality
Quantity
How welldid we do it?
How much did we do?
Effect Effort
Is anyonebetter off?


45
Connecticut Program Template
45
46
How Population Performance AccountabilityFIT
TOGETHER
47
THE LINKAGE Between POPULATION and PERFORMANCE
POPULATION ACCOUNTABILITY
Healthy Births Rate of low birth-weight
babiesStable Families Rate of child abuse
and neglectChildren Succeeding in School
Percent graduating from high school on time
POPULATIONRESULTS
Contributionrelationship
PERFORMANCE ACCOUNTABILITY
Alignmentof measures
Child Welfare Program
withMultiplePlacements
Foster ChildrenServed
Appropriateresponsibility
RepeatAbuse/Neglect
RepeatAbuse/Neglect
CUSTOMEROutcomes
48
THE LINKAGE Between POPULATION and PERFORMANCE
POPULATIONRESULTS
Contributionrelationship
Alignmentof measures
Job Training Program
Unit costper persontrained
personsreceivingtraining
Appropriateresponsibility
who getliving wage jobs
who getliving wage jobs
CUSTOMEROutcomes
49
(No Transcript)
50
What is Happening in the Legislature?

• Institutionalizing RBA in the legislature
• The Appropriations Committee has created a new
  RBA sub-committee
• Early Childhood Cabinet must develop an
  accountability plan and make other
  recommendations for changes necessary to ensure
  coordination, service integration, and
  accountability
• The 2007 budget requires new and expanded
  programs to report to the General Assembly and
  OPM using an approved RBA framework, an effort
  lead by the Office of Fiscal Analysis (OFA) and
  OPM
• The co-chairs and rankings members of the other
  sub-committees of Appropriations are identifying
  particular public policy issues (result
  statements) for RBA development
• All this month, the early childhood agencies that
  were part of the RBA Phase II initiative last
  session reported back to the legislature on the
  progress they have made

50
51
What is Happening at State Agencies and in
Communities?

• The Child Poverty and Prevention Council and
  the Juvenile Justice Policy and Operating
  Coordinating Committee are both using RBA to
  guide their implementation efforts
• Municipalities and non-profit groups, e.g.,
  workforce boards, are embracing RBA in areas
  beyond early childhood
• State agencies will be developing program
  quality and outcome measures in response to the
  new Appropriations Committee initiatives
• They will be passing the requirements along to
  their vendors

52
What is Happening at State Agencies and in
Communities?

• Organizations that contract with the State will
  have new data collection and reporting
  requirements, including outcome measures
• Managing performance will be more complicated
• Likely to be greater requirements for integration
  of programs and collaboration with other agencies
• Opportunity for contractors to have input on
  measurement
• Opportunity for program improvement

52
53
If you do what you always did,you will get
what you always got.
Kenneth W. JenkinsPresident, Yonkers NY NAACP
53
54
Thank You

• The Charter Oak Group, LLC
• www.charteroakgroup.com
• Ron Schack
• ron_schack_at_yahoo.com
• [phone]
• Bennett Pudlin
• bpudlin_at_charteroakgroup.com
• [phone]

55
Turn the Curve Exercise Population Well-being
5 min Starting Points - timekeeper and
reporter - two hats (yours plus partners)
5 min Baseline - forecast Where is the
trend line going? - turn the curve Is
forecast OK or not OK?
15 min Story behind the baseline -
causes/forces at work - information research
agenda part 1 - causes
Two pointers to action
15 min What works? (What would it take?) -
what could work to do better - each partners
contribution - no-cost / low-cost ideas -
information research agenda part 2 what works
10 min Report Convert notes to one page
56
ONE PAGE Turn the Curve Report
Result _______________
Indicator(Lay Definition)
Indicator Baseline
Story behind the baseline
---------------------------
--------------------------- (List as many as
needed)
Partners ---------------------------
--------------------------- (List as many
as needed)
Three Best Ideas What Works 1.
--------------------------- 2.
--------------------------- 3.
---------No-cost / low-cost
---------Off the Wall
SharpEdges
4.