Performance Budgeting and Management

1
Performance Budgeting and Performance Management
in the U.S. Government Lessons from the PART
Initiative
John Pfeiffer U.S. Office of Management and
Budget jpfeiffer_at_omb.eop.gov Presentation at a
World Bank Brown-Bag Seminar June 6, 2007
2
Overview

• Background
• PART Process
• PART Questions
• Challenges
• PARTWeb and ExpectMore.gov
• How OMB Manages the PART Initiative

3
What is the PART?

• The Program Assessment Rating Tool is a
  diagnostic tool used to assess program
  performance and to drive improvements.
• The PART is designed to provide a consistent
  approach to assessing and rating programs across
  the Federal government.
• PART assessments review overall program
  effectiveness, from design through implementation
  and results.
• Once completed, PART reviews help inform budget
  decisions and identify actions to improve
  results.
• Agencies are held accountable for implementing
  PART follow-up actions, i.e., improvement plans,
  for each program.

4
When We Began in 2002

• Many systems in place to collect and report data
• Unclear relationship between strategic and annual
  goals
• Tendency to measure what we could instead of what
  we should
• Uneven attention to performance measurement
• Lots of measures, but priorities not transparent
• Performance data used more for reporting than
  decision-making

5
Where We Are TodayDistribution of Cumulative
Ratings 2002 - 2006
6
PART Process

• The PART questionnaire is divided into four
  sections program purpose/design, planning,
  management, and results/accountability.
• Answers must be clearly explained and cite
  relevant supporting evidence, such as agency
  performance information, independent evaluations,
  and financial information.
• Answers translate into section scores weighted to
  generate an overall score and rating Effective,
  Moderately Effective, Adequate, Ineffective.
• Programs without performance measures or data are
  rated Results Not Demonstrated.
• Additional questions are asked about particular
  types of programs Block/Formula Grant, Capital
  Assets and Service Acquisition, Competitive
  Grant, Credit, Direct Federal, Regulatory-based ,
  Research Development.

7
2007 PART Schedule (p. vii)

• 2007 PARTs identified - January 19.
• PARTWeb available for data entry - January 22.
• Questionnaire guidance available - January 29.
• PART training for OMB and agencies - mid
  February.
• PART drafts due - March 30.
• Consistency check performance measures review -
  May 3-10.
• OMB revises PARTs passes back results to
  agencies - May 18
• Agencies submit appeals - May 25.
• Summaries improvement plans ready for
  ExpectMore.gov - July 9.
• Data entry locked - August 3.
• PARTs published on ExpectMore.gov - mid August.

8
PART Questions

• Four sections
• I. Program Purpose and Design (20)
• II. Strategic Planning (10)
• III. Program Management (20)
• IV. Results (50)

9
Section I Program Purpose and Design (pp. 16-22)

• 20 weight of total score
• Clarity and relevance of program purpose
• Soundness of program design
• Addresses programs structural issues
• Clear design and purpose an essential for
  identifying performance measures

10
Section II Strategic Planning(pp. 23-37)

• 10 of total score, with links to Section IV
  questions
• Addresses program plans and approach to long-term
  goals
• Programs must have long-term and annual
  performance measures and ambitious targets
• Emphasizes independent, quality performance
  evaluations, plus budget transparency and
  budget-performance integration

11
Section III Program Management(pp. 38-54)

• 20 of total score
• Addresses
• Accountability of managers, performance of
  partners
• Coordination with related programs
• Financial management and efficiency improvements
• Correction of deficiencies
• Do programs have procedures in place to measure
  and achieve efficiencies and cost effectiveness?

12
Section IVProgram Results/Accountability(pp.
55-61)

• 50 of total score
• Assesses achievement of long-term and annual
  performance and efficiency goals
• Compares actual performance to targets
  (identified in Sections II and III)
• Effectiveness in achieving goals based on
  independent evaluations
• Compares with performance of similar programs

13
Performance Measures are Central to the PART

• Allows tailoring to the specific program
• Results are the most valuable information
  product of the PART
• Takes most time in completing the PART
• Biggest determinant of overall score and rating
• Updated regularly to help keep PART information
  current

14
How PART improves performance measurement

• Outcome-oriented long-term measures reflecting
  program purpose
• Outcome-oriented annual measures that directly
  support long-term goals
• If goals are outputs, must explain how they
  reflect progress toward desired outcomes
• Challenging but realistic quantifiable targets
  and timeframes
• Clear baseline from which to measure changes in
  performance
• Credit in results section tied to measures in
  strategic planning section
• Performance measures used to manage
• Accountability for achieving performance goals

15
Where performance measures are today

• Of programs assessed
• 86 have long term measures
• 72 have ambitious targets for long-term measures
• 87 have annual measures
• 72 have ambitious targets for annual measures

16
Examples of Improved Measures

• Coast Guard Aids to Navigation
• Old focus Percentage of time radio navigational
  systems available
• Current focus Five year average of number of
  collisions, allisions, and groundings
• National Bone Marrow Donor Registry
• Old focus Number of donors in registry
• Current focus Number of transplants facilitated
  and post-transplant survival rate
• Leaking Underground Storage Tanks
• Old focus Number of clean-ups completed
• Current focus Number of clean-ups that exceed
  state risk-based standards for human exposure and
  ground water migration
• Community Health Centers
• Old focus Numbers and characteristics of
  persons served and services provided
• Current focus Heath outcomes such as low birth
  weight babies
• Small Business Development Centers
• Old focus Number of small businesses counseled
  or trained
• Current focus Number of jobs created (new
  businesses v. old businesses)

17
Performance Measures

• Outcome Events or conditions external to the
  program and of direct importance to the public,
  beneficiaries and/or customers. They relate to
  the programs mission, purpose, and strategic
  goals.
• Output Internal program activities products
  and services delivered to the public,
  beneficiaries.
• Efficiency Reflect economical and effective
  acquisition, use, and management of resources to
  achieve program outcomes or produce program
  outputs.
• Outcome efficiency
• Output efficiency
• Input productivity

18
Performance Goals

• Targets Improved levels of performance needed
  to achieve stated goals.
• Programs must have ambitious but realistic,
  achievable targets and timeframes for performance
  measures.
• Together, measures, targets, and timeframes
  establish the programs performance goals.

19
Program Evaluations

• Scope - Examine underlying cause and effect
  relationship between program and achievement of
  performance targets.
• Independence - Performed by non-biased parties
  with no conflict of interest.
• Quality
• Applicability All programs expected to undergo
  some type of evaluation.
• Impact Prefer effectiveness evaluations
  (outcome, e.g., whether Federal intervention
  makes a difference).
• Rigor The most rigorous evidence that is
  appropriate and feasible for that program.

20
Does It Ever End?

• Steps after PARTs are completed
• Draft summaries for ExpectMore.gov
• Spring Updates in PARTWeb
• Complete Improvement Plans
• All programs must have, regardless of PART rating
• Focus on findings in the PART assessment
• Implement plans and report on progress
• ExpectMore.gov release mid-August

21
Challenges Lessons to Learn Quickly

• Share drafts, communicate frequently.
• Use clear, direct language.
• Stick to deadlines.
• Dont take the PART personally.
• Rely on evidence, not anecdotes.

22
Challenges Measurement

• Uneven quality of performance measures in PARTs
• Several areas difficult to measure
• Increasing the timeliness of performance
  reporting
• Consistency agencies and OMB answer some
  questions differently

23
Challenges Program Evaluation

• Want to promote evaluation to measure and improve
  program design, implementation, and
  effectiveness, including cost-effectiveness..
• Evaluations are not used enough to assess impact
  and improve performance
• Decision makers do not appreciate and,
  consequently, do not routinely invest in
  evaluations.
• Technical complexity can make them hard to
  understand and thus undermine confidence in
  results.

24
Challenges Improvement Plans

• Aggressiveness varies
• Unclear how they impact program results (versus
  PART score)
• Uneven attention to plans across agencies and OMB

25
Challenges Improving Performance

• Improving PART score versus improving performance
• Ensuring that program managers are empowered and
  accountable
• Assessing improvement plans fairly
• Sharing good approaches and models

26
Challenges Impact

• Executive Branch
• Management, funding, or authorization decisions
  are not regularly based on the PART
• Presidents Management Agenda Budget and
  Performance Integration initiative is being used
  to leverage greater use of PART results
• Congress
• Rare, diverse references to PART
• Not the basis for legislative action
• Few oppose vigorously
• Crosscutting
• Opportunity for collaboration among like programs

27
(No Transcript)
28
ExpectMore.gov Summary
29
PART Resources Online

• www.omb.gov/part
• Information on process and schedule
• Guidance for completing PART
• PARTWeb link, users manual
• Supporting materials
• www.ExpectMore.gov

30
How OMB Manages the PART Initiative
31
Presidents Management Agenda

• A strategy for improving Federal management and
  performance with five government-wide and nine
  agency-specific goals.
• Strategic Management of Human Capital
• Competitive Sourcing
• Improved Financial Performance
• Expanded Electronic Government
• Budget and Performance Integration
• The President directed agency heads to designate
  a Chief Operating Officer for day-to-day
  operations.
• The President designated the Presidents
  Management Council (PMC) as an integrating
  mechanism for policy implementation across
  government, headed by OMBs Deputy Director for
  Management and comprised of the COOs.

32
(No Transcript)
33
BUDGET AND PERFORMANCE INTEGRATIONCriteria for
Achieving GREEN
Senior managers meet at least quarterly to
examine reports integrating financial and
performance information for all major Department
responsibilities. Agency works to improve
program performance and efficiency each
year Strategic plans contain a limited number of
outcome-oriented goals and objectives. Annual
budget and performance documents incorporate
measures identified in the PART and focus on the
information in the senior management
report Reports the full cost of achieving
performance goals accurately in budget and
performance documents and can accurately estimate
the marginal cost of changing performance
goals Has at least one efficiency measure for
all PARTed programs Uses PART evaluations to
direct program improvements and hold managers
accountable for them, and uses PART findings and
performance information to justify funding
requests, management actions, and legislative
proposals and Less than 10 of agency programs
receive a Results Not Demonstrated rating for two
years in a row. To maintain green status,
agency Improves program performance and
efficiency each year and Uses marginal cost
analysis to inform resource allocations, as
appropriate.