Getting your money

1
Getting your moneys worth return on investment

• Craig Thomas, CDC-OSTLTS
• Greg Randolph, North Carolina Center for Public
  Health Quality
• Brynn Riley, Maine Department of Health

2
Measuring the Impact of Improvement Lessons
from the National Public Health Improvement
Initiative (NPHII)

• Craig Thomas, PhD
• Division of Public Health Performance Improvement
• Office for State, Tribal, Local and Territorial
  Support
• Centers for Disease Control and Prevention

• Open Forum for Quality Improvement
• June 20, 2012

• Office for State, Tribal, Local, and Territorial
  Support

• Division of Public Health Performance Improvement

3
National Public Health Improvement Initiative

• Five year initiative for health departments
• Adopt and institutionalize cross-cutting
  performance management and quality improvement
  methods
• Improve accountability, efficiency, and
  effectiveness of public health programs and
  services
• Advance health departments readiness for public
  health accreditation

4
NPHII Return On Investment Workgroup

• Why is ROI for improvement important?
• Greater accountability in times of scare
  resources
• Stronger evidence base for what works to improve
  business practices and service delivery for
  greater health impact
• Promotes a business culture of improvement in
  public health
• Purposes
• Identify practical approaches to measuring the
  costs and benefits of improvement with select
  NPHII grantees
• Provide technical assistance as requested
• Develop and share ROI methods, tools, and
  approaches

5
Analytic Approaches to ROI

• Cost Benefit Analysis
• QI methods on health outcomes is important from
  societal perspective
• Not always feasible
• Budget Impact Analysis
• Offers practical approach to assessment of the
  costs and benefits of QI methods
• Calculates the cost and benefits of improving
  business processes and service delivery
• Focused on efficiency and effectiveness outcomes
• Documents both tangible and intangible benefits
  of QI

6
Key QI Outcome Measures

• Efficiency Doing something well with the least
  amount of waste
• Time saved
• Money saved
• Reduced steps in a process
• Effectiveness Achieving a purpose or desired
  result
• Quality enhancement (e.g., completeness and
  accuracy of data)
• Increased customer service satisfaction
• Increased reach of services
• Implementation of evidence-based practices

7
Other QI Outcome Measures

• Improvements that relate to both efficiency and
  effectiveness
• Consolidation and sharing of services
• Leverage of additional funds / revenue streams
• Integration of networks
• Pilot testing and evaluation of innovative ideas

8
Overview of ROI Methodology for QI

• Planning
• Develop objectives AIM Statement, measurement
  and data collection plans
• Data Collection
• Establish baseline, collect cost data and
  outcomes before, during, and after implementation
  of improvement effort
• Data Analysis
• Control for alternative hypotheses convert hard
  date (e.g., time and staff) to monetary value
  calculate the ROI (net project benefits/project
  costs) times 100
• Identify Intangible Benefits
• Increased customer satisfaction improved buy-in
  and organization support greater credibility
  within the community

9
NPHII Strategies to Increase Efficiencies

• Out of 74 grantees, the following have
  prioritized strategies to target efficiencies

Percent Focus
70 Decreasing cost to deliver services
69 Decreasing time to deliver services
57.5 Decreasing time to award contracts
54 Decreasing staff allocation to deliver services
10
NPHII Increasing Efficiencies through QI

• Early examples in saving time and money
• Lab data reporting to CDC Reduced lag time from
  14-21 days to 23 days
• IT costs Identified strategies to save over 1.2
  million annually

11
NPHII Strategies to Improve Effectiveness

• Out of 74 grantees, the following are
  prioritizing strategies to strengthen
  effectiveness

Percent Focus
94.4 Using data to prioritize activities for maximum impact
94.5 Identifying and implementing evidence-based practices
92 Engaging in cross-jurisdictional partnerships
93 Using an organization-wide performance management system
12
NPHII Improving Effectiveness thru QI

• Early successes achieved by some grantees
• Increased enrollment in Medicaid family planning
  program by 109 percent in one year
• Streamlined clinic flow of family planning visits
  to improve patients experience
• Increased referrals of pregnant women in WIC to
  smoking cessation programs

13
Resources

• Phillips, J., Phillips, P. (2007). Show me the
  Money The Use of
• ROI in Performance Improvement, Part 1.
  Performance
• Improvement V. 46, No 9.
• OSTLTS Performance Management / Quality
  Improvement Resources
• http//www.cdc.gov/stltpublichealth/Performance/in
  dex.html

• Office for State, Tribal, Local and Territorial
  Support

• Division of Public Health Performance Improvement

14
Application of ROI to Quality Improvement
Projects

• Greg Randolph MD, MPH
• NC Center for Public Health Quality

15
Why is ROI Important?

• Need to improve performance given substantial
  burden of population health issues and health
  disparities in US
• PH agencies are increasingly applying QI methods
  to their work involves a substantial upfront
  investment
• Sound business practices can help in a time of
  fiscal scrutiny and dwindling budgets/resources

16
How We Measure and Use ROI

• ROI Model
• Template linked to aim statements measurable
  goals (see handout)
• Communications

17
ROI Process Model
Data Collection
Planning
Data Analysis
More Clues
Develop AIM and Measurement Plan
------------------ Think about Potential
Business Impact
Before Improvements (Baseline) ------------------
After Improvements ------------------- Capture
costs throughout
Waste identified via VSM and observation ---------
--------- QI Tools used for improvements
Convert Measurement and Benefits Data to Monetary
Values
Calculate ROI Using Cost and Benefit Values
Identify Intangible Benefits
Reporting
Generate Report with Conclusions and Lessons
Learned
Communicate Results to Target Stakeholders
Lou Anne Crawley-Stout, MBA, CLSSBB, PMP Adapted
from ROI at Work by Jack J. Phillips and
Patricia P. Phillips
17
18
Terminology and Formula
ROI (return on investment) A performance measure
used to evaluate the efficiency of an
investment. ROI Benefits-Costs/Costs
EI (economic impact) Refers to costs and
benefits of an activity. EI Benefits-Costs
Example NC Institute for Public Health
(NCIPH) Project Benefits (annualized
savings) Project Costs Archiving
(paper/printing) 360 Staff
time/travel 9850 Rent elimination
1490 Archiving supplies
200 Labor/time savings 16330
10050 Overtime reduction 2300
20480 ROI 20480-10050/10050 1.04
For every 1 invested in QI, NCIPH received in
return 1.04 after costs. EI 20480-10050
10430
19
Communication Tips

• Do stakeholder analysis early and plan for
  when/how/what will communicate to stakeholders
• customize to specific stakeholder audience
• Since ROI relatively new for public health, best
  to report face-to-face
• Stress that ROI data should not be used to reduce
  budgets further
• by cutting budget of organization generating
  substantial ROI, would starving most innovative
  and effective organizations and halt spread of
  use of ROI
• Communicate how ROI illustrates you are better
  able to achieve your mission and other intangibles

20
Lessons Learned

• Begin with the end in mind
• Begin with your aim statement
• Capture tangibles and intangibles throughout
• When converting tangible benefits into monetary
  values, be conservative
• Vet with your team and selected stakeholders
• Intangible Benefits (non-monetary benefits) are
  extremely valuable

21
Additional ROI Results in Jan / Feb 2012 issue
of Journal of Public Health Management
PracticeApplying Lean Principles and Kaizen
Rapid Improvement Events in Public Health
Practicehttp//journals.lww.com/jphmp/toc/2012
/01000
Resources
22
Resources

• ROI at Work Best-Practice Case Studies from the
  Real World Jack J. Phillips and Patricia Pulliam
  Phillips
• www.ncpublichealthquality.org
• www.ies.ncsu.edu/
•  
• Greg Randolph greg.randolph_at_ncphf.org

23
Using AIM Statements to Capture Return on
Investment

• Brynn Riley
• Performance Improvement Manager, Maine
• June 20, 2012

24
Introduction

• Quality Improvement (QI) teams have an
  opportunity to show projects return on
  investment (ROI) to stakeholders
• QI teams should consider the
• upfront investment before applying QI methods
• costs and benefits throughout the projects
  duration.

25
Focusing on Benefits and Costs

• Accounting for costs and benefits will
• Help improve the organizations case for
  undertaking a QI project
• Help prioritize competing interests
• Help account for community or societal benefits
  where data doesnt exist
• Include intangible benefits, which are a
  significant and growing part of an organizations
  and communitys worth.

26
Focusing on Benefits and Costs

• Consider how it translates into public health and
  community value.
• Part of public health is preventing adverse
  health outcomes for a population, essentially
  creating non-events.
• Including intangible benefits in cost benefit
  calculations account for the organizations added
  value to the community that may not otherwise be
  included.
• Concerns
• Intangible benefits cannot be easily measured in
  dollar terms
• The benefits may not be viewed as valid by some
  people

27
Applying the AIM Statement

• Use the AIM statement as a springboard to hone
  the a QI Projects focus and evaluate return on
  investment.

28
What is an AIM Statement

• The Aim is sometimes called the Charter or
  Opportunity Statement. It is a goal.
• A written discrete, measurable, and time-bound
  description.
• Includes the scope of an improvement effort, as
  viewed by the organization and the
  team/individual striving for improvement.
• It should be developed with input from Senior
  Leadership to ensure support for the Team and
  alignment with the strategic goals of the
  organization.

29
AIM Statement General Features

• What will improve?
• When will it improve?
• How much will it improve?
• For whom will it improve?

30
AIM Statement Guidelines

• When developing the AIM statement stay at a
  strategic level - try not to get too operational.
• Determine baseline measurements while framing the
  AIM statement
• Outline the expected tangible and intangible
  benefits

31
Review the AIM on a Regular Basis

• AIM statements are not written once and set in
  cement.
• A regular team task is to review the aim
  statement
• "Remember, were here to increase immunizations
  by 50 within 12 months,"
• How are we doing?
• Regular review of progress quantitatively
• It is acceptable to tweak the aim, as the team
  learns more about their work or new information
  changes the teams direction.

32
AIM Statement Sections

• Section 1 Problem Description, Boundary, and
  Team Composition
• Describe the problem or opportunity to be
  addressed
• Example Physician/Immunizing Stakeholders are
  not utilizing the Immunization Information System
  (IIS) tools provided, resulting in lower than
  acceptable immunization rates.
• This process is important to work on now because
  of (describe the impact this problem or
  opportunity is having on the agency, program,
  customers, employees, or the community
• Example Not utilizing the provided public
  health improvement tools in the IIS results in
  improperly immunized people, wastage of State
  supplied vaccine, client susceptibility to
  disease, and community risk of disease.
• Team Sponsor _______________________
•  
• Team Leader _________________________
•  
• Team Members ______________________ Area of
  Expertise ________________
•  

33
AIM Statement Sections

• Section 1 Problem Description, Boundary, and
  Team Composition
• The problem or opportunity starts with and ends
  with
•  
• For Example
• The problem or opportunity starts with the lack
  of utilization of IIS tools and with the
  assignment of staff to conceptualize and
  recommend an improvement plan.
• and ends with the publishing of a results
  document that demonstrates an increase in
  immunization rates via the developed
  methodologies and that also quantifies the cost
  of the improvement effort in order to identify
  the cost associated with sustainability
• IIS Immunization Information System (ImmPact),
  Maines tracking system

34
AIM Statement Sections

• Section 2 Internal and External Benefit and Cost
  Description
•  
• This opportunity has the following estimated
  potential benefits internally and to the external
  community

Internal and External Tangible Benefits (cost benefit) Intangible Benefits
Are benefits produced by an investment that is immediately obvious and measurable. (i.e. IT software) A tangible benefit can usually be expressed in terms of a monetary savings. Are indirect benefits which cannot be felt or touched - such as improved health access, staff morale, or the image of the health department by stakeholders.
35
AIM Statement Sections
Section 2 Internal and External Benefit and Cost
Description  

• What is unique to many public health
  organizations is the need for a way to account
  for intangible benefits to the community being
  served.
• For example, tangible benefits may be a decrease
  in the smoking rate, an increase in the
  immunization rate, reduced infant mortality, etc.
  
• An intangible benefit may be a program that
  contributes to increased self-esteem resulting in
  greater use of preventive services and adherence
  to an asthma management plan.

36
AIM Statement Sections

• Section 2 Internal and External Benefit and Cost
  Description
•  
• This opportunity has the following estimated
  potential benefits internally and to the external
  community

Internal Tangible Benefits (cost benefit) Intangible Benefits
Reduced vaccine wastage Increased number of doses recorded within the IIS Increased Immunization Rates Increased Stakeholder awareness of the IIS Increased Stakeholder satisfaction with the IIS Increased use of IIS tools

External Tangible Benefits (cost benefit) Intangible Benefits
Increased Client Visits Increased Services Rendered Increased Billing Activities Increased Immunization Rates Additional opportunity to serve the client Increased Community Marketing Opportunity Improved Assessment Rates
IIS Immunization Information System (ImmPact),
Maines tracking system
37
Focusing on Benefits and Costs

• Some questions to consider are
• If we implement a solution to this problem what
  are the potential cost categories that will be
  impacted and by how much?
• How much will it cost to develop a solution and
  implement it?
• What is the cost of doing nothing?
• What is the cost of sustaining the solution?
• Are there possible unintended consequences, and
  if so, what are the costs?
• If we plan to use existing resources, what will
  we have to stop doing to divert resources to this
  problem/solution?
•  

38
AIM Statement Sections

•  Section 3 Current State Performance and Desired
  Future State
• Describe the current state of the problem or
  opportunity and its current performance (baseline
  data)
• External Stakeholders are not using the
  integrated Public Health Improvement tools
  resident in the IIS for their use. These tools
  include, but are not limited to
•  Immunization Coverage Reports (ICR),
• Reminder/Recall Letters (R/R),
• Client Vaccine Forecasting (ACIP Forecasting)
• Describe the ideal future state
• What are the key driving and restraining forces
  impacting the current state and the ideal future
  state?
•  

39
AIM Statement Sections

• Section 4 Improvement Description
•  
• This effort should improve the current state by
• Describe the improvement goals to be achieved
• The timing of these improvements
•  
• We will measure improvements by
•  

40
AIM Statement Sections

• Section 5 Internal and External Customer
  Identification
•  
• For the following customers/clients (customers,
  staff or those affected by the process under
  improvement)
• For Example
• Maine Immunization Program Staff
• Consumer Services Staff
• IIS Support Staff
• Administrative Stuff
• Physicians Office
• Nurse Staff
• Administrative Support
• Billing/Fiscal Support 

41

• This presentation is based on the article
• Calculating the Real Value of Process
  Improvement Factoring in Intangible Benefits.
  J. Moran and B. Riley, Process Excellent Network,
  April 2012
• Web Address when ready
• Process Excellence Network April 2012
• http//www.processexcellencenetwork.com/people-per
  formance-and-change-in-process-improveme/articles/
  calculating-the-real-value-of-process-improvement/
  macSSIQ_OI_Featured_2011utm_sourceprocessexcel
  lencenetwork.comutm_mediumemailutm_campaignHrO
  ptInutm_content4/12/12