A Practical Guide to ROI

1
A Practical Guide to ROI
2
A Practical Guide to ROIWhy do we need this?

• Members are being asked to provide return on
  investment data by funders, decision-makers,
  politicians, etc.
• Most of us in the public health arena are not
  accustomed to performing ROI

3
Purpose

• Provide resources and tools to
• Understand concepts and processes of ROI
• Conduct an effective ROI
• Identify other methods to assess a programs
  economic impact when ROI is not feasible or
  appropriate
• Make ROI a routine part of the evaluation process

4
So, a few basics

• What is ROI?
• ROI return on investment
• Economic measure of the financial return on
  investment in a system or program
• An ROI analysis attempt to attribute value to
  program outcomes (direct indirect)

5
How is ROI calculated?

• ROI RETURN (value of benefits) X 100
• INVESTMENT (cost of programs)

6
Where does ROI fit into our evaluation planning?

• ROI is special case of evaluation that focuses on
  value expressed in
• However, other non-dollar indicators of value
  (i.e., participant satisfaction, team building,
  employee loyalty) should also be considered as
  part of an overall evaluation plan.
• Thus, comprehensive ROI analyses utilize
  evaluation data collected at multiple levels

7
What evaluation data are we talking about?

• Data collected throughout the program
• Level 1 Participant satisfaction
• Level 2 Actual learning acquired
• Level 3 Behavior changes
• Level 4 Business Impact
• Level 5 ROI

8
So, how do I do this?

• Seven steps in ROI Process
• Prepare for ROI
• Plan Your Work
• Collect your data
• Isolate the program impact
• Calculate the ROI
• Communicate the results
• Make ROI a routine
• Each step has multiple tasks

9
Step One Prepare for ROI

• First task Is it appropriate for me to perform
  an ROI on this program?
• Do I have clearly defined goals objectives?
• Do I have knowledgeable, unbiased staff to do the
  work?
• Do I have an internal sponsor for this?

10
Preparing for ROI

• Does my program meet specific criteria?
• Does this program last for 12-18 months?
• Is it expensive?
• Does it have a large audience?
• Is it highly visible and of interest to leaders?
• Will it drive major change?
• Can I get the data I need?
• Can I measure program performance?

11
Preparing for ROI

• Task Two Establish your timetables
• Include time and resources needed to complete the
  entire analysis
• Be sure to consider time needed to overcome
  possible obstacles
• Use this as the master plan for the work to be
  done.

12
Preparing for ROI

• Task Three Laying your groundwork
• Identify staff who will be involved in or
  affected by the ROI process, and ask for their
  input, experience, and concerns
• Talk with the management team to understand their
  expectations concerns
• See if the organizational policies support the
  endeavor

13
Preparing for ROI

• Task Four Assembling the Project Team
• Assess the project team and resources
• Involve people who are knowledgeable about ROI
• Utilize an objective evaluator
• Identify a Champion
• Delegate responsibilities

14
Step Two Plan Your Work

• Task One Develop strong program objectives
• Consider the entire program do you have
  objectives at all five levels of evaluation,
  especially Levels 3-5?
• Are they easily measurable?
• Are they going to measure the most significant
  aspects of the program?

15
Plan your work

• Task two Design your evaluations
• What do you ask?
• Do your questions provide the data you want?
• How do you ask it?
• Document review to surveys
• Whom do you ask?
• Participants, stakeholders, significant others
• When do you ask?
• Timing of data collection is critical

16
Step Three Gather Credible Evidence

• Task collect the data
• Define the credibility of the data.
• Look for data in all parts of the program
• Remain objective
• Look for red flags
• Never alter the standards you set

17
Step Four Isolate the Program Impact

• Task One Identify other factors
• Could something other than your program have
  contributed to improvements seen
• Task Two Look at techniques for assessing
  factors of success
• Comparison Group Analysis
• Trend-line Analysis
• Estimation
• Using multiple techniques

18
Isolate the program impact

• Task Three Decide on your technique
• Consider
• Feasibility of resources, access, support
• Accuracy of the technique
• Credibility with the stakeholders
• Disruption it will cause in normal work
• Time needed at all levels to complete the work

19
Step Five Do the Math Calculate the ROI

• Task One establish the value of benefits
• The most difficult of this process in PH
• Identify all tangible benefits
• Convert tangible benefits to
• Focus on a unit of measure
• Determine the value of that unit of measure
• Calculate the change in performance of the
  measure
• Determine the annual amount of change
• Calculate the annual total value of improvement

20
Do the Math

• Techniques to convert benefits to
• Converting output to contribution
• Calculating the cost of quality
• Historical costs
• Internal External experts
• External databases, cost calculators
• Linking with other measures
• Estimation

21
Do the Math

• Identify the intangible benefits
• Image, satisfaction, increased commitment,
  enhanced leadership, improved teamwork, increased
  efficiency
• Convert soft data to monetary terms
• Assess your intangible benefit list to see if any
  of them can be linked to tangible events
• For instance - Can attendance at and satisfaction
  with health education programs be linked to
  decreased absenteeism?

22
Do the Math

• Task Two establish the investment of cost of
  program
• Basic cost categories
• Development costs
• Delivery costs
• Evaluation costs
• Analysis costs
• Task Three complete the calculations

23
Step Six Communicating your results

• Task One keep lines of communication open
  throughout the analysis process
• At the end of the project, create a plan.
• Decide who needs full report, who needs only
  portions
• When do the various groups need to hear it?
• Who will tell them?
• In what format?
• Use data to drive improvement

24
Step SevenMaking ROI Routine

• Show leaders of the organization how beneficial
  the ROI process was and can be for future
  program.
• Develop templates, policies and procedures that
  make the most of lessons learned.
• Do it again!

25
Lots of Concepts

• Weve given an overview only
• A Practical Guide to ROI Analysis gives much
  more detail (81 pages) and several templates and
  tools to simplify your process.
• The Guide can be downloaded from
  www.chronicdisease.org, or ordered in hard copy
  at mnoble_at_chronicdisease.org