Value Measuring Methodology

1
(No Transcript)
2
Agenda

• Introduction
• VMM Overview
• Concluding Thoughts
• References for More Information

3
Introduction
4
We go into space because whatever mankind must
undertake, free men must fully share...I believe
that this nation should commit itself to
achieving the goal, before this decade is out, of
landing a man on the moon and returning him
safely to the earth."
Special Message to the Congress on Urgent
National Needs, May 25, 1961, Public Papers of
the Presidents 1961, p. 404.
5
How would you justify this investment? How would
you determine the best plan of action for
achieving President Kennedys goals? How would
you link the performance of this investment and
the budget?
Will realizing the Presidents vision have a
positive ROI?
6
Current processes supporting government
investment decisions are insufficient for
effective planning and analysis of investments
Primarily focused on financial benefits (e.g.,
ROI) that impact the government only
Non-financial benefits (results) are not directly
factored into analysis
No structure to force the development of
quantifiable measures
Analysis is viewed as a means to get funding,
not a tool for on-going management evaluation
7

• The Value Measuring Methodologyprovides a
  scalable and flexible approach for estimating and
  analyzing.
• value, risk, and cost
• and evaluating the relationships among them

8
VMM Was Developed Through a Public / Private
Partnership
research analysis
Critical Inputs Research
development
presentation
discussion
9
History of Value Measuring Methodology (VMM)

• July 2001 SSA and GSA took on the task of
  developing an effective methodology to assess the
  value of electronic services that would be
• Jan 2002 VMM first articulated in Building a
  Methodology for Measuring the Value of e-Services
• Oct 2002 Release of VMM How-To-Guide and VMM
  Highlights document by the Best Practices
  Committee of the CIO Council
• April 2003 VMM Roll-Out, held by the Council
  for Excellence in Government in cooperation with
  the CIO Councils Best Practices Committee, OMB,
  and GSA
• Feb 2004 VMM is incorporated into the Strategic
  and Tactical Advocates for Results (STAR)
  Program, a joint initiative of the CIO Council
  and GSA

1. Compliant with current federal regulations
OMB guidance 2. Applicable across the federal
government 3. Do-Able
10
VMM Has Been Used Successfully to Analyze and
Plan Investments Across the Federal Government
at the enterprise level
SSAs Benefit Value Score single set of
measures and weights used to evaluate the
non-financial benefits of IT and Non-IT
investments
at the project level
Business cases have been developed to support the
funding of specific investments at GSA, the
Department of the Interior, the United States
Department of Agriculture, the Department of
Justice, the Internal Revenue Service and the
Department of Navy
11
VMM Overview
12
VMM Considers the Essential Factors for Any
Decision that Needs to Be Made
value

• What Benefits will it Provide to
• Direct Users
• Society
• Government

risk
value
cost
What Could Make Costs Go Up or Performance Slip
from Projected Levels?
cost
How Muchwill it Cost?
risk
13
The Essential Factors are Critical to Each of
VMMs Four Steps
Step 1 Develop a Decision Framework
Step 3 Pull the Information Together
Step 2 Alternatives Analysis
Step 4 Communicate Document

• Tasks
• Identify define the value structure
• Identify define the risk structure
• Identify define the cost structure
• Begin documentation

• Tasks
• Identify define alternatives
• Estimate value cost
• Conduct risk analysis
• On-going documentation

• Tasks
• Aggregate the cost estimate
• Calculate the return on investment
• Calculate the value score
• Calculate the risks score
• Compare value, cost, and risks

• Tasks
• Communicate value to customers stakeholders
• Prepare budget justification documentation
• Satisfy ad hoc reporting requirements
• Use lessons learned to improve processes

14
Step 1 Requires the Decision Framework to Be
Defined to Reflect the Specific Business
Challenge Being Assessed.
Define User Needs Priorities Quantifiable
Measures of Performance (Metrics,
Targets) Foundation for Analysis On-going
Performance Measurement Early Consideration of
Risk
Task 1 Identify Define the Value Structure
Task 2Identify Define the Risk Structure
Risk Inventory Risk Tolerance Boundary
Task 3 Identify Define the Cost Structure
Task 4Begin Documentation
15
The Value Structure is Comprised of Five Value
Factors
DIRECT USER (CUSTOMER) VALUE Benefits directly
realized by users or multiple user groups. Users
or customers will vary based on the type of
initiative being assessed. Users may include,
but are not limited to, government employees,
other government organizations, and
citizens SOCIAL (NON-DIRECT USER/PUBLIC) VALUE
Benefits not related to direct users (e.g.,
society as a whole) GOVERNMENT OPERATIONAL /
FOUNDATIONAL VALUEOrder of magnitude
improvements realized in current government
operations and processes and in laying the
groundwork for future initiatives GOVERNMENT
FINANCIAL VALUEFinancial benefit (e.g., cost
savings, cost avoidance) realized by the
government, including financial benefits received
by the managing or sponsor agency as well as
other federal agencies STRATEGIC / POLITICAL
VALUEBenefits that move an organization closer
to achieving its strategic goals, the priorities
established by the Executive Office of the
President, and congressional mandates
The Five Value Factors Developed based on input
from the government, academics comprehensive
research All-Inclusive Applicable to All
Government Projects Incorporate the priorities of
the PMA, OMB GPRA
16
Value Measures Are Used to Identify, Define and
Quantify the Full Range of Desired Value in Each
of the Value Factors
The Definition of Each Measure Has 4 Parts
17
Value Factors Value Measures Are Weighted to
Reflect the Priorities of Senior Leadership and
Stakeholders
These weightings are used to aggregate the final
value score

• The sum of the weights of the Value Factors
  equals 100
• The sum of the weights of each Value Measure
  equals 100

18
Defining the Risk Structure Early in the Planning
Process Provides Critical Inputs for the
Definition of Alternatives

• How a specific risk affects an alternatives
  benefits and/or costs depends on
• The probability (likelihood) that it will occur
• The severity of its impact if it does occur

Risk will cause expected value to decrease and
expected cost to increase
19
It Is Also Important to Understand an
Organizations Tolerance for Risk
20
The Cost Element Structure (CES) Captures the
Individual Elements of Cost Associated with
Delivering Value and Mitigating Risk
When Building the CESConsider Value
Ensure a complete, comprehensive cost
estimate Alleviate the risk of missing costs or
double-counting Investments made on incomplete
or inaccurate estimates are likely to run out of
funding and, therefore, require justification for
additional funding or a reduction of scope.
21
The Decision Framework is Used in Step 2 to
Identify and Define Viable Alternatives
22
The Base Case Always an Alternative to Be
Examined Whether Viable or Not
rising demand
Projects the results of maintaining current
systems and processes while attempting to keep
pace with changes over time.
base case
workforce attrition
customer satisfaction
status quo
T I M E
23
Avoid Analysis Paralysis When Estimating Value
and Cost
Match Information to the Phase of Development
Data sources and detail depend upon the
initiatives stage of development Use the best
information available rather than looking for
information that doesnt exist Update this
information as better information becomes
available ALWAYS DOCUMENT DATA SOURCES
ASSUMPTIONS
24
Estimate Value and Cost
Use ranges to increase confidence in value
projections
Estimated Range of Performance (RAW SCORE)
and Cost Estimates!
High
Med
Low
Inputs
200
150
100
of Employees to be Trained/year Annual Cost
per Employee Trained
EXAMPLE
1500
1200
1000
25
Cost estimating and cost analysis are part
science and part art
Choose a cost estimating technique that best fits
the scope of the effort to be estimated, the
detail of technical definition, the availability
of usable historical costs, the maturity of the
program, and your own skill and experience
26
Conduct Uncertainty and Sensitivity Analyses on
Both Value Value Estimates to Increase
Confidence

• Uncertainty Analysis
• Based on considerationsof requirement, cost
  estimating, and technical uncertainty
• Increases confidence in the estimate. Doesnt
  increase the precision of the estimate
• Tool Monte Carlo Simulation
• Output Most Likely or Expected Cost Value

• Sensitivity Analysis
• Based on the output of the Monte Carlo Simulation
  
• Sensitive variables have a significant impact on
  the overall estimate
• Output Identification of which variables have a
  significant impact on the overall estimate. Can
  be used to determine which variables merit
  additional research

27
Assess Risk by Considering the Interaction
Between Probability and Impact
The impact of a single risk factor may differ in
magnitude at each point where it interacts with
cost and value
The probability of a specific risk occurring
remains constant through out the analysis of a
specific alternative, regardless of where it
impacts the value or cost of that alternative
28
Adjust Cost and Value Estimates for Risk
X
(30 X 5)
5.1M

.08M
X
(50 X 15)
11.4M

.86M
X
(50 X 5)
15.0M

.38M
1.32M
31.5M
31.5M 1.32M 32.82M
Expected Cost of System Planning Development
x
(Probability x Impact )
29
Step 3 Requires the Aggregation and Comparison of
Data
Aggregate the Cost Estimate
Based on the analysis conducted to date, the
expected total investment cost associated with
Alternative 2 the Federated Model is 7.7
million, with a total life-cycle cost (FY 2004-FY
2013) of 59.4 million.
30
Calculate the Appropriate Financial Metrics
Benefit to Cost Ratio
BENEFITS

COST

Net Present Value
PV(Internal Project Cost Savings, Operational)
PV(Mission Cost Savings) -
PV(Initial Investment) Net Present
Value



Cost Savings
Mission Benefits

Mission Cost
Mission Cost



Preferred Alternative LCCE

Preferred Alternative LCCE

(1)

(1)



Avoidances
1.0 Development

Savings

Savings
Avoidances

1.0 Investment



2.0 Production

2.0 OS

3.0 Operations Support

3.0 Status Quo P
hase Out

Status Quo LCCE

Status Quo LCCE
1.0 Development
1.0 Investment (maybe)
2.0 Production
2.0 OS
3.0 Operations Support
(1)
Supported by LCBE Comparisons.

31
Aggregate the Expected VALUE SCORE
32
Calculate the Risk Scores
cost increase between the Expected Cost the
Risk Adjusted Expected Cost the Cost Risk Score
of performance slippage between the Expected
Value Score the Risk Adjusted Value Score
the Value Risk Score
33
Compare Value to Cost
QUESTIONWhich Alternative Provides the Greatest
Value for the Dollar?
70 points / 25M Dollars 2.8 points per M
Only appropriate at the initiative level where
each alternative is analyzed against the same
measures
34
Outputs of the Analysis are Used to Understanding
the Interaction of Value, Risk and Cost and
Compare Alternatives
Comparing Cost To Value (Expected Risk-Adjusted)
100
90
80
Alt 1
70
Alt 3
Alt 2
60
V A L U E
Expected Alt 1
50
Risk Adjusted Alt 1
40
Expected Alt 2
30
Risk Adjusted Alt 2
20
Expected Alt 3
Risk Adjusted Alt 3
10
0
-
5
10
15
20
25
30
35
40
C O S T (M)
35
Step 4 Focuses Primarily on Effective
Communication
Which Soldier Is Taller?

• How does the value of an investment change if you
  are speaking to the
• Users
• Business Partners
• Internal Stakeholders (e.g., CFO, CIO)
• External Stakeholders (e.g., OMB or the Congress)

It is a question of perspective.
36
Drilling Down to a Particular Area of Interest
Provides Information for Effective Decision
Making, Communication, and Management.
Which alternative presents the greatest risk in
terms of Strategic / Political Value?
Which alternative will provide the greatest value
from the perspectiveof Direct Users?
37
Concluding Thoughts
38
VMM Value Proposition
A decision framework, built on the foundation of
the Essential Factors, creates an outline for
designing, analyzing, and justifying an
investment

• Quantitative analysis provides the information
  and insight required to identify and assess
  alternatives and create a baseline for on-going
  management and evaluation

Comparison of value, cost, and risk provide the
insight required to make tradeoffs in order to
optimize value, minimize cost, and diminish risk
Granularity and depth of analysis provides the
basis of effective and tailored communication to
stakeholder holders, customers, and funding
authorities
Creates the baseline information necessary to
effectively track, manage and evaluate the cost,
performance and risk associated with an
investment
39
Additional Information
40
References for more information

• Building a Methodology for Measuring the Value of
  e-Services
• http//www.estrategy.gov/documents/measuring_final
  report.pdf
• VMM How-To-Guide and VMM Highlights
• http//www.cio.gov/ best practices page
• http//www.cio.gov/documents/ValueMeasuring_Method
  ology_HowToGuide_Oct_2002.pdf
• http//www.cio.gov/documents/ValueMeasuring_Highli
  ghts_Oct_2002.pdf