Classification%20Overview

1
Classification Overview
2
Overview

• A classification chart is one type of
  bootstrapping that produces a solution space of
  two variables
• The vertical axis is a bootstrapped variable
  called Confidence Index it is the probability
  of success (defined in a particular way) of a
  project
• The horizontal axis is the estimated size of the
  investment
• For small and low-risk investments, the decision
  to accept should be made without a full RRA
  assessment
• Larger and riskier investments will tend to
  require a full RRA assessment

3
Classification Chart
1.0
Accept w/o Further Analysis
.8
Proceed with Full RRA
Confidence Index (confidence about value)
.6
.4
Reject w/o Further Analysis
.2
0
100k
1M
10M
100M
10k
Expected Investment Size
4
Defining the Confidence Index

• The confidence index is meant to be an indicator
  of the chance of success of an investment
• Success must be defined by the participants in
  the workshops
• Then a bootstrap model is built (see
  bootstrapping procedure) and evaluations are
  given on the following two questions
• What is the chance of success of this investment?
  (0 to 100)
• How much analysis should be required? (accept w/o
  further analysis, reject w/further analysis,
  continue RRA)

5
Placing the Boundaries

• Three methods are used in concert
• Decision maker interviews
• Checking against classification boundary
  constraints
• Checking responses to the How much analysis is
  required for this investment question from the
  bootstrap list

6
Example Questions for Building the Chart
Even if the confidence index were 100, how big
would an investment need to be to proceed with a
RRA assessment?
O
1.0
Accept w/o Further Analysis
At the minimum size required for classification,
how much does the confidence index have to be for
you to accept the investment without further
analysis?
.8
.6
O
.4
Reject w/o Further Analysis
.2
What is the minimum size of an investment before
classification is required?
O
0
100k
1M
10M
100M
10k
7
Classification Boundaries Constraints
No point on the boundaries of the Risk/Return
Analysis area can be to the left of the Triple
Point
Upper bound of Risk/Return Analysis area must
touch this range
100k
1M
10M
100M
10k
Must be flat or slope up to the Triple Point
1.0
These areas should not be touched by the
classification boundaries
.8
.6
.4
The Triple Point should be within this zone
.2
Must be a vertical line
0
100k
1M
10M
100M
10k
Lower bound of Risk/Return Analysis area must
touch this range
8
Check Boundaries with Bootstrap

• If we ask the question What action would you
  take with this investment we may find that our
  boundaries need adjustment
• Plot the various responses with color-coding so
  that we can check boundaries against bootstrapped
  preferences

1.0
.8
Accept
Proceed w/RRA
.6
No Classification Required
Reject
.4
.2
0
100k
1M
10M
100M
10k
9
Plotting the Investment

• When an individual project is actually classified
  the investment size and the confidence index have
  error
• The two ranges produce the shape of an ellipse in
  two dimensions

1.0
.8
.6
Confidence Index
No Classification Required
.4
.2
0
100k
1M
10M
100M
10k
Expected Investment Size
10
Optional Zones

• Optionally, additional zones may be added if
  there is a dilemma about how to proceed
• Sometimes simply changing those success factors
  that are controllable can make the investment
  acceptable this may indicate another zone
• If RRA Light should be used for investments under
  a certain size, then a zone can be added for that.

1
Accept
RRA Standard
0.9
RRA Light
0.8
SF Adj.
0.7
Confidence Index
No Classification Needed
RRA Light Just a spreadsheet and a page or two
of explanatory material
0.6
Success Factor Adjustments a better technology
record, single sponsor, acquiring capability for
ITG, Sponsor w/better track record could make the
difference
Reject Consider Options
0.5
0.4
0.3
10
100
1,000
10,000
Expected Investment Size (000)
11
Confirm Results

• To confirm results show each of the following
• Plot of the original estimates vs. the model
• The test classification chart
• Plot actual projects on classification chart and
  discuss discrepancies
• Determine volumes in each zone to check if
  support is realistic
• Present results to group

12
Actual Classification Plots

• An Illinois insurance company created a
  classification chart to help prioritize the
  current list of proposed investments
• They wanted to determine which investments could
  be accepted without more analysis and which need
  more analysis
• 18 investments were plotted on the classification
  chart
• The results had a profound effect on investment
  priorities
• Some investments that were assumed to be
  beneficial now required analysis and some that
  required analysis could now be approved
  immediately

13
Regression Example

• Input to the model was based on average VP
  calibrated estimates of the probability of
  success of 42 hypothetical investments.
• Each investment was described by 13 variables
  like project duration, of ITG units involved,
  sponsorship, etc.
• To test consistency, 3 investments were
  duplicates of 3 others.
• Disagreement among VPs on the same investment was
  30 on average.

Comparison of estimates to model
1
0.9
0.8
0.7
0.6
Computed Confidence Index
0.5
0.4
0.3
0.2
0.1
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Average of VPs calibrated estimates
14
Classification of Example Projects
Do Abbreviated Risk-Return Analysis 6. DLSW
Router Network Redesign 9. Extended Hours 18.
Doc. Access Strategy
Do Full Risk-Return Analysis 8. Pearl
Indicator and Pearl I/O interface 11. Richardson
Data Center Consolidation 15. MVS DB2 Tools
1
Accept without Further Analysis 5. Lucent
switch upgrade 7. Image Server Relocation 17.
Enterprise IntraNet to all sites
5
17
7
0.9
6
11
10
0.8
15
9
4
18
8
No Classification Needed
0.7
Confidence Index
3
16
14
12
0.6
1
2
Success Factor Adjustments 4. Network OS
migration to Novell 5.x 10. Optimize Single Code
Base
0.5
13
Reject Consider Other Options 1. Data
Strategy 2. Enterprise Security Strategy 3.
Remote Server Redundancy 12. MQ Series Base 13.
Development Environment 2000 (mf) 14. Source
Control Source Code Mgmt 16. Enterprise InterNet
0.4
0.3
10
100
1,000
10,000
Expected Investment Size (000)
15
Impact of Classification

• Although it was a non-standard application of
  classification, the exercise had a significant
  impact on the IT priorities
• 3 investments plotted in the Accept without
  further analysis area each of these were
  accepted and unnecessary analysis effort was
  avoided
• Some of the more popular projects plotted very
  poorly, causing them to rethink the approach and
  scope of these projects
• Risk return assessments were required for some
  that were assumed to be low risk

16
Proportion of Investments Analyzed

• When classification is applied we find that
  larger companies will do RRA Standard on a larger
  percentage (by budget) of their portfolios
• Even though they are a small percentage of the
  budget, a very large number of smaller
  investments are accepted or rejected on the
  classification chart alone

Belgian HDR client
Australian HDR client
RRA Standard
RRA Standard
RRA Light
RRA Light
Decide by Classification Index alone
None
Decide by Classification Index alone
None