NOAA Observing System Portfolio Management

1
NOAA Observing System Portfolio Management

• A Decision Analysis Approach

2
Background

• NOSA tasking
• Develop observing system baseline architecture
• Develop observing system target architecture
• Base prospective observing systems on validated
  requirements
• Investigate system consolidation and ownership
  transition
• Conduct a systemic review of all observing
  systems

3
Purpose

• Introduce a study based on the observing system
  portfolio management approach to help
• Develop observing system target architecture
• Investigate system consolidation and ownership
  transition
• Conduct a systemic review of all observing
  systems

4
The Portfolio Question

• The question How valuable are NOAAs observing
  systems? decomposes into two
• How effective are NOAAs observing system
  projects?
• What are the costs of NOAAs existing and
  candidate observing system projects?

5
The Effectiveness Question

• Effectiveness should be measured
• Measurement should reference NOAA mission goals
• Question becomes To what extent do NOAAs
  existing and candidate observing system projects
  contribute to NOAA mission goals?

6
Measuring Contribution to NOAA Mission Goals

• Methodology based on Analytical Hierarchical
  Process (AHP)
• Structured, traceable decision-making process
• Used by USDA, DOE, HUD, VA, FAA, SSA
• Cited by CIO Council as an important decision
  support methodology

7
Methodology Outline

• Link observing system projects to NOAA mission
  goals
• Determine relative weights of NOAA mission goals
• Determine observing system projects relative
  contributions to NOAA mission goals
• Synthesize (2) and (3) to determine relative
  values of observing systems
• Introduce cost derive cost-effectiveness
  comparison
• Generate improvement possibilities

8
1. Linking Observing Systems to Goals

• Example linking

9
2. Determining Weights for Goals

• Present a series of pairwise comparisons to
  senior leadership

Which goal is more important to realizing NOAAs
mission to understand predict changes in the
Earths environment and conserve and manage
coastal and marine resources to meet our Nations
economic, social, and environmental needs?
Understand Climate
Weather Water Info
Equal
Slight
Slight
Moderate
Moderate
Large
Large
Extreme
Extreme

• Derive weights from comparisons

10
3. Determine Observing Systems Contribution to
Mission Goals

• Present a series of pairwise comparisons to goal
  teams

Which observing system contributes more to
understanding the climate?
Marine Optical Buoy
Fish LIDAR
Equal
Slight
Slight
Moderate
Moderate
Large
Large
Extreme
Extreme

• Derive weights from comparisons

11
4. Synthesize

• Apply goal weights to observing system weights
• Obtain relative values of observing systems to
  NOAA

12
5. Introducing the Cost Question
100
Low Cost / High Effectiveness
High Cost / High Effectiveness





x
x

x
Effectiveness



x
x


Low Cost / Low Effectiveness
High Cost / Low Effectiveness
0
100
0
Cost (pv)
- existing project
x - candidate project
13
6. Improvements - Phase A

• Resource allocation
• Select cost-effective observing systems
• Knapsack budgeting given finite funds what is
  the best mix of existing and candidate observing
  system projects?

14
Phase A Best Mix of Observing Systems
100
Low Cost / High Effectiveness
High Cost / High Effectiveness





x
x

x
Effectiveness



x
x


Low Cost / Low Effectiveness
High Cost / Low Effectiveness
0
100
0
Cost (pv)
- existing project
x - candidate project
15
6. Improvements - Phase B

• Internal improvements
• Increase effectiveness, and or
• Cut costs

16
Phase B Internal Improvements
100
Low Cost / High Effectiveness
High Cost / High Effectiveness





x
x

x

Effectiveness



Increase Effectiveness
x
x


Lower Costs
Low Cost / Low Effectiveness
High Cost / Low Effectiveness
0
100
0
Cost (pv)
- existing project
x - candidate project
17
Phase B New Mix of Observing Systems
100
Low Cost / High Effectiveness
High Cost / High Effectiveness





x
x

x

Effectiveness



x
x


Low Cost / Low Effectiveness
High Cost / Low Effectiveness
0
100
0
Cost (pv)
- existing project
x - candidate project
18
Study Execution and Timeline
19
Back-up Slides
20
AHP and Expert Choice

• T. L. Saaty, The Analytic Hierarchy Process
  Planning, Priority Setting, Resource Allocation,
  McGraw-Hill, New York, 1980
• Software Expert Choice 2000 for Groups
• Price ?