Statistical Process Control of Project Performance

1
Statistical Process Controlof Project Performance
Walt Lipke Software Division Tinker AFB, OK
SCEA 2002 June 11-14 Scottsdale, AZ
2
Objective
To discuss the application of SPC Control Charts
to the EVM indicators,SPI and CPI
CPI
Control Charts
EVM
SPC
SPI
3
Overview

• Introduction
• SPC applied to Software Development?
• Review EVM SPC
• SPC with EVM Does What?
• Problems / Cause
• Solution Criteria
• Proposed Solutions
• Testing / Results
• Summary

4
Introduction

• Software Division
• SEI CMM Level 2 (1993) First in Air Force
• SEI CMM Level 4 (1996) First in Federal Service
• ISO 9001 / TickIT (1998)
• IEEE / SEI Software Process Achievement Award
  (1999)
• EVM Facilitated the Achievements

5
Why SPC?

• SEI CMM Level 4 Then Now
• Statistically Manage the Sub-process
• CMM Evaluators
• Show me the SPC Control Charts
• Quality Control vs Performance Management

6
SPC Review

• Several Methods Control Charts
• Control Charts Several Types
• Individuals and Moving Range

Anomalous Behavior
Anomalous Behavior
Process Behavior
7
Control Chart
Observed Values
Anomalous (signal)
Observations in sequence
8
EVM Review
Total Allocated Budget
Management Reserve

Budget at Completion
Negotiated Completion Date
BCWS
ACWP
BCWP
Project Completion Date
Time
9
SPC with EVM Does What?

• Performance Prediction
• Probability of Success
• EAC ECD range
• Project Planning
• Historical Data
• Risk MR Strategy
• Process Improvement
• Plan Execution
• Decreasing Variation

10
Planning/Performance/Improvement
Schedule Distribution
Performance Window (PW)
Total Allocated Budget
Negotiated Performance (gt 50 PW)

Budget at Completion
Planned Performance ( 50 PW)
Negotiated Project Completion
Cost Distribution
Planned Project Completion
Time
11
Problems
SPI Control Chart
SPI-1 Control Chart
12
Problems
SPI (signal removed)
SPI-1 (no signal)
13
Problems
14
More Problems
Observations
15
Problem Example
SPI-1
SPI
16
Problem Summary

• ltPIgt gt PIcum ltPI-1gt gt PI-1cum
• ltPIgt-1 ltPI-1gt
• Signals (nearly always) gt 1.0
• PI signals PI-1 signals
• PI sigma PI-1 sigma
• Histograms Normal Distribution
• Without Resolution SPC
  Application

17
Problem Cause?
Skewed Distribution

• Average
• Signals
• Sigma

PI or PI-1
Normal Distribution
18
Solution Criteria

• ltPIgt-1 ltPI-1gt
• PI Signals PI-1 Signals
• PI Sigma PI-1 Sigma
• (4) Histograms Normal Distribution

19
Problem Solution
SPIa
SPIb-1
5.0
Invert Data lt 1.0 - Inverted Data behave
as if 1.0
1.0
SPIb
0.2
Distinguish Inverted Data
0.0
Use Inverted Data and Unchanged Data for
SPC analysis
-3.0
SPIb-1
20
Data Transform Rules

• If PI ³ 1.0, then PI PI
• If PI lt 1.0, then PI 2 - PI-1
• If ltPIgt ³ 1.0, then ltPIgtu ltPIgt
• If ltPIgt lt 1.0, then ltPIgtu (2- ltPIgt)-1

Perform SPC analysis with Transformed Data
21
Problem Solution -Example
SPI
SPI
22
Proposed Solution Evaluation

• Demonstrates meeting criteria 1, 2, and 3
• Mathematically meets criteria 1, 2, and 3
• Proof enough?

23
Data Transform Histogram Test
CPI-1 Histogram
CPI-1 Histogram
24
Proposed Solution - 2

• Resolves PI vs PI-1
• Resolves PI lt 1.0
• Transformation Simplicity
• Satisfy Criteria?

SPI
(1.609)
0.2
Logarithm Property x ln x x-1 -ln
x ln 1 0
ln SPI
(-1.609)
25
Natural Log Criteria Test
ln SPI
ln SPI-1
26
Natural Log Histogram Test
Legend Solid Line ( ) - actual Dashed
line ( ) - expected
Count
ln CPI-1 Histogram
27
Testing Summary
28
Sensitivity Analysis
áSPIñ (0.625,0.112)
áSPIñu (0.651,0.082)
ïáPIñ - PIcum ï
áSPIsñ (0.284,0.025)
áln SPIñu (0.384,0.018)
áln SPIsñu (0.266,0.01)
áSPIsñu (0.327,0.007)
Note

• Subscript s indicates the signal is removed from
  the calculations.
• Subscript u indicates the average value is
  untransformed from the average value determined
  from the SPC analysis

29
Summary

• SPC application to Software Development
• SPC applied to CPI SPI
• Project Execution
• Project Planning
• Process Improvement
• Problems
• Data Representation
• SPC Results

30
Summary

• Solutions
• Data Transform
• Natural logarithm
• Criteria
• Results independent from data representation
• Results derived from Normal Distribution
• Testing/Results
• Data Transform Good
• Natural Logarithm - Better

31
Final Remarks
Managing to CV and SV

• Equivalent to CPI and SPI
• CV 1 CPI-1
• SV SPI 1
• Distribution is skewed
• Data transformation is needed

32
Final Remarks
Weekly EV vs Monthly SPC

• SPC Better Management Decisions
• Weekly EV More Management Decisions
• Weekly EV w/o SPC Process Tampering
• Try SPC Its Not Difficult