Title: Army Cost Review Board Office
1Cost Analysis for Non-Costers
- Army Cost Review Board Office
- Mort Anvari Mort.Anvari_at_US.Army.mil
- Steve Loftus Steve.Loftus_at_US.Army.mil
2HQ Department of the Army
Secretary of the Army
Chief of Staff, Army
Under Secretary of the Army
Vice Chief of Staff, Army
Assistant Secretaries of the Army
Financial Management Comptroller
Acquisition, Logistics Technology)
Manpower Reserve Affairs
Installations Environment
Civil Works
Cost Economics
Budget
G8
3DASA, Cost Economics
Director Programs Strategy
Director Acquisition Costing
Chief, Cost Review Board
Installation Costing
Cost Management
Unit Mission Costing
Weapon Systems
C4ISR Costing
Cost Policy Research
4Cost Review Board (CRB)
- The ASA(FMC) Formed the Army Cost Review Board
to review Cost Estimates for Major Weapon
Information Systems - Created in Response to the Need for
- A Comprehensive Army Cost Position, Acceptable to
Acquisition Financial Management Communities
Supports PPBES
The CRB Reviews Major Weapon Information
Systems at Their Critical Acquisition Decision
Points. All ACAT I Programs Programs of
Special Interest Must Have a Recommended ACP
Briefed to the CRB.
5Cost Analysis Requirement
MS 0
MS I
MS II
MS III
O L D
Program Definition Risk Reduction
Engineering Manufacturing Development (EMD)
Production, Fielding/ Deployment
Concept Exploration
Cost Estimates Needed
Concept Exploration
Component Advanced Development
System Integration
System Demo
Production Readiness LRIP
Rate Production Deployment
N E W
Support
Concept Technology Development
System Development Demonstration
Production Deployment
Pre-Systems Acquisition
Systems Acquisition (Engineering and
Manufacturing Development, Demonstration, LRIP
Production)
Sustainment
6Cost Estimating Process
Final Document
Data Collect Normalize
Estimate Formulation
Definition Planning
Review/ Presentation
DOCUMENTATION
7Cost Analysis Model
Data/CERs LCC Estimates Cost Drivers
Develop Cost EstimateStructure and WBS
EstablishGround Rules and Assumptions
Prepare Cost Estimatesfor Each Element
Test TotalSystem Estimate
Prepare Documentation
Start
CompileData Base/ CERs/Models
Engineering Analogy Parametrics Expert Opinion
Reasonableness SensitivityAnalysis Cost-RiskAsse
ssment
8Situation
- You have just been tasked to develop a cost
estimate, that is, a professional opinion about
the cost of an item, a service or a thing. - Lets discuss a process for organizing and
developing this estimate.
9Definition and Planning
- Influences the success of the estimate
- Understanding the requirements and how you
approach the process will establish the
guidelines and procedures for the estimate. - Ask lots of questionsThey help you understand
the requirement.
10Questions
- Why is this cost estimate needed?
- What decisions are pending on the results of this
estimate? - Will the estimate be briefed and to whom?
- Will the results be incorporated into some
document? - What does the recipient expect to have included
or excluded? - What excursions or variations from the baseline
are anticipated?
11Questions Continued
- What are the program and funding constraints
especially if the program is a Joint Program? - What are the time constraints for this estimate?
- What is the acquisition phase of the program?
- Is the program definition mature?
- Does technology exist today to design, develop,
test and manufacture the system? - What is the interrelationship with other systems?
- Are there previous contracts? How many? What
type? - How have the contractors performed to date?
12Definition and PlanningKnow Purpose of the
Estimate
- Main purposes of estimates
- Budget Formulation
- Comparative Studies
- Source Selection
13Purpose of EstimateBudget Formulation Estimates
- Program Office Estimate (POE)
- Component Cost Analysis (CCA)
- Independent Cost Estimate (ICE)
- What-if exercises
- Rough Order of Magnitude (ROM)
- Should Cost Estimates
- CAIV
14Purpose of EstimateComparative Studies Estimates
- Making cost benefit comparisons between
alternatives - Economic Analysis (EA)
- Analysis of Alternatives (AoA)
- Force Structure
- Trade-off Studies
- Source Selection
- Prioritization
15Definition and PlanningDefining the System
- Adequate description of the technical and program
characteristics of the system - What are the physical and performance
characteristics? - What are the development, production, and
deployment schedules? - How many systems are to be produced?
- How will the systems be supported contract,
in-house, two or three levels of maintenance?
16Defining the System Integrated System Schedule
17Defining the System Work Breakdown Structure
(WBS)
- WBS defined in MIL-HDBK-881 as
- product-oriented breakdown of hardware,
software, services, data and facilities that
define the system. - WBS breaks a total job down into manageable
pieces portrays the way work is to be done. - WBS displays a companys reporting structure.
- Program managers may cite MIL-HDBK-881 for
guidance only in contract solicitations.
18MIL-Handbook-881 (WBS)
19Defining the System Cost Element Structure (CES)
- 1.0 Research Development Test Evaluation
(RDTE) - 2.0 Production
- 3.0 Military Construction (MILCON)
- 4.0 Military Pay and Allowances (MPA)
- 5.0 Operating and Maintenance Army (OMA)
20Defining the System COST ELEMENT STRUCTURE
1.0 RDTE
CES ELEMENT FY00CM
TYM 1.01 DEV. ENG.
1.02
PEP 1.03 DEV. TOOL.
1.04 PROTO MFG.
1.05 SEPM 1.06
SYS TE
1.07 TRAINING
1.08 DATA
1.09
SUPP EQUIP. 1.10 DEV. FACILITIES
1.11 OTHER
39.039M 38.260M 0.408
0.386 0.457 0.450
110.421 107.724 78.266
76.363 11.112 10.927
1.989 1.954 3.439
3.413 4.897
4.758 0.0 0.0 0.968
0.928
21Defining the System COST ELEMENT STRUCTURE
2.0 PROCUREMENT
CES ELEMENT
FY00CM TYM 2.01 NON REC PROD.
2.02 REC. PROD
2.03 ENG. CHG 2.04
SEPM 2.05 SYS TE 2.06
TRAINING 2.07 DATA
2.08
SUPP. EQUIP. 2.09 OPER./SITE/ACT.
2.10
FIELDING
2.11 TRAIN. AMMO/MSLS
2.12 WAR
RESV. 2.13 MODS
2.14
OTHER
16.110M 16.583M 1,169.348
1,312.377 0.0
0.0 116.637 132.258
12.564 14.437 28.880
31.499 2.073 2.300
146.460 158.304 0.0
0.0 89.525
101.635 59.001
79.482 0.0
0.0 236.619 280.729 51.739
64.129
22Defining the System COST ELEMENT STRUCTURE 3.0
MILCON
CES ELEMENT
FY00CM TYM 3.01
DEVELOP. CONSTRUCTION 3.02 PRODUCT.
CONSTRUCTION
3.03 OPERATION/SITE
ACTIVATION 3.04 OTHER MILCON
23Defining the System COST ELEMENT STRUCTURE 4.0
Military PayAllowances (MPA)
CES ELEMENT
FY00CM TYM 4.01 CREW 4.02
MAINTENANCE
4.03 SYSTEM SPECIFIC
SUPPORT 4.04 SEPM 4.05 REPLACEMENT
PERSONNEL 4.06 OTHER MPA
24Defining the System COST ELEMENT STRUCTURE 5.0
Operating Maintenance Army (OMA)
CES ELEMENT
FY00CM TYM 5.01 FIELD MAINT.,
CIV LABOR 5.02 SYS. SPECIFIC BASE OPS
5.03 REPLENISHMENT DLRs 5.04
REPLEN. CONSUMMABLES 5.05 POL 5.06
END ITEM MAINTENANCE 5.07
TRANSPORTATION
5.08
SOFTWARE 5.09 SEPM
5.10 TRAINING
5.11 OTHER OMA
25Defining the System Cost Analysis Requirements
Description (CARD)
- Source of a systems description
- Describes important features
- Is provided to other groups preparing cost
estimates - Helps ensure all groups are costing out the same
program. - Prepared by program office approved by
- DoD Component Program Executive Officer
26Defining the System Cost Analysis Requirements
Description (CARD) Continued
27Defining the System CARD Continued
28Definition and PlanningGround Rules Assumptions
- State the conditions which must take place in
order for the estimate to be valid - Ground rules and assumptions must be documented
since changes in these areas provide an audit
trail for changes in the cost estimate.
29Data Collection and Analysis
- Collection and analysis represent a significant
amount of the overall estimating task in terms of
time. The analysis will include decisions on
what programs to include in the data set to
whether to truncate lot data on a program for
which you are calculating a learning curve. - Document data in your analysis, and any
assumptions you make
30Data Collection and Analysis
- The direction we take in collecting historical
data will be determined by our choice of
estimating methodologies. - This step may also dictate a change in estimating
approach due to the availability or
non-availability of certain data. - Data collection is not limited to cost data. We
must also collect technical and program data if
we want the total picture of the historical
systems. This will help us ensure the
comparability of the systems that we are
collecting data on with the system we are
estimating.
31Data Collection and Analysis Most Difficult Task
in Cost Estimating
- Data Sources
- Data Types Cost/Resource, Technical, Program
- Categories Primary, Secondary
- Data Problems
- Wrong Format Matching up Definition
- Temporal Factors - comparability
- Normalization
- Data Location
32Army Tools for Total Ownership Cost Estimating
Life Cycle Cost Management Tools
33Definition and PlanningSelect the Estimating
Approach
- Techniques available
- Analogy
- Parametric
- Engineering
- Extrapolation
- Expert Opinion
- Select the technique that is most applicable to a
specific WBS element
34Definition and PlanningEstimating Methods
- Analogy
- Basic Comparison
- Factors
- Parametric
- Regression Analysis
- Engineering
- Detailed
- Expert Opinion
- Committee
- Delphi
35Life Cycle Cost Estimates
- Cost estimates based on confidence intervals
- Parametric analysis based on similar systems and
similar attributes (regression) - Engineering data based on reliability projections
used for bottoms-up estimate - Actual system costs used to extrapolate future
system costs - Cost estimate revised every two years after
production - Weights associated with non-actuals decrease as
system matures - MS B is a true hard stop for systems
36Cost Estimating MethodsAnalogy Method
- Based on direct comparison with historical
information of similar existing activities,
systems, or components. - Compares new system with one or more existing
similar systems where there is accurate cost and
technical data. - Analyst must show validity of comparison.
37Cost Estimating MethodsAnalogy Method
- Based on known costs of a similar program
- Adjustments for complexity, technical, physical
- Strengths
- Based on representative experience
- Less time consuming than others
- Can be used as a check on other techniques
- Weaknesses
- Small sample size
- Heavy reliance on judgment
- Sometimes difficult to identify analogy and
associated costs
38Analogy Estimating with Factors
Cost(New) Cost(Old) x Adjustment Factor
Element Old Sys1 Old Sys2 Old Sys3 New Sys
Airframe 500/lb 250/lb 750/lb 1.25S1
Engine 2M/Unit 3M/Unit 5M/Unit .8S3
Avionics 3K/lb 2K/lb 4K/lb 1.0S2
Payload 6M/Unit 8M/Unit 7M/Unit .65S1
39Cost Estimating MethodsParametric Method
- Known as Statistical Method or Top Down Method
- Relates cost to physical attributes or
performance characteristics - Uses database of elements from similar systems
- Uses multiple systems
- Most beneficial in earlier stages of the system
or project life cycle
40Cost Estimating MethodsParametric Method
- Statistical relationships between cost and
physical or performance parameters of past
systems. - Strengths
- Captures major portion of cost
- Quick what if type estimates
- Weaknesses
- Less detailed
- Getting accurate data
41Cost Estimating MethodsParametric
Method(Extrapolation)
- Use historical values to establish a trend for
the future. - Example problem Given the actual productivity
and labor rates in the given table. How much
will it take to complete a 3-year software
development project of 10K lines of code, if 50
is completed in the second year and 25 is
completed in first and third years?
42Cost Estimating MethodsEngineering Method
- Known as bottom up method
- Requires extensive knowledge of system
characteristics - Divide into segments estimate costs for each
segment - Combine segments plus integration cost
- Uses a combination of cost estimating methods
- Detailed knowledge of new technologies may not be
available.
43Cost Estimating MethodsEngineering Method
- Strengths
- Detailed
- Best when long stable production process
- Weaknesses
- Requires a lot of time
- Cost
- Cannot be used until system well defined
44Cost Estimating MethodsExpert Opinion Method
- Subjective judgment of an experienced individual
or group - Use if time does not permit a more thorough
analysis - Document source(s) of opinion of experts
- List attributes of the source(s)
- Example Delphi Technique
45Cost Estimating MethodsExpert Opinion Method
- Consulting with one or more experts who use their
knowledge and experience to arrive at an estimate - Group techniques include
- Consensus (Committee)
- Delphi
- Strengths and weaknesses
46Expert Opinion MethodDelphi Technique
- Query expert opinion from group
- Seek information from each expert
- Summarize the results
- Send report to each expert
- Gather second opinion after each individual
reviews report - Summarize results
- Iterative process continues until the experts
reach a consensus, or near-consensus.
47Expert Opinion Method Example
Labor type Hours Needed Hourly Rate of Total Hrs rate
Senior Engineer 1000 13 10.5 1.37
Design Engineer 3000 11 31.6 3.48
Tool Die 500 11 5.3 .58
Machinist 5000 9 52.6 4.73
Totals 9500 10.16
48Learning Curve Theory
As the quantity of a product produced doubles,
the man-hours- per-unit expended to produce the
product decreases at a fixed rate or constant
percentage (usually 10 to 15).
49Learning Curve Theory Factors Contributing to
Efficiency
- Job familiarization by both production workers
and supervisory personnel. - Changes in product design which do not
materially affect the product, but result in
increased ease and speed of production. - Changes in tooling, machinery, and equipment
which simplify or speed up the production
process. - Improved production planning and scheduling, and
improvements in production techniques and
operational methods. - Improvements in shop organization, engineering
coordination and liaison. - Improvements in the handling and flow of
materials, and in the materials and parts supply
systems
50Learning Curve Theory
The table is based on the assumption that the
first unit required 100 person-hours to produce.
The table indicates a constant rate of reduction
of 20 for each doubling of the unit number the
value of the second and each succeeding item in
the table is 80 of the value of the preceding
item.
51Learning Curve Theory
80 Unit Curve on Arithmetic Paper
52Learning Curve Theory
53Learning Curve TheoryUses
- Evaluating contract production costs.
- Assessing impact of production interruptions,
product changes and production rate change. - Rate of improvement experienced by a particular
contractor on a prior product may be indicative
of rate of improvement expected on new product
of similar size, complexity, and construction. - Improvement curve pattern experienced in the
production of past item can be extended to
calculate costs of future items. -
54Estimate Formulation
- We have defined our tasks, planned the estimate,
assigned cost responsibilities, and performed
data collection and analysis. - Here we apply our estimating methodologies and
tools develop the factors, analogies, CERs, and
learning curves. - We will aggregate the various cost elements into
development, production, and OS estimates,
fiscally spread the costs, and apply inflation.
55Cost Estimating MethodsLaying Out the Estimating
Approach
- Aircraft System sum of level II elements
(cross-check with analogy) - Air Vehicle sum of level III elements
- Airframe CERs
- Air Vehicle Software Expert Opinion
- Propulsion CERs
- Avionics Analogy
- Armament Catalog Price
- System Eng/Program Mgt Factor of Air Vehicle
- System Test Evaluation Factor of Air Vehicle
- Training Factor of Air Vehicle
- Data Factor of Air Vehicle
- Peculiar Support Equipment Factor of Air
Vehicle - Initial Spares Factor of Air Vehicle
56Review and Presentation
- We want to ensure that the estimate is
reasonable, realistic, and complete. - Reasonableness addresses areas such as using
appropriate and acceptable methodologies
presenting methodologies systematically the use
of relevant data and ensuring that assumptions
are valid and clearly stated.
57Review and Presentation (Continued)
- The realism test checks to see if the assumptions
and ground rules are consistent with the
statement of work and if the costs are in line
with historical data. - We evaluate completeness by determining whether
the estimate includes all the work stated in the
request for proposal and whether the costs are
traceable and reconcilable.
58Risk and Uncertainty Analysis
- Detailed Network Stochastic
- Discrete Technical, Schedule, and Estimating
Risks
- Detailed Monte Carlo Simulation (each WBS)
Effort
- Bottom Line Monte Carlo Simulation
- Add a Risk Factor/Percentage
Detail
59Risk and Uncertainty Analysis
- Probability Distributions
WBS Cost Distribution
Total Cost
60Risk and Uncertainty Analysis
- Probability Density Function
61Risk and Uncertainty Analysis
- Cumulative Distribution Functions
62Reasons for Risk
63Cost Schedule Curve
Fixed Cost Technology Outdate
Parallel Effort More ECP Less Mature Design
Life Cycle Cost
Typical
Min
Optimal
Development Schedule
64Development Schedule
Mean Cycle Time to IOC
DoD Pre-1992 Starts 132 months (11
years) Post-1992 Starts 89 months (7.4
years) on-going F-22
216 months (18 years) IOC 2004 Comanche
264 months (22 years) IOC
2006 Commercial Boeing 777 54
months (4.5 Years)
65Schedule Goals
- Commercial Drivers
- Technology Drives Schedule
- Constraint Schedule
- Goal is ROI Maximization
Defense
Cost
- DoD Drivers
- Funding Drives Schedule
- Unconstrained Schedule
- Goal is Cost Reduction
Commercial
Development Schedule
66DoD Program Schedule Drivers
Funding Allocation
Cost
Acquisition Process
Program Execution
Cycle Time Reduction Goal
Development Schedule
67Final Documentation
- Provide the means for other analysts to get the
same results that we have in our cost estimate. - Providing good directions and a clear trail to
follow are essential in having an estimate that
can be replicated. - Provide step-by-step documentation of the
methodologies, supporting data, ground rules, and
assumptions, equations, examples, etc., - Ability to interpret or evaluate someone elses
cost documentation is as important as ability to
prepare good cost documentation.
68Life Cycle Management Changes
Current State
ICD
Multiple bill payer drills to offset actual costs
Defined, Refined, Fairly Accurate
Program forEquipment
ST
RDTE
Initial Cost Estimate
Revised Cost Estimate
Disposal
Revised Cost Estimate
MS A
MS B
MS C
SRR
Procurement
OPA
MCA
Construction
MPA
Personnel
Training
OS Cost Estimate
Program forOS Costs
Operations
OMA/MPA
Support
Only compete procurement costs against Army
Investment Strategy over the EPP. Impact of
sustainment on Army program ignored.
Develop SSP
Unbalanced process that emphasizes procurement
over sustainment Large unplanned costs
Future State
ICD/DOTMLPF Change Request
Periodic Cost Estimate
Develop Supportability Sustainability Plan
Program forEquipment
ST
RDTE
Revised Cost Estimate
Disposal
Initial Cost Estimate
Revised Cost Estimate
MS A
MS B
SRR
OPA
Procurement
FOC
MCA
Construction
Capabilities Based Programming
Personnel
MPA
Plan for Sustainment
Training
Compete total cost against Army TOA extended over
EPP.
Operations
OMA
POM for Sustainment
Support
Balanced process that considers investment and
sustainment Informed decision makers
69CAIV Process
Set realistic but aggressive cost objectives
early in acquisition program
Manage risks
Track progress using appropriate metrics
Motivate Government and industry
managers to achieve program objectives
Incorporate incentives to reduce OS costs for
fielded systems
70Cost As An Independent Variable
(CAIV)
- Best time to reduce cost is early in the process.
- Involves the stakeholders in the process.
- Cost tradeoffs must be addressed early in the
acquisition process and embedded in program
requirement documents, Request For Proposals
(RFPs), contract provisions, and source selection
process.
71Cost Analysis Domain
Types of Cost Studies
Performance Based and Design Based Cost Models
Currently do not Exist
72Analysis of Alternatives
Set realistic, aggressive cost objectives
early in development
Manage risks
Track progress with appropriate metrics
Motivate government/industry managers to achieve
program objectives
Incorporate incentives to reduce OS costs for
fielded systems.
73The Coster's Challenge
Cost Analysis Art or Science?