Acquisition Cost Estimating Update 28 Feb 08

1
Acquisition Cost Estimating Update28 Feb 08
Air Armament Center
War-winning CapabilitiesOn Time, On Cost
Ken Kennedy, AAC/FMC Kristy Golden, AFCAA/FMAE
DISTRIBUTION STATEMENT A Approved for public
release distribution is unlimited.
1
2
Outline

• Estimating Trends
• Leadership Concern
• Congressional Reaction
• Root Cause Analysis
• 2007/2008 Initiatives
• AAC Cost Demand
• AFCAA Standup
• Industry Help
• Summary

3
Estimating Trends

• GAO-07-406SP Defense Acquisitions Assessments of
  Selected Weapon Programs
• GAO assessed 62 weapon systems with a total
  investment of over 950 billion, some two-thirds
  of the 1.5 trillion DOD plans for weapons
  acquisition
• Fully mature technologies were present in 16
  of the systems at development start - the point
  at which best practices indicate mature levels
  should be present.
• Programs that began development with immature
  technologies (84) experienced a 32.3 percent
  cost increase, whereas
• Those that began with mature technologies (16)
  increased just 2.6 percent.

3
4
Leadership Concern

• Sue Payton Jane's Defence sic Weekly,
  07/17/2007
• Reduce pressure on budget by improving methods
  for calculating the overall cost of acquisition
  programme sic
• Stop taking contractors' one-off cost estimates
  at face value
• Develop own methods for calculating the true,
  long-term cost of a weapon system
• Tendency to highlight the basic price tag of a
  weapon system and play down the much bigger
  'lifecycle cost
• Wait until a preliminary design review has been
  completed, rather than trying to guess a weapon's
  price before the final requirements are defined
• Push contractors to build more prototypes
  important insight into future development costs

4
5
Congressional Reaction

• Nunn-McCurdy Breach Thresholds
• 2006 Defense Appropriations Act (Public Law No
  109-148)
• Requires Increased emphasis on
• Accurate TRL assessments
• Quantification of risk and uncertainty
• Optimal decisions within trade space constraints
• Better cost and schedule forecasts
• Budgeting to Independent Cost Estimates

5
6
Root Cause
Range of Missile Program Cost Outcomes
AOA
Risk

Reduction
SDD
LRIP
FRP
We are too optimistic about risk/uncertainty
Risk/Uncertainty
Cost risk levels
Extremely High
Extremely High
Low
Low
Med-Low
Med
Med-High
High
Med-Low
Med
Med-High
High
1
2
3
4
5
6
7
8
9
TRL Level
Risk is retired as a program progresses and
technology matures
7
2007/2008 Initiatives

• Better Risk ID and Modeling
• Continued Sufficiency Reviews
• New Source Selection Focus
• More Contract Risk Sharing

8
Better Risk ID and Modeling

• Integrate Probability of Consequence Screening
  (P/CS) risk cube and Probability of Program
  Success (PoPS) assessment into cost estimating
  process
• Existing infrastructure tools
• Realistic and consistent approach
• Results in a time now WBS risk rating
• Use Monte Carlo simulation with WBS correlation
  to define total program cost
• Present decision maker with better picture of
  uncertainty based on knowledge today

9
Continued Sufficiency Reviews

• Cost gray beard review
• Reinforce estimating best practices
• Verify data sources and methodology
• Suggest alternate benchmarks
• Request appropriate SME inputs
• Independent technical assessment
• Independent schedule assessment

10
New Source Selection Focus

• Cost risk 1 evaluation priority
• High, medium, or low rating based on percent
  difference between proposed contractor cost and
  government estimate of offerors approach
• Requires clear communication
• Full data source ID and extract for element basis
  of estimates (BOEs)
• Full data source adjustment justification in
  element BOEs
• Full discussion of risk and uncertainty in
  element BOEs

11
More Contract Risk Sharing

• Solicitation and proposal
• Request WBS element uncertainty/risk and
  magnitude as part of each narrative basis of
  estimate worksheet
• Request sum of element risk to be included as a
  separate proposal line
• Evaluation
• Description of element uncertainty used to model
  risk for independent estimate
• Execution
• Proposal risk line becomes earned value MR
• SOO requirement for discussion before use

12
Air Force Cost Analysis Agency (AFCAA) Standup
13
Organization Headquarters
THE ASSISTANT SECRETARY OF THE AIR
FORCE (Financial Management Comptroller)
DEPUTY ASSISTANT SECRETARY (Cost Economics)
DEPUTY ASSISTANT SECRETARY (Budget)
DEPUTY ASSISTANT SECRETARY (Plans, Systems
Analysis)

• Deputy Assistant Secretary (Cost Economics),
  AFCAA
• Policy and functional oversight
• Major program milestone decisions -- Independent
  Cost Estimates (ICE) for delegated programs
  (required by law) Component Cost Analysis (CCA)
  AF CAIG service cost position (SCP)
• Headquarters/functional support (analysis,
  review, etc.)

14
Deputy Assistant Secretary(Cost Economics)
SAF/FMC
Deputy Assistant Secretary for Cost Economics
(SAF/FMC) Executive Director (AFCAA/FM) Mr
Hartley, SES Associate Deputy Assistant
Secretary for Cost Economics Ms Woods, SES
AFCAA Technical Director Mr. Jordan
Space Programs Division (AFCAA/FMS) Mr. Seeman
Info Tech Division (AFCAA/FMI) Mr. Moul
Force Analysis Division (AFCAA/FMF) Lt Col
Hurley
A/C Weapons Division (AFCAA/FMA) Mr. McVicker
Resources Division (AFCAA/FMR) Ms. Cann
Integration Division (AFCAA/FMC) Col DuPre
ASC/AAC OL
AFSCP/ SMC OL
ESC OL
The Deputy Assistant Secretary holds two titles
as SAF/FMC and AFCAA/FM
15
AFCAA Eglin OL
16
Operating Locations
Hanscom AFB MA
AFCAA - DC
Wright-Patterson AFB OH
Peterson AFB CO
Los Angeles AFB CA
Eglin AFB FL
17
Air Force Concern
There is a Cost Growth Problem. Cost Estimating
is a Significant, but not Lone, Contributing
Factor. However, it is a factor we can manage!

• Current AF-wide policy does not consistently
  require adequate, objective cost analysis or
  review
• Limited to infrequent milestone reviews
• Inconsistent leadership value placed on cost
  analysis (contrast DAB to AFIPT)
• Cost resources down 60 since 1992 --
  approximately 400 positions!
• Staffs hit hardest, no career path/pipeline,
  compare poorly with Army, Navy, others
• Little functional governance
• Inexperienced analysts isolated in programs
  with little functional support -- fractured
  estimating methods, databases, models --
  inconsistent data/estimate quality, analyst
  qualifications

18
Acquisition Cost Capability Initiative
Improve Air Force Cost Estimating

• Better cost estimating processes
• Policy and standards
• Re-writing AFPD 65-5 and accompanying AFIs and
  AFMANs
• Building standards and templates
• Performance metrics
• Collaboration
• Encourage team efforts
• Establish operating locations at Product Centers
  and MAJCOMs
• Transparency
• Share data / research / methods
• Repopulate personnel pipeline
• Upgrade education, training, and certification
  programs
• Provide Cost Estimators with the tools to succeed

19
Current Status

• Policy AFD 65-5
• The acquiring organization shall build program
  cost estimates
• Annually
• Milestone decisions
• Nunn-McCurdy Cost Breaches
• Source Selections
• POM initiatives, disconnects, and offsets
• Headquarters shall provide non-advocate cost /
  risk assessments to decision makers
• Collaboratively develop and execute training,
  data collection/methods and model development
  programs
• Headquarters shall review cost, economic, or
  business case analyses to be presented to SECAF,
  USECAF, CSAF, VCSAF
• Personnel
• Hired 20 positions
• Operating Locations
• Completed beddown agreements at Peterson AFB,
  Wright-Patterson AFB, Eglin AFB, Hanscom AFB, and
  LA AFB
• Working Host Tenet Support Agreement at each
  location
• Training and Certification
• Partnered with SCEA and DAU to improve
  certification program
• Developed Training OI for new analysts

Led by government employees SAF/FMC defined
standards and procedures
20
AFCAA OL Mission
Shorten Kill Chain (Cost Budget)

• AFCAA-OL will function as the liaison with the
  Center
• Work collaboratively with the program office
  estimators
• Non Advocate Cost Assessments (NACAs)
• Independent Cost Estimates (ICEs)

21
How the AFCAA-OLs will Help

• Collaborative program cost estimates
• Build cost models
• Cost estimating relationships
• Risk analysis
• Data / research / methods
• Potential roles
• Team lead
• Team member
• Sufficiency reviewer
• Consultant
• Researcher
• Facilitator between Program Office and DCARC (OSD
  CAIG)

22
Industry Help

• Dont over promise capability with unrealistic
  resources
• Define and quantify uncertainty for your proposed
  concept
• Provide true risk now versus what it could be in
  a future year
• Identify the right level of MR to neutralize
  problems before they overwhelm the program

23
Summary

• Cost estimating community needs to provide more
  realistic resource projections for Congress and
  DoD
• Additional AFCAA resources are in place now with
  more FMC positions coming to help programs
• New initiatives are in work to improve estimate
  quality, quantify uncertainty, and fund
  appropriate risk efforts