Quality Assurance Task Interagency Agreement with Dept' of Navy

1
Quality Assurance TaskInteragency Agreement
with Dept. of Navy

• Navy Public Works Center conduct three (3) QA
  tests at three DOE sites.
• First three sites will be non-CAIS users.
• SLAC, ANL-E and SRS (EM NNSA).
• First test underway November 18.
• Inspect an admin., storage and lab asset.
• SRS inspect 4 assets.. 2 from EM NNSA.

2
Quality Assurance Task

• Purpose of Task
• are sites using industry standards as their basis
  for inspections.
• Evaluate the effectiveness, completeness and
  accuracy of site cost estimating systems to
  estimate deferred maintenance costs.

3
Quality Assurance Task

• Task is initially funded by OECM.
• NNSA and SC show interest for additional QA.
• Future task will include QA of CAIS and
  remaining non-CAIS sites.
• When ..?
• (addition sites will require funds from PSOs.)

4
Quality Assurance TaskGeneral Methodology

• Sites select 3 assets.
• Assets inspected current year and no repairs
  undertaken.
• NPWC will use mech., elect., and civil to inspect
  assets.
• CAIS Team- Ken Rowe Chuck Siegfried.
• NPWC collect data in their software and CAIS data
  sheets.

5
Quality Assurance TaskGeneral Methodology
(cont..)

• Data compared to Site inspections.
• Comparison made between site, NPWC and CAIS
  costs.
• Detailed analysis made comparing all costs.
• Comparison made to base costs only.
• Analysis notes deficiencies not covered and cost
  differences in the three systems.

6
Quality Assurance TaskGeneral Methodology
(cont..)

• Analysis will discuss inflators or adjustments
  that make DMC more accurate.
• CAIS team members will enter inspection results
  in CAIS.
• CAIS team will study conversion possibilities of
  site database data to CAIS.
• Final Report issued two weeks after task
  completion.

7
Quality Assurance TaskGeneral Methodology
(cont..)

• Schedules
• SLAC .. November 18 November 22
• ANL-E.. 1st or 2nd week of December.
• SRS.. Late January 2003.
• Assessments last 1 week and assets should be from
  10-15,000 sq. ft.

8
Quality Assurance TaskDeliverables

• Descriptions of assets and inspected systems.
• List of site contractors deficiencies.
• List of NPWC deficiencies.
• Discussion of differences of methods and data.
• Comparison between the 3 costing system results.

9
Quality Assurance TaskDeliverables (cont..)

• Discussion of conversion effort from site
  estimating system to CAIS and NPWC systems.
• A list of inflators or adjustments that may make
  QA site DMC more accurate.
• Any other site QA volunteers?

10
Problems with Original CAIS

• Handheld was Extremely Inefficient
• Desktop Data Entry was Inefficient
• Many Cost Estimates were Bad
• Reports were Nearly Worthless
• Custom Reports were nearly Impossible
• UNIX System was a Terrible Idea
• System Administrator Costs were High

11
Original Problems (Continued)

• Deficiency Standards were / are Worthless
• Training only focused on software details
• Re-Inspection requires a new blank sheet.
• No Maint vs. Improvement Field
• Half-baked Need Year Field
• No Pick-list for Method of Correction
• Costing does not consider Method of C

12
Problems (Continued Again)

• No Export to or Interface with WO System
• No FIMS Interface

13
Reasons Not to Convert to CAIS

• Some old problems still remain.
• Other Systems may be Better
• Whitestone MARS / TEC
• ISES
• AME
• Contracted Insp with non-CAIS system may be less
  expensive.
• PS. Consider the benefits of changing
  contractors.

14
An Important RPAM Requirement

• To be compliant with RPAM non-CAIS systems must
  have unit costs updated annually.
• Escalators for Old Estimates are not acceptable.
  
• ISES system does not comply. But it should be
  okay to escalate 3 yr old estimates.

15
Problems with CAS / DM Reporting

• Scope and Type of Inspections
• ID all needs including improvements
• ID big ticket needs not needed until future years
  
• ID needs of all sizes for In-House and Contract
• Categorizing and Prioritizing Needs
• Not using three expert inspectors plus roof insp.
• Classifying needs as M or I.
• ID of Rehabs as a combination of DM, non-DM, and
  I.
• Was not performed when it should have been, or
  was scheduled to be.

16
CAS / DM Problems (Cont)

• Correctly Identifying the Need Year.
• DOE Guidance only says Use Optimal ROI.
• Possible Need Year Questions
• During what FY will we have to do the work?
• Do we need to correct this deficiency this FY in
  order to avoid conditions that are inadequate for
  mission accomplishment?
• During what FY do we expect mission failure if
  the work is not done?
• During what FY do we really, really, really need
  to do this work?

17
More Possible Need Year Questions

• Do we need to accomplish this work in the current
  FY in order to maintain or achieve conditions
  appropriate for conducting world-class research?
• Would it be a definite waste of taxpayer if
  this work was accomplished during the current FY?
  
• If there were a pot of HQ available to all
  sites, would you tell HQ that you do not need
  to accomplish this work during the current FY?
  (Give the to some other site that has needs.)
• All the above questions comply with DOE Guidance
  except the first one.

18
How to Fix DM Reporting

• Publish detailed HQ Guidance on M vs. I.
• Publish Additional HQ Guidance on the correct
  Need Year Question or Questions.
• Do the above very soon for use in 03 Inspections.
  
• Every Inspector and Multiple Reviewers Need to Be
  Totally Up-to-Speed on above.
• Contract the Inspections to have better, more
  comprehensive deficiency ID.

19
Experienced Based CAS Estimates are a No! No!
Should they Be a Yes! Yes!?

• None of the books correctly cover all or even
  most of the deficiencies identified.
• Those who say they use the books for DM estimates
  are not telling the whole truth.
• Estimates based on history are the best!
• Estimates based on expertise are often better
  than book estimates for complicated work..
• Estimates based on experience usually have
  sufficient accuracy (plus or minus 30) and are
  much easier and faster to generate.

20
Blank

• Blank
• Blank
• Blank
• Blank
• Blank
• Blank
• Blank
• Blank
• Blank
• Blank

21
Blank
22
Blank

• Blank
• Blank
• Blank
• Blank
• Blank
• Blank
• Blank
• Blank

23
Costs to be, or not to be, part of Maint
Reporting and MII Calculation

• Planners and Estimators
• Work Reception and Data Entry Clerks
• Training costs
• Computer Hardware and Software Costs
• Travel costs, eg. to EFCOG
• CAS Costs
• of Maintenance Manager and Staff

24
Color of Money Issues

• During the 1980s the Accounting Handbook said
  retirement units determine the color of money for
  replacement work.
• Sites were unable to accomplish needed work
  because of scarce GPP funds.
• CH was successful in getting a commitment to
  change the Accounting Handbook to fix above
  problems.
• Another change Alterations were classified as
  non-betterments to be funded with OE.

25
Color of Money (Cont)

• Another change, implemented by the old GPP Order,
  was that GPP funds could not be used for roof
  replacements and most other replacements.
• During the early 1990s there were mixed messages
  (Budget Examples, unchanged Handbook) and lack of
  continued emphasis on the above changes.
• Currently some sites are using GPP for roof
  replacements and similar replacement work..

26
Color of Money (Cont)

• According to the Current Accounting Handbook
• Replacements must be significant improvements to
  be betterments and GPP.
• Replacements must feature a very significant
  increase in capacity to be betterments and GPP.
  (Replacing a 200 ton chiller with a 300 ton.)
• Any significant improvement or very significant
  increase in capacity is not a betterment or GPP
  if the cost is less than 25,000.
• If the betterment portion of a combination
  project is over 25K, the entire project must be
  funded with GPP. (Therefore there is a valid
  reason to ID the maint )

27
Color of Money (Cont)

• Currently anyone can argue either way as to
  whether or not an improvement is significant and
  therefore to be GPP funded.
• Therefore, currently there is flexibility in
  deciding the color of money. Its a fielders
  choice. All decisions are okay.
• Upgrade is a common word to use to justify
  funding with GPP.
• Currently IGPP and PSO level re-programming from
  OE to GPP is possible.

28
Color of Money (Cont)

• If the Accounting Handbook was followed correctly
  and consistently, the maintenance portion of GPP
  would be a low .
• The Accounting Handbook is not clear and even
  contradictory. The Handbook needs improvement.
  Not changes, but more clearly stating what is
  currently intended.
• If a project must be OE funded and not GPP
  funded, another legal option is to make it a
  Line Item project. But Line Item funds are
  scarce.

29
Color of Money (Cont)

• SC-82 wrote, Maint may be funded with OE or
  Capital Funds.
• SC-82 wrote, Replacements are Generally Capital
  Funded.
• The above statements were not intended to change
  funding rules, or to grant permission, or to
  encourage use of GPP or LI for maintenance. They
  were intended to be statements that recognize
  past and current practices.

30
Color of Money (Cont)

• It is my understanding that currently SC-1 and
  SC-82 desire to fund most replacement work from
  OE, not from GPP or with LI.
• There is a desire to use GPP LI for Rehab and
  Replacement Bldgs (Recapitalization)
• If GPP funds are required for a project that the
  site really needs, IGPP or re-programming is a
  possibility.
• It is very important for Color of Money to not be
  a roadblock for any needed project.

31
Color of Money (Cont)

• I highly recommend identifying three categories
  DM based on planned funding OE or GPP or LI.
• I highly recommend identifying and reporting the
  amount and prioritized list of backlogged
  improvements. (This is an old Order requirement
  that has never been implemented.)
• We are over-focused on DM Maint funding and
  have little focus on improvement backlog and
  improvement funding..

32
Experienced Based CAS Estimates are a No! No!
Should they Be a Yes! Yes!?

• None of the books correctly cover all or even
  most of the deficiencies identified.
• Those who say they use the books for DM estimates
  are not telling the whole truth.
• Estimates based on history are the best!
• Estimates based on expertise are often better
  than book estimates for complicated work..
• Estimates based on experience usually have
  sufficient accuracy (plus or minus 30) and are
  much easier and faster to generate.