SOFTWARE CONTRACTING

1
SOFTWARE CONTRACTING

• MN3309 Session 9
• Prof. Mark Nissen

2
Agenda

• Key Contracting Needs
• Difficulties Approaches
• Contracting Exercise
• Key RFP Elements
• Software CMM
• Summary
• Lots of hidden slides

3
Key Contracting Needs

• COM DoD?
• Select capable contractor
• Document requirements effectively
• Incentivize desired behavior
• Arrange for necessary info
• Guard against problems
• Strike good business deal
• Follow regulations

4
S/W Contracting Difficulties

• Nature of software
• Invisible, intangible, abstract, creative
• ID requirements? prove correct?
• Estimate measure?
• Management inexperience
• PMOs prime contractors
• Immature processes (PMO COM)
• Incredible project baselines
• Hard to anticipate problems

5
S/W Contracting Difficulties

• Agency Difficulties
• Principal/Agent problem
• Asymmetric information
• Incentives to shirk conceal
• Incentives to transfer risk
• Other difficulties?

6
Conventional Contracting Approaches

• Fixed-price contracting
• Nature? S/W difficulties?
• Cost-reimbursement contracting
• Nature? S/W difficulties?
• Alpha contracting
• Nature? S/W difficulties?
• Other conventional approaches?

7
Other Contracting Approaches

• Other transactions (OTAs)
• Nature?
• Advantages disadvantages?
• Schedule-plus contracting
• Nature?
• Advantages disadvantages?
• Contingency contracting
• Nature?
• Advantages disadvantages?

8
Other Contracting Approaches

• Modular contracting
• Nature? Advantages disadvantages?
• Options pricing
• Nature? Advantages disadvantages?
• Puts and calls?
• Competitive procurements
• Nature? Advantages disadvantages?
• Joint ventures
• Nature? Advantages disadvantages?

9
Contracting Exercise

• Walk with book on head (flat)
• 5 times (min) across room, round trip
• No hands, pins or drops (back to Start)
• Project groups
• Estimate steps, seconds, drops, hands
• 2 proposals from each team (10 if win)
• FPIF 5 each R/T -10 each drop -/1 each step
  or second over/under target (no max)
• CPIF 2 each R/T -0 each drop -/1 each step
  or second over/under target (10 max)
• Which is best for com? for PMO?

10
Key S/W RFP Elements

• Software development plan (SDP)
• Plan for project execution
• Assess contractors understanding
• Capability (potential)
• Maturity (CMM/SCE)
• Assess contractors processes
• Past performance (track record)
• Experience very important
• Reputation is key to com firms

11
Key S/W RFP Elements

• Documentation formats
• Use contractor formats?
• In-place data access?
• Baseline control
• Contractor control?
• At some level, up to a point?
• Technical visibility
• Measure technical progress?
• How to ensure accurate reporting?

12
Key S/W RFP Elements

• Risk Management
• Collect informative metrics
• ID and manage key risk-exposures
• Starts with source selection
• Key personnel
• Number, buildup, experience, skills
• MITRE skills matrix
• Data rights, COTS considerations
• Others (STSC has in detail)

13
SEI Capability Maturity Model

• Basic CMM for software
• 5 levels of S/W process maturity
• Initial (chaotic, heroics)
• Repeatable (program management)
• Defined (standard S/W process)
• Managed (process measurement)
• Optimized (feedback for CPI)
• Each level is a binary gate
• Focus on Key Process Areas (KPAs)

14
CMM Framework
15
Key Process Areas

• Level dependent
• Repeatable
• Requirements management
• Project planning
• Project tracking oversight
• Subcontract management
• SQA
• Configuration management

16
S/W Capability Evaluation

• Strengths/weaknesses wrt KPAs
• Eval of S/W contractors processes
• Organization resource management
• S/W engr process management
• Tools techniques
• S/W development expertise
• Analysis of multiple programs
• Organizational level evaluation
• Often part of source selection criteria

17
Maturity Cost Risk
Lower target narrower risk profile
18
S/W Acquisition CMM

• Five levels
• Initial - no one is well-versed in the
  specialized acquisition of S/W, nor is anyone
  dedicated to managing its acquisition
• Repeatable - the program team is knowledgeable
  of policies, regs, standards related to
  S/W-intensive aspects of program new program
  planning and tracking is based on experience with
  similar programs
• Defined - standard S/W acquisition process is
  defined integrated for both contract program
  management org-wide training is implemented
  management has visibility in to technical program
  progress

19
Summary

• Software contracting is difficult
• Many contracting approaches
• Agency influence over behavior
• Efficacy begins with RFP
• CMM offers some insight
• Must select manage good contractor

20
ROI Exercise

• 3 alternative approaches A, B, C
• Approach A
• Spend 1M, ROI 101 over 6 years
• Approach B
• Spend 1.5M, ROI 81 over 8 years
• Approach C
• Spend 1M, ROI 101 over 8 years
• Which is best?

21
Cum Returns (nominal)
Breakeven
22
Cum Returns (NPV, i5)
Breakeven
23
Cum Returns (NPV, i10)
Breakeven
24
RFP Preparation

• Overview
• Contracting Process
• RFP Preparation Process
• Uniform Contract Format
• RFP Items to Include
• IVV, SOW, S/W-specific issues

25
RFP Preparation Process
26
Contracting Process
27
RFP Preparation - Overview

• Culmination of acquisition planning
• Formal comm of rqmts to offerors
• Contractor(s) part of IPT
• Early involvement key
• Competitive issues?

28
Uniform Contract Format(see FAR 15.406-1)
A Solicitation/contract form
B
Supplies or services and prices/costs C
Description/specifications/work
statement D Packaging and marking
E Inspection
and acceptance F
Deliveries or performance
G Contract administration data

29
Uniform Contract Format(see FAR 15.406-1)
H Special contract requirements
I
Contract clauses
J List of
attachments
K Representations,
certifications,
and other statements from
offerors L Instructions, conditions,
and notices M
Evaluation factors for award

30
RFP Items to Include

• Clear, concise statements of specific tasks
  (quantifiable, measurable, and testable),
• Specifications and standards tailored to program
  needs (relying on commercial standards and
  practices, whenever possible),
• Planned use of government-furnished equipment
  (GFE), government-furnished information (GFI),
  and/or government-furnished software (GFS),
• Requirements for the contractor to provide a
  comprehensive layout of program schedules
  (internal reviews, formal peer inspections,
  testing, technical meetings, etc.),
• Requirements for relevant and pertinent domain
  experience,
• Requirements for a thorough Software Development
  Plan (SDP) discussed in Chapter 14, Managing
  Software Development and plans for its
  implementation and updates, to include a proposed
  test process plan,

31
RFP Items to Include

• Requirements to describe use of Ada, Ada SEEs,
  COTS, and reuse,
• Requirements for appropriate software
  documentation,
• Requirements for an open systems architecture and
  architecture performance verification,
• Requirements for resources growth/margin
  verification,
• Requirements for a total life cycle/total systems
  perspective,
• Requirements for prototyping and/or
  demonstrations (ideally, a demonstration of an
  executable architecture as part of the proposal),
• Requirements for a progress, process, and quality
  measurement program, including a specific Metrics
  Usage Plan,
• Requirements for a software quality assurance
  (SQA) program

32
RFP Items to Include

• Requirements for a Risk Management Plan and its
  implementation,
• Requirements for a Process Improvement Plan and
  its implementation,
• Requirements for a process control mechanism
  e.g., Process Weaver (discussed in Chapter 10,
  Software Tools) or equivalent,
• Requirements for developing interface software
  with other system software and/or hardware,
• Requirement for assessing software development
  maturity/capability,
• Planned communications with any IVV contractors
  or agencies
• Low page count, streamlining com practice?

33
IVV

• Independent Verification Validation
• Independent of development org
• Verification that each development step meets
  requirements, follows procedures
• Validation that completed system meets system
  requirements, satisfies needs
• Used for
• Real-time, critical S/W (FBW, GBM, crypto)
• High-development -maint cost S/W
• Evaluate as a risk-mgmt investment

34
Statement of Work

• Translate performance requirements into
  contractual tasks
• Tie-into summary WBS
• Offeror uses to prepare contract WBS
• Tasking information for CDRLs
• Says what to do, not how to do it!
• Is SOW even required?

35
Software-Specific Issues

• S/W development process
• Contractor documentation/formats
• Contractor control of baselines
• Direct technical visibility
• Proactive risk management
• COTS reuse
• IVV data rights

36
COTS Considerations

• COTS advantages
• Cheaper, better, faster, more reliable
• Easy to procure, competition
• COTS disadvantages
• Designed for broad, com market
• Products change frequently
• Contractors come go
• Mod support can be troublesome

37
COTS Lessons Learned

• The customers willingness to pay to gain the
  benefits of COTS must be understood.
• Metrics for COTS integration are difficult to
  extract.
• While COTS are less expensive than developed code
  for large, general applications (e.g., operating
  systems, DBMS), experience does not indicate this
  is true for smaller, niche products.
• Savings are anticipated in FQT and OM. However,
  the customer is not always willing to be flexible
  enough to achieve reduced costs from COTS.
• Non-technical COTS selection criteria are as
  important as the technical. These include
  vendor stability product cycle availability of
  support for back-level releases and willingness
  of the vendor to work with the contractor.
• Managing product licenses can be a big headache.
  Therefore, get as flexible a set of licensing
  terms as possible

38
COTS Lessons Learned

• Do not believe what you read fly before you
  buy!
• Understand prerequisites and dependencies between
  products, especially when planning upgrades.
• Make sure procurement processes are in place to
  expedite COTS delivery.
• Establish hardware and COTS product baselines
  early.
• Work with the customer to negotiate modifications
  to documentation requirements for COTS products.
• The cost of integrating COTS is much more
  front-end loaded than the cost of developing
  code.
• Spend as little time as possible on paper trade
  studies. Get products to the lab for prototyping
  and integration. RAND95
• Do not purchase COTs with the intention of
  modifying it!

39
Data Rights

• COTS vs. developmental S/W
• S/W is technical data
• Gov position?
• Com position?
• Three categories of data rights
• Unlimited rights
• Limited rights
• Restricted rights
• Life cycle supportability

40
Contract Types

• Types of Government Contracts
• Contract Type f(risk)
• Maturity Cost Risk
• Contract vs. Program Contractor Risk
• Contract Type Program Phase
• Cost-Fee/Profit Graphs

41
Types of Government Contracts

• Contract types vary according to
• The degree and timing of the responsibility
  assumed by the contractor for the costs of
  performance.
• The amount and nature of the profit incentive
  offered to the contractor for achieving or
  exceeding specified standards or goals.
• 2 General Types
• Fixed price
• Cost reimbursement

42
Fixed Price Contracts

• Contractor delivers products or services.
• Government pays a firm price.
• No price adjustments (except economic price
  adjustment).
• Maximum risk to contractor.
• Incentive for cost control.
• Definite specification.
• Certainty of total cost.

43
Cost Reimbursement Contracts

• Government reimburses contractor for costs.
• Government pays a fee.
• Fee limited (RD 15, Supply 10).
• Minimum risk to contractor.
• Contractor is required to do best efforts.
• For RD and complex programs.

44
Contract Type f (risk)
0
100
GOVERNMENT
FFP
FPEA
FPR

FPIS
RISK ASSUMED BY GOVERNMENT
FPIF
RISK ASSUMED BY CONTRACTOR
COST SHARING
CPIF
CPAF
COST
CONTRACTOR
CPFF
100
0
3301KRR1
45
Contract vs. Contractor Program Risk

• Contract risk
• Incentivize performance, share risks
• Overrun --gt reduced profit/fee
• Contractor risk
• Contract performance --gt bankruptcy
• Deliver minimum, contractually-required
• Program risk
• Cost exceeds program baseline
• Request additional funds, risk cancellation

46
Contract Type Program Phase
47
Cost-Fee/Profit Graphs
100/0
0/100
100/0
Share lines represent Govt/Contractor risk
assumption
0/100
3301KRFE
48
Source Selection Process
49
Source Selection

• SSEB
• SSA
• CRWG
• RFP sections L M
• Evaluation factors
• Valid discriminators
• Traceable to requirements
• Technical, management, cost

50
Technical Factors

• A well-written, comprehensive draft Software
  Development Plan (SDP),
• An approach towards management of an evolutionary
  requirements process,
• An understanding of CSCI control and management,
• An understanding of the technical/functional
  elements comprising the software they propose to
  build,
• A robust Ada development environment and
  applicable prior domain experience,
• Adoption of DoD-sanctioned SEEs and development
  tools, or equivalents,
• Automated process control software (or
  equivalent),
• A plan to comply with DoD open systems standards,

51
Technical Factors

• Proposed reuse of both program-developed and
  Government-approved software assets,
• A plan for training development team users,
  maintainers, and software professionals (as
  needed),
• A plan for risk identification, assessment and
  management,
• A plan to produce current source code
  documentation mirroring delivered code,
• A plan for clear, up-to-date user documentation,
• A plan for compliance with each CDRL,
• A plan to deliver data in DoD-approved format,
• A comprehensive SQPP,
• A plan for tailoring the use of MIL-STD-498 (if
  its use is proposed),

52
Technical Factors

• An understanding of the interface and
  interrelationships between the software and
  hardware,
• Acceptance criteria for software deliverables
  that includes a criterion for program reliability
  expressed as MTTD at the time of delivery,
• Adequate software test resources and a plan for
  their application,
• A plan to incorporate COTS software and reuse to
  meet overall software needs,
• The proposal of a specific architecture(s) in
  executable Ada code
• A plan to provide an Ada-driven demonstration as
  part of their proposal,
• A plan to constantly improve the software
  development process (SPIP), and
• A high score on the SCE or SDCE (a Level 3 or
  better).

53
Management Factors

• A Software Development Plan (SDP),
• A current SEI Level 3 or SDCE equivalent rating,
• Relevant domain experience,
• A satisfactory Skills Matrix,
• Provision for on-line access to their development
  environment,
• Provision for on-line access to their process
  control tool,
• Description of peer inspection usage and extent,
• Corporate quality control processes and plans,
• Corporate management commitment to the proposed
  program,
• A plan to make supportability a software
  development priority,

54
Management Factors

• Plans and procedures for compliance with proposed
  reviews and audits,
• Metrics Usage Plan, purposes for which metrics
  will be used, and experience and knowledge with
  government-specified metrics and indicators,
• Software development scheduling tools and
  schedule realism, importance, and detail,
• Bonus points for a status room approach to
  software development and Government visibility
  (and/or on-line government access to the SEE),
• A comprehensive Subcontractor Management Plan,
  and
• An agreement to unlimited data rights for
  delivered software.

55
Cost Factors

• Low cost vs. best value
• Comparison with budget
• Realism problem understanding
• Buy in
• Reuse, maturity, tools
• Time phasing
• Contract type clauses

56
PROTESTS
PROTESTER MAY SEEK REVIEW OF ADVERSE DECISION
PROTESTER MAY SEEK REVIEW OF ADVERSE DECISION
GAO ENCOURAGES PROTESTER TO FIRST PROTEST TO
AGENCY
FEDERAL COURTS
CONTRACTING AGENCY
PROTESTER MAY PROTEST DIRECTLY TO COURTS, AGENCY,
GAO
3301KRPT
57
Protest Safeguards

• Be fair equitable
• Document well
• Prepare sections L M carefully
• Ensure evaluation factors discriminate
  effectively, are few in number, are traceable
  to requirements
• Adhere to section M in selection
• Prepare for sell debriefs
• Process streamlining?