Process Principles for COTSBased Systems

1
Process Principles for COTS-Based Systems
Tricia Oberndorf

• 5 September 2003
• Software Engineering Institute
• Carnegie Mellon University
• Pittsburgh PA 15213
• Sponsored by the U.S. Department of Defense

2
Agenda

• COTS Background
• A/APCS Framework
• CMMI Background
• Implications of Using CMMI for COTS-Based
  Systems

3
COTS Background

• Building systems today
• Few entirely custom-built to order
• Commercial products expected to play major role
• Diverse things influence the shape and function
  of a COTS-based system (CBS)
• Stakeholder needs
• Product characteristics and marketplace dynamics
• Degree of interaction with legacy systems
• Together, these compel many changes in system
  development and maintenance methods for CBSs.

4
What is a COTS-based system?

• A COTS product is one that is
• Sold, leased, or licensed to the general public
• Offered by a vendor trying to profit from it
• Is supported and evolved by the vendor, who
  retains intellectual property rights
• Available in multiple, identical copies
• Used without modification of its internals

• A COTS-based system is one that
• Uses one or more off-the-shelf components from
  one or more suppliers plus any needed custom
  components
• Is integrated to achieve new or expanded system
  functionality

Legacy
COTS products
Custom
Reuse assets
5
Conceptual Bases Requirements1

• Nail down the requirements first will not work
  the marketplace will not cooperate
• Think of it as a new sphere of influence

6
Conceptual Bases Requirements2

• Requirements (cont.)
• Must distinguish between negotiable and
  non-negotiable
• Keep the non-negotiable set as small as possible
• Understand how to prioritize and trade-off the
  negotiables
• A risk-driven spiral approach is key
• Frequent use of prototypes
• Considerable interaction with systems end users
• Gradual refinement of understanding of system

7
Conceptual Bases Knowledge

• CBS development and maintenance is dependent on
  an evolving Body of Knowledge.

Non-negotiables
Prioritized negotiables
Marketplace offerings
Legacy system context
Evolving System View
Architecture and design constraints
End-user business processes
Acquisition constraints
Risks
8
Marketplace Affects CBS Approach
Requirements-driven
Negotiation-driven
9
The Process Framework

• Based on these realizations and concepts, we have
  defined a framework that sets out the basics of a
  process that would be appropriate for the
  creation of a COTS-based system
• A/APCS The Assembly/Acquisition Process for
  COTS-based Systems

10
Process Overview

• Three classes of activities
• Iterative short-term, engineering-oriented
• Discovery gather and refine system knowledge
• Assembly construct a prototype
• Assessment determine success of iteration plan
• Pervasive long-term, organizational scope
• e.g., CM, license management, vendor relationship
  management, contract tracking and oversight
• Executive event-driven, deal with decision
  making
• e.g., cost estimating, contract negotiation,
  project oversight
• Today we focus on the Iterative.

11
Discovery

• Gather and Refine activities occur
  simultaneously.
• Gather
• Sources take many forms.
• Stakeholders, business processes, legacy systems,
  COTS marketplace
• Each is independent of the others.
• There is no optimal order.
• Refine
• Analysis and harmonization of the gathered
  knowledge
• Yields the technical definition of the emerging
  system
• Finds gaps, negotiates conflicts

12
Discovery Activities
Knowledge is sufficient for constructing
executable
Continue Discovery
Harmonized data
Data from stakeholders products
Mismatch
Agreement
Refine
Gather
Gap - need more data
13
Assembly

• Produces a prototype that reflects whats been
  discovered
• Reflects a traditional sequential process
• Guided by local candidate requirements
• Iteration Objectives, Detailed Iteration Plan
  plus hypotheses and desired behaviors expect to
  derive from prototype
• Vets candidate requirements
• Produces an executable version of full system
• Likely to have less than complete functionality
• Executable, not paper or specification or small
  piece
• Does not preclude prototyping in the small
  throughout

14
Evaluation and Assessment

• Occur at two levels
• Evaluation of the prototype in its own context
• Measures actual outcome against expected outcome
• Considers degree to which prototype satisfies its
  local requirements
• Assessment of the iteration itself
• Addresses issues at project (not iteration) level
• Considers whether objectives were good ones
• Determines whether iteration results indicate
  changes in project schedule, budget, etc.

15
On-going Iterations of A/APCS
Discovery
Assembly
Prototype
16
Key Aspects of COTS Approach

• Simultaneous Definition and Trades
• Parallel Business Process Engineering
• Negotiable Requirements
• Flexible Architecture
• Current Market Knowledge
• Integral Programmatics and Risks
• Continuous Stakeholder Involvement
• Disciplined Spiral or Iterative Practices
• Frequent Executables

17
CMMI

• Guidance for improving an organizations
  processes and ability to develop, acquire, and
  maintain products or services
• Provides guidance for developing processes
• is not processes or process descriptions
• will not map one to one with the processes in
  your organization
• Integrates four disciplines (bodies of knowledge)
• System Engineering
• Software Engineering
• Integrated Product and Process Development
• Supplier Sourcing
• Expects all practices to be interpreted using
  knowledge of
• CMMI
• the organization
• the business environment

CMMI process areas must be interpreted for each
targeted domain
18
Presentation Format

• Important process considerations for a
  COTS-based system approach

• Work Process Implications
• High-level guidance for selected CMMI Process
  Areas or disciplines
• Highlight particular relevance
• Provide unique interpretation
• Suggest new process areas
• Correct misconceptions
• Guidance for additional CMMI Process Areas for
  later reference

Take-away message for each aspect
19
Simultaneous Definition and Trades

• Balanced consideration of four diverse spheres of
  influence
• - Simultaneous discovery of information in each
• - Decisions in one inform and constrain decisions
  in others
• - Continuous identification and analysis of
  trades
• - Trades negotiated as definition of the solution
  evolves

• Selected Work Process Implications
• Decision Analysis and Resolution continuous
  stakeholder negotiations require robust,
  agreed-upon decision processes
• Technical Solution alternative solutions
  (including COTS product selection) are analyzed
  continuously to accommodate new information
• Risk Management a key project risk is
  over-defining one sphere without adequate
  understanding of implications on other spheres
• Project Planning project activities start
  concurrently with extensive interaction among
  them from project start until the system is
  retired
• Verification and Validation continuous
  determination that information in each sphere is
  sufficient, complete, and meets operational needs

Continuously reconcile what stakeholders want
with what is available
20
Parallel Business Process Engineering

• End-users must be willing/able to change
  business processes to match those in COTS
  products
• Business process changes must be explicitly
  managed and coordinated as part of the project
• Business processes may continue to change with
  new COTS releases or new COTS products

• Selected Work Process Implications
• CMMI does not address changes to end-user
  business processes expand concepts in
  Organizational Process Focus to plan and
  implement end-user business process improvement
• Organizational Environment for Integration a
  shared vision of success among stakeholders, with
  suitable incentives and leadership, is critical
  to aligning business processes with alternative
  solutions
• Project Planning/Integrated Project Management
  for IPPD implementing agreed upon business
  processes must be integrated in system planning

Align business process engineering with system
engineering
21
Negotiable Requirements

• Requirements must be flexible enough to leverage
  available and projected COTS products
• Committing to a requirement is premature until
  COTS product behavior is understood
• As marketplace continues to change, requirements
  must be renegotiated

• Selected Work Process Implications
• Requirements Development must prioritize
  requirements to aid tradeoffs
• stakeholders agree on minimum set of critical
  must-have requirements
• evolvability is a high-priority, required quality
  attribute
• Requirements Management disciplined and
  controlled management of requirements begins at
  project start to track identified and negotiated
  trades
• Project Planning/Integrated Project Management
  for IPPD managing the project to encourage and
  reinforce the continual discovery of requirements
  while establishing sufficient stability to
  deliver a solution is challenging

Requirements formation is a journey of discovery
22
Flexible Architecture

• Architecture is considered early evolves and is
  maintained until the system retired
• Potential drivers of change accommodated in
  architecture

• Selected Work Process Implications
• Technical Solution each COTS product release
  requires evaluation and analysis of alternatives
  to support any incorporation decision
• Describe behavior and linkage among system
  components for each
• Product Integration composing and evaluating
  executable representations from project start is
  critical to verify and validate architecture
  suitability and evolvablity
• Project Planning/Integrated Project Management
  for IPPD appropriate skills and resources are
  necessary to form, evaluate, and maintain system
  flexibility
• include effort to create and maintain wrappers
  or glue and re-integrate solution as COTS
  products change

Architecture is created and maintained as a
corporate asset
23
Current Market Knowledge

• Anticipate and track changes to relevant market
  segments until system retired
• Anticipate and prototype system changes from
  updates to COTS products critical to the system
• Influence (not direct) COTS product changes,
  technology investments, and standards development

• Selected Work Process Implications
• Supplier Agreement Management license
  agreements with COTS vendors must be negotiated
  to meet project needs
• Integrated Supplier Management partnering with
  key vendors is critical (not just supplier
  management)
• vendors seldom allow process monitoring use
  hands-on evaluation
• relationships with vendors other customers helps
  to amplify leverage
• Technical Solution modification of COTS
  products introduces long term maintenance
  considerations and sizeable risk to project
  avoid if possible
• Project Planning significant resources may be
  required to monitor the marketplace and conduct
  COTS product evaluation including
  experimentation facilities

Marketplace is proactively monitored
24
Integral Programmatics and Risks

• Analysis of alternative solutions includes team
  skills and expertise, cost, schedule, and
  associated risks for
• building, fielding, and supporting the system
• implementing any needed changes to business
  processes (functional, operational, support)

• Selected Work Process Implications
• Technical Solution engineering trades should
  include risk, cost, schedule and other
  programmatic factors associated with each
  alternative solution
• Project Planning estimates of work product and
  task attributes should be generated for each
  alternative

Programmatic factors shape technical alternatives
25
Continuous Stakeholder Involvement

• Significant commitment from all stakeholders
  required
• identify, evaluate, and select alternative
  solutions
• confirm results of any and all negotiations
• agree evolving system meets their needs
• Stakeholders must reflect full diversity of
  interests

• Selected Work Process Implications
• IPPD discipline integrated teaming among
  disparate stakeholders throughout the development
  and maintenance is essential
• Validation end users must always be involved in
  validating solution suitability
• Integrated Project Management for IPPD
  accommodates resources for stakeholder
  involvement
• necessary changes to end-user operational
  processes must be explicitly and continuously
  managed and coordinated with solution development

Required stakeholder commitment may be
unprecedented
26
Disciplined Spiral or Iterative Practices

• Continuously determine a compatible and feasible
  set of business processes, requirements, plans,
  design, COTS products, and other components
• Enterprise business objectives drive solution
  definition
• Risk considerations drive degree of detail
• Marketplace dynamics drive development and
  maintenance processes

• Selected Work Process Implications
• Project Planning if not already implemented,
  extensive effort may be needed to revamp planning
  and engineering processes for a spiral
  development approach
• Risk Management tracking effectiveness of risk
  mitigation is key
• highest priority remaining risks should be used
  to (re) direct and manage the project

Spiral development facilitates developing a
viable solution
27
Frequent Executables

• Frequent executable representations reduce risk
  and reduce misunderstandings
• Provide critical insight into the solutions
  behavior
• Explore critical system attributes
• Validate end user business process
• Verify technical viability

• Selected Work Process Implications
• Requirements Development, Technical Solution, and
  Product Integration each iteration should
  produce an executable representation reflecting
  current understanding of requirements, COTS
  products, business processes, and alternative
  designs explored and negotiated

Executable representations demonstrate
stakeholder buy-in
28
Conclusion

• CBS
• changes many things about your approach
• requires even more discipline than custom
  development
• Although it needs interpretation, CMMI is a sound
  basis for CBS process improvement.
• Applying CMMI to CBSs is more than just Supplier
  Management
• All four disciplines
• All process area categories
• Process areas will need to be integrated and
  applied differently
• Must add in your own activities to plan, design,
  and deploy changes to enterprise processes

29
For more information

• TR coming CMU/SEI-2003-TR-022
• www.sei.cmu.edu/cbs
• Tricia Oberndorf Lisa Brownsword
• Director, Dynamic Systems Sr. MTS, CBS (ISIS)
• 412-268-6138 703-908-8203
• po_at_sei.cmu.edu llb_at_sei.cmu.edu