Software Process Dynamics II

1
Software Process Dynamics II
Ray Madachy madachy_at_usc.edu CSCI 510 September
27, 2006
2
Outline

• Demonstrations
• Value-based business and software process/product
  model
• Spiral hybrid process model for
  Software-Intensive System of Systems (SISOS)
• References

3
Outline

• Demonstrations
• Value-based business and software process/product
  model
• Spiral hybrid process model for
  Software-Intensive System of Systems (SISOS)
• References

4
Value-Based Model Background

• Purpose Support software business
  decision-making by experimenting with product
  strategies and development practices to assess
  real earned value
• Description System dynamics model relates the
  interactions between product specifications and
  investments, software processes including quality
  practices, market share, license retention,
  pricing and revenue generation for a commercial
  software enterprise

5
Model Features

• A Value-Based Software Engineering (VBSE) model
  covering the following VBSE elements
• Stakeholders value proposition elicitation and
  reconciliation
• Business case analysis
• Value-based monitoring and control
• Integrated modeling of business value, software
  products and processes to help make difficult
  tradeoffs between perspectives
• Value-based production functions used to relate
  different attributes
• Addresses the planning and control aspect of VBSE
  to manage the value delivered to stakeholders
• Experiment with different strategies and track
  financial measures over time
• Allows easy investigation of different strategy
  combinations
• Can be used dynamically before or during a
  project
• User inputs and model factors can vary over the
  project duration as opposed to a static model
• Suitable for actual project usage or flight
  simulation training where simulations are
  interrupted to make midstream decisions

6
Model Sectors and Major Interfaces

• Software process and product sector computes the
  staffing and quality over time
• Market and sales sector accounts for market
  dynamics including effect of quality reputation
• Finance sector computes financial measures from
  investments and revenues

7
Software Process and Product
effort and schedule calculation with dynamic
COCOMO variant
product defect flows
8
Finances, Market and Sales
investment and revenue flows
software license sales
market share dynamics including quality
reputation
9
Quality Assumptions

• COCOMO cost driver Required Software Reliability
  is a proxy for all quality practices
• Resulting quality will modulate the actual sales
  relative to the highest potential
• Perception of quality in the market matters
• Quality reputation quickly lost and takes much
  longer to regain (bad news travels fast)
• Modeled as asymmetrical information smoothing via
  negative feedback loop

10
Market Share Production Function and Feature Sets
Cases from Example 1
11
Sales Production Function and Reliability
Cases from Example 1
12
Example 1 Dynamically Changing Scope and
Reliability

• Shows how model can assess the effects of
  combined strategies by varying the scope and
  required reliability independently or
  simultaneously
• Simulates midstream descoping, a frequent
  strategy to meet time constraints by shedding
  features
• Three cases are demonstrated
• Unperturbed reference case
• Midstream descoping of the reference case after ½
  year
• Simultaneous midstream descoping and lowered
  required reliability at ½ year

13
Control Panel and Simulation Results
Descope
Case 1
Unperturbed Reference Case
Descope Lower Reliability
Case 2
14
Case Summaries
15
Example 2 Determining the Reliability Sweet Spot

• Analysis process
• Vary reliability across runs
• Use risk exposure framework to find process
  optimum
• Assess risk consequences of opposing trends
  market delays and bad quality losses
• Sum market losses and development costs
• Calculate resulting net revenue
• Simulation parameters
• A new 80 KSLOC product release can potentially
  increase market share by 15-30 (varied in model
  runs)
• 75 schedule acceleration
• Initial total market size 64M annual revenue
• Vendor has 15 of market
• Overall market doubles in 5 years

16
Cost Components
3-year time horizon
17
Value-Based Model Conclusions

• To achieve real earned value, business value
  attainment must be a key consideration when
  designing software products and processes
• Software enterprise decision-making can improve
  with information from simulation models that
  integrate business and technical perspectives
• Optimal policies operate within a multi-attribute
  decision space including various stakeholder
  value functions, opposing market factors and
  business constraints
• Risk exposure is a convenient framework for
  software decision analysis
• Commercial process sweet spots with respect to
  reliability are a balance between market delay
  losses and quality losses
• Model demonstrates a stakeholder value chain
  whereby the value of software to end-users
  ultimately translates into value for the software
  development organization

18
Value-Based Model Future Work

• Enhance product defect model with dynamic version
  of COQUALMO to enable more constructive insight
  into quality practices
• Add maintenance and operational support
  activities in the workflows
• Elaborate market and sales for other
  considerations including pricing scheme impacts,
  varying market assumptions and periodic upgrades
  of varying quality
• Account for feedback loops to generate product
  specifications (closed-loop control)
• External feedback from users to incorporate new
  features
• Internal feedback on product initiatives from
  organizational planning and control entity to the
  software process
• More empirical data on attribute relationships in
  the model will help identify areas of improvement
• Assessment of overall dynamics includes more
  collection and analysis of field data on business
  value and quality measures from actual software
  product rollouts

19
Outline

• Demonstrations
• Value-based business and software process/product
  model
• Spiral hybrid process model for
  Software-Intensive System of Systems (SISOS)
• References

20
Spiral Hybrid Process Introduction

• The spiral lifecycle is being extended to address
  new challenges for Software-Intensive Systems of
  Systems (SISOS), such as coping with rapid change
  while simultaneously assuring high dependability
• A hybrid plan-driven and agile process has been
  outlined to address these conflicting challenges
  with the need to rapidly field incremental
  capabilities
• A system-of-systems (SOS) integrates multiple
  independently-developed systems and is very
  large, dynamically evolving, unprecedented, with
  emergent requirements and behaviors
• However, traditional static approaches cannot
  capture dynamic feedback loops and interacting
  phenomena that cause real-world complexity (e.g.
  hybrid processes, project volatility, increment
  overlap and resource contention, schedule
  pressure, slippages, communication overhead,
  slack, etc.)
• A system dynamics model is being developed to
  assess the incremental hybrid process and support
  project decision-making
• Both the hybrid process and simulation model are
  being evolved on a very large scale incremental
  project for a SISOS (U.S. Army Future Combat
  Systems)

21
Future Combat Systems (FCS) Network
22
Scalable Spiral Model Increment Activities

• Organize development into plan-driven increments
  with stable specs
• Agile team watches for and assesses changes, then
  negotiates changes so next increment hits the
  ground running
• Try to prevent usage feedback from destabilizing
  current increment
• Three team cycle plays out from one increment to
  the next

23
Spiral Hybrid Model Features

• Estimates cost and schedule for multiple
  increments of a hybrid process that uses three
  specialized teams (agile re-baseliners,
  developers, VVers) per the scalable spiral
  model
• Considers changes due to external volatility and
  feedback from user-driven change requests
• Deferral policies and team sizes can be
  experimented with
• Includes tradeoffs between cost and the timing of
  changes within and across increments, length of
  deferral delays, and others

24
Model Input Control Panel
25
Model Overview

• Built around a cyclic flow chain for capabilities
• Arrayed for multiple increments
• Each team is represented with a level and
  corresponding staff allocation rate
• Changes arrive a-periodically via the volatility
  trends time function and flow into the level for
  capability changes
• Changes are processed by the agile team and
  allocated to increments per the deferral policies
• Constant or variable staffing for the agile team
• For each increment the required capabilities are
  developed into developed capabilities and then
  VVed into V Ved capabilities
• Productivities and team sizes for development and
  VV calculated with a Dynamic COCOMO variant and
  continuously updated for scope changes
• Flow rates between capability changes and V
  Ved capabilities are bi-directional for
  capability kickbacks sent back up the chain
• User-driven changes from the field are identified
  as field issues that flow back into the
  capability changes

26
Volatility Cost Functions

• The volatility effort multiplier for construction
  effort and schedule is an aggregate multiplier
  for volatility from different sources (e.g. COTS,
  mission, etc.) relative to the original baseline
  for increment
• The lifecycle timing effort multiplier models
  increased development cost the later a change
  comes in midstream during an increment

27
Sample Response to Volatility

• An unanticipated change occurs at month 8 shown
  as a volatility trend 1 pulse
• It flows into capability changes 1 which
  declines to zero as the agile team processes the
  change
• The change is non-deferrable for increment 1 so
  total capabilities 1 increases
• Development team staff size dynamically responds
  to the increased scope

28
Sample Test Results

• Test case for two increments of 15 baseline
  capabilities each
• A non-deferrable change comes at month 8 (per
  previous slide)
• The agile team size is varied from 2 to 10 people
• Increment 1 business value also lost for agile
  team sizes of 2 and 4

29
Spiral Hybrid Model Conclusions and Future Work

• System dynamics is a convenient modeling
  framework to deal with the complexities of a
  SISOS
• A hybrid process appears attractive to handle
  SISOS dynamic evolution, emergent requirements
  and behaviors
• Initial results indicate that having an agile
  team can help decrease overall cost and schedule
• Model can help find the optimum balance
• Will obtain more empirical data to calibrate and
  parameterize model including volatility and
  change trends, change analysis effort, effort
  multipliers and field issue rates
• Model improvements
• Additional staffing options
• Rayleigh curve staffing profiles
• Constraints on development and VV staffing
  levels
• More flexible change deferral options across
  increments
• Increment volatility balancing policies
• Provisions to account for (timed)
  business/mission value of capabilities
• Additional model experimentation
• Include capabilities flowing back from developers
  and VVers
• Vary deferral policies and volatility patterns
  across increments
• Compare different agile team staffing policies
• Continue applying the model on a current SISOS
  and seek other potential pilots

30
Outline

• Demonstrations
• Value-based business and software process/product
  model
• Spiral hybrid process model for
  Software-Intensive System of Systems (SISOS)
• References

31
Major References

• Abdel-Hamid T, Madnick S, Software Project
  Dynamics, Englewood Cliffs, NJ, Prentice-Hall,
  1991
• Brooks F, The Mythical Man-Month, Reading, MA,
  Addison-Wesley, 197
• Forrester JW, Industrial Dynamics. Cambridge,
  MA MIT Press, 1961
• Kellner M, Madachy R, Raffo D, Software Process
  Simulation Modeling Why? What? How?, Journal of
  Systems and Software, Spring 1999
• Madachy R, A software project dynamics model for
  process cost, schedule and risk assessment, Ph.D.
  dissertation, Department of Industrial and
  Systems Engineering, USC, December 1994
• Madachy R, System Dynamics and COCOMO
  Complementary Modeling Paradigms, Proceedings of
  the Tenth International Forum on COCOMO and
  Software Cost Modeling, SEI, Pittsburgh, PA, 1995
  
• Madachy R, System Dynamics Modeling of an
  Inspection-Based Process, Proceedings of the
  Eighteenth International Conference on Software
  Engineering, IEEE Computer Society Press, Berlin,
  Germany, March 1996

32
Major References (cont.)

• Madachy R, Simulation in Software Engineering,
  Encyclopedia of Software Engineering, Second
  Edition, Wiley and Sons, Inc., New York, NY, 2001
  
• Integrating Business Value and Software Process
  Modeling, Proceedings of the 2005 Software
  Process Workshop, Beijing, China,
  Springer-Verlag, May 2005
• Madachy R, Software Process Dynamics, Wiley-IEEE
  Computer Society Press, Washington, D.C., 2006
  (to be published)
• Madachy R, Tarbet D, Case Studies in Software
  Process Modeling with System Dynamics, Software
  Process Improvement and Practice, Spring 2000
• Richardson GP, Pugh A, Introduction to System
  Dynamics Modeling with DYNAMO, MIT Press,
  Cambridge, MA, 1981
• USC Web Sites
• http//rcf.usc.edu/madachy/spd
• portions of forthcoming book Madachy R, Software
  Process Dynamics, IEEE Computer Society Press,
  Washington, D.C.
• includes models available for download
• http//sunset.usc.edu/classes/cs599_99
• USC-CSE Software Process Modeling Course
  (includes other system dynamics links)

33
Backup Slides