Software Process Dynamics

1
Software Process Dynamics
Ray Madachy CS 510 November 1, 2004
2
Outline

• Introduction
• process models and system dynamics
• model structures and system behaviors
• Examples
• Brookss Law
• Dynamic COCOMO
• Abdel-Hamid project model
• earned value
• Research Studies
• software inspections
• software process concurrence
• other contributions
• Future Work and References

3
Terminology

• System a grouping of parts that operate together
  for a common purpose a subset of reality that is
  a focus of analysis
• open, closed
• Software Process a set of activities, methods,
  practices and transformations used by people to
  develop software.
• Model an abstract representation of reality.
• static, dynamic continuous, discrete
• Simulation the numerical evaluation of a
  mathematical model.
• System dynamics a simulation methodology for
  modeling continuous systems. Quantities are
  expressed as levels, rates and information links
  representing feedback loops.

4
Software Process Models

• Used to quantitatively reason about, evaluate and
  optimize the software process.
• Demonstrate effects of process strategies on
  cost, schedule and quality throughout lifecycle
  and enable tradeoff analyses.
• Can experiment with changed processes via
  simulation before committing project resources.
• Provide interactive training for software
  managers process flight simulation.
• Encapsulate our understanding of development
  processes (and support organizational learning).
• Benchmark process improvement when model
  parameters are calibrated to organizational data.
• Process modeling techniques can be used to
  evaluate other existing descriptive
  theories/models.
• force clarifications, reveal discrepancies, unify
  fields

5
Process Modeling Characterization Matrix and
Examples
Example Litton studies in Madachy et al. 2000
6
System Dynamics Approach

• Involves following concepts Richardson 91
• defining problems dynamically, in terms of
  graphs over time
• striving for an endogenous, behavioral view of
  the significant dynamics of a system
• thinking of all real systems concepts as
  continuous quantities interconnected in
  information feedback loops and circular causality
• identifying independent levels in the system and
  their inflow and outflow rates
• formulating a model capable of reproducing the
  dynamic problem of concern by itself
• deriving understandings and applicable policy
  insights from the resulting model
• implementing changes resulting from model-based
  understandings and insights.
• Dynamic behavior is a consequence of system
  structure

7
Systems Thinking

• A way to realize the structure of a system that
  leads to its behavior
• Systems thinking involves
• thinking in circles and considering
  interdependencies
• closed-loop causality vs. straight-line thinking
• seeing the system as a cause rather than effect
• internal vs. external orientation
• thinking dynamically rather than statically
• operational vs. correlational orientation
• Improvement through organizational learning takes
  place via shared mental models
• The power of models increase as they become more
  explicit and commonly understood by people
• a context for interpreting and acting on data
• System dynamics is a methodology to implement
  systems thinking and leverage learning efforts

8
Applicability to Software Processes

• Since software development is a dynamic and
  complex process with many factors, system
  dynamics is well-suited to analysis of software
  process improvement strategies
• global system perspective
• accounts for process feedback effects
• can model inherent tradeoffs between schedule,
  cost and quality
• accounts for critical path flows to analyze
  schedule as opposed to traditional cost reduction
  analyses
• enables low cost process experimentation

9
Software Process Control System
Software Development or Evolution Project
Software Process
Software Artifacts
Requirements, resources etc.
internal project feedback
external feedback from operational environment
10
System Dynamics Notation

• System represented by x(t) f(x,p).
• x vector of levels (state variables), p set of
  parameters
• Legend
• Example system

11
Model Elements
12
Model Elements (continued)
13
Software Product Transformations
Cycle time per phase start time of first
flowed entity - completion time of last flowed
entity
Cycle time per task transit time through
relevant phase(s)
14
Error Co-flows
15
Error Detection and Rework

• Cost/schedule/quality tradeoffs available when
  defects are represented as levels with the
  associated variable effort and cycle time for
  rework and testing as a function of those levels.

16
Personnel Pool
17
Feedback Loops

• A feedback loop is a closed path connecting an
  action decision that affects a level, then
  information on the level being returned to the
  decision making point to act on.

18
Learning Curve
19
Software Production Structure

• Combines task development and personnel chains.
• Production constrained by productivity and
  applied personnel resources.

20
Typical Behavior Patterns
21
Delays

• Time delays are ubiquitous in processes
• They are important structural components of
  feedback systems.
• Example hiring delays in software development.
• the average hiring delay represents the time that
  a personnel requisition remains open before a new
  hire comes on board

22
Negative Feedback

• Negative feedback exhibits goal seeking behavior,
  or sometimes instability
• May represent hiring increase towards a staffing
  goal. The change is more rapid at first and
  slows down as the discrepancy between desired and
  perceived decreases. Also a good trend for
  residual defect levels.

positive goal

• rate (goal - present level)/time constant

zero goal
Analytically Level Goal (Level0 - Goal)e
-t/tc
23
Positive Feedback

• Positive feedback produces a growth process
• Exponential growth may represent sales growth (up
  to a point), Internet traffic, defect fixing
  costs over time
• rate present levelconstant

Analytically exponential growth
LevelLevel0eat exponential decay
LevelLevel0e-t/TC
24
S-Curves

• S-curve graphic display of a quantity like
  progress or cumulative effort plotted against
  time that exhibits an s-shaped curve. It is
  flatter at the beginning and end, and steeper in
  the middle. It is produced on a project that
  starts slowly, accelerates and then tails off as
  work tapers off
• S-curves are also observed in the ROI curve of
  technology adoption, either time-based return or
  in production functions that relate ROI to
  investment.

25
Outline

• Introduction
• process models and system dynamics
• model structures and system behaviors
• Examples
• Brookss Law
• Dynamic COCOMO
• Abdel-Hamid project model
• earned value
• Research Studies
• software inspections
• software process concurrence
• other contributions
• Future Work and References

26
Sample Insights from Field

• Inspection policy tradeoff analysis - diminishing
  returns from inspections as a function of error
  generation rates Madachy 94
• QA policy tradeoff analysis - finding the optimal
  QA effort Abdel-Hamid/Madnick 91
• Rework staffing allocation Tvedt 95
• Organizational process improvement transition
  requires temporary productivity setbacks Rubin,
  Johnson, Yourdon 95
• Maximize your pro-SPI people Burke 96

27
Sample Insights (continued)

• Leverage of experienced staff (several)
• Internal workings of Brookess Law - training and
  communication losses Madachy 00
• Schedule compression not a static decision
  Abdel-Hamid 90
• Anchor-dragging in project control Abdel-Hamid
  93
• Competing feedback loops in software reuse
  factory Abdel-Hamid 93b
• Many others

28
Brookss Law Modeling Example

• Adding manpower to a late software project makes
  it later Brooks 75.
• We will test the law using a simple model based
  on the following assumptions
• new personnel require training by experienced
  personnel to come up to speed
• more people on a project entail more
  communication overhead
• experienced personnel are more productive then
  new personnel, on average.
• An effective teaching tool

29
Model Diagram and Equations
30
Model Output for Varying Additions
Function points/day
Days
Sensitivity of Software Development Rate to
Varying Personnel Allocation Pulses (1
no extra hiring, 2 add 5 people on 100th day, 3
add 10 people on 100th day)
31
Dynamic COCOMO
32
Abdel-Hamid Model Subsystems
33
Abdel-Hamid Model Behavior

• Underestimation factor .67

1 Scheduled Compl
2 Cumul Man Days
3 Tasks Developed
4 Cumulative Task
5 Perceived Job Size
1
600.00
2
5000.00
3
1500.00
4
5
3
5
5
5
1
2
1
1
300.00
4
3
2
2500.00
5
3
1
1
4
750.00
2
5
3
2
1
0.00
2
0.00
3
3
4
2
4
4
4
5
0.00
0.00
107.50
215.00
322.50
430.00
1102 AM 2/26/95
Days
Graphs Page 1
34
QA Policy Tradeoff Analysis

• Note non-standard QA definition

35
Exhaustion Model
36
Exhaustion Model Assumptions

• Workers increase their effective hours by
  decreasing slack time or working overtime
• The maximum shortage that can be handled varies.
  
• Workers are less willing to work hard if deadline
  pressures persist for a long time.
• The overwork duration threshold increases or
  decreases as people become more or less
  exhausted.
• The exhaustion level also increases with
  overwork.
• The multiplier for exhaustion level is 1 when
  people work full 8 hour days, and goes over 1
  with overtime. The exhaustion increases at a
  greater rate in overtime mode up to the maximum
  tolerable exhaustion.
• The exhaustion level slowly decreases when the
  threshold is reached or deadline pressures stop
  with an exhaustion depletion delay.
• During this time, workers dont go into overwork
  mode again until the exhaustion level is fully
  depleted.

37
Exhaustion Model Relationships
exhaustion flow
multiplier to overwork duration threshold
38
Exhaustion Behavior
39
Earned Value

• Earned value is a method for measuring project
  performance.
• It compares the amount of work that was planned
  with what was actually accomplished to determine
  if cost and schedule performance is as planned.
• Cost performance index (CPI) is the ratio of
  budgeted costs to actual costs (BCWP/ACWP)
• Schedule performance index (SPI) is the ratio of
  work performed to work scheduled (BCWP/BCWS)

40
Earned Value Example
Project 1 (easy problems first) appears more
productive at beginning but finishes later

• Project 2 (hard problems first)
• finishes sooner

Middle-road planassuming equal tasksthroughout
lifecycle
Project 1 (easy problems first) appears more
productive at beginning but finishes later
41
Earned Value Model
42
Outline

• Introduction
• process models and system dynamics
• model structures and system behaviors
• Examples
• Brookss Law
• Dynamic COCOMO
• Abdel-Hamid project model
• earned value
• Research Studies
• software inspections
• software process concurrence
• other contributions
• Future Work and References

43
Introduction to Inspection Model

• Research problem addressed
• What are the dynamic effects to the process of
  performing inspections?
• Model used to evaluate process quantitatively
• demonstrates effects of inspection practices on
  cost, schedule and quality throughout lifecycle
• can experiment with changed processes before
  committing project resources
• benchmark process improvement
• support project planning and management
• Model parameters calibrated to Litton and JPL
  data
• error generation rates, inspection effort,
  efficiency, productivity, others
• Model validated against industrial data

44
System Diagram
45
System Diagram (continued)
46
Effects of Inspections
1 total manpower rate
2 total manpower rate
1
26.00
1
2
2
2
1
1
13.00
1
2
1
1
0.00
0.00
75.00
150.00
225.00
300.00
318 PM 10/21/28
Days
task graphs Page 7
1 with inspections, 2 without inspections

• Qualitatively matches generalized effort curves
  from Michael Fagan
• (Advances in software inspections, IEEE
  Transactions on Software Engineering, July 1986),
  
• which were used in problem definition.

47
Inspection Policy Tradeoff Analysis

• Varying error generation rates shows diminishing
  returns from inspections Madachy 94

48
Derivation of Phase Specific Cost Driver
49
Contributions of Inspection Model

• Demonstrated dynamic effects of performing
  inspections.
• New knowledge regarding interrelated factors of
  inspection effectiveness.
• Demonstrated complementary features of static and
  dynamic models.
• Techniques adopted in industry.

50
Process Concurrence Introduction

• Process concurrence the degree to which work
  becomes available based on work already
  accomplished
• represents an opportunity for parallel work
• a framework for modeling constraint mechanics
• Increasing task parallelism is a primary RAD
  opportunity to decrease cycle time
• System dynamics is attractive to analyze schedule
• can model task interdependencies on the critical
  path

51
Process Concurrence Basics

• Process concurrence describes interdependency
  constraints between tasks
• can be an internal constraint within a
  development stage or an external constraint
  between stages
• Describes how much work becomes available for
  completion based on previous work accomplished
• Accounts for realistic bottlenecks on work
  availability
• vs. a model driven solely by resources and
  productivity that can finish in almost zero time
  with infinite resources
• Concurrence relations can be sequential,
  parallel, partially concurrent, or other
  dependent relationships

52
Internal Concurrence Examples
less parallel integration
region of parallel work
initial work on important segments other
segments have to wait until these are done
Complex system development where tasks are
dependent due to required inter-task
communication.
Simple conversion task where tasks can be
partitioned with no communication
53
External Concurrence Examples
1 - a linear lockstep concurrence in the
development of totally independent modules 2 -
S-shaped concurrence for new, complex
development where an architecture core is
needed first 3 - highly leveraged instantiation
like COTS with some glue code development 4 -
a slow design buildup between phases
54
Architecture Approach Comparison
Opportunity for increased task parallelism and
quicker elaboration
55
External Concurrence Model
the time profile of tasksready to elaborate
ideal staffing curve shape
56
Simulation Results and Sample Lessons
flat Rayleigh COTS
pulse
at front
Critical customer decision delays slow progress
- e.g. cant design until timing
performance specs are known Early stakeholder
concurrence enables RAD - e.g. decision on
architectural framework or COTS package
N/A
57
Additional Considerations

• Process concurrence curves can be more precisely
  matched to the software system types
• COTS by definition should exhibit very high
  concurrence
• Mixed strategies produce combined concurrence
  relationships
• E.g. COTS first thennew development

58
Process Concurrence Conclusions

• Process concurrence provides a robust modeling
  framework
• a method to characterize different approaches in
  terms of their ability to parallelize or
  accelerate activities
• Gives a detailed view of project dynamics and is
  relevant for planning and improvement purposes
• a means to collaborate between stakeholders to
  achieve a shared planning vision
• Can be used to derive optimal staffing profiles
  for different project situations
• More generally applicable than the Rayleigh curve
• More empirical data needed on concurrence
  relationships from the field for a variety of
  projects

59
Additional Original Models

• Already published
• Earned value
• Interactive Rayleigh staffing
• Resource contention
• Product line reuse
• Training and tutorial models
• Project planning and control
• Monte-Carlo simulation
• To-be published in Software Process Dynamics
• Several software business case models
• Defect cost-to-fix model
• Variations on model infrastructures
• Risk-driven task selection policies
• Brookss Law enhanced for partitioning
• Large-scale systems acquisition
• Enhanced tutorial models
• More coming

60
CS599 Student Research Projects

• Dynamics of architecture development process in
  MBASE inception and elaboration phases
• Application of RAD techniques to pre-IPO internet
  companies
• COTS glue-code development and integration
  dynamics
• Reuse and language-level effects in software
  development
• CMM-based process improvement strategies
• Achievements
• at least 3 student works have been published
  externally
• former student Jongmoon Baik, Ph.D. continues
  work in software process modeling at Motorola
  Laboratories

61
Outline

• Introduction
• process models and system dynamics
• model structures and system behaviors
• Examples
• Brookss Law
• Dynamic COCOMO
• Abdel-Hamid project model
• earned value
• Research Studies
• software inspections
• software process concurrence
• other contributions
• Future Work and References

62
Future Work

• Research Areas
• Complex systems-of-systems analysis, including
  distributed projects
• Adaptation to rapid change on projects
• Trends like COTS-intensive solutions
• Software business value
• Reusable model structures
• Process model selection
• Knowledge-based techniques
• Related simulation research
• Industrial data analysis
• Contracts
• Ongoing work on FCS, NASA-HDC, USC CSE affiliate
  collaboration
• Research proposal opportunities

63
Major References

• Abdel-Hamid T, Madnick S, Software Project
  Dynamics, Englewood Cliffs, NJ, Prentice-Hall,
  1991
• Brooks F, The Mythical Man-Month, Reading, MA,
  Addison-Wesley, 197
• Forrester JW, Industrial Dynamics. Cambridge,
  MA MIT Press, 1961
• Kellner M, Madachy R, Raffo D, Software Process
  Simulation Modeling Why? What? How?, Journal of
  Systems and Software, Spring 1999
• Madachy R, A software project dynamics model for
  process cost, schedule and risk assessment, Ph.D.
  dissertation, Department of Industrial and
  Systems Engineering, USC, December 1994
• Madachy R, System Dynamics and COCOMO
  Complementary Modeling Paradigms, Proceedings of
  the Tenth International Forum on COCOMO and
  Software Cost Modeling, SEI, Pittsburgh, PA, 1995
  
• Madachy R, System Dynamics Modeling of an
  Inspection-Based Process, Proceedings of the
  Eighteenth International Conference on Software
  Engineering, IEEE Computer Society Press, Berlin,
  Germany, March 1996

64
Major References (cont.)

• Madachy R, Simulation in Software Engineering,
  Encyclopedia of Software Engineering, Second
  Edition, Wiley and Sons, Inc., New York, NY, 2001
  
• Madachy R, Software Process Dynamics, IEEE
  Computer Society Press, Washington, D.C. (to be
  published)
• Madachy R, Tarbet D, Case Studies in Software
  Process Modeling with System Dynamics, Software
  Process Improvement and Practice, Spring 2000
• Richardson GP, Pugh A, Introduction to System
  Dynamics Modeling with DYNAMO, MIT Press,
  Cambridge, MA, 1981
• USC Web Sites
• http//rcf.usc.edu/madachy
• portions of forthcoming book Madachy R, Software
  Process Dynamics, IEEE Computer Society Press,
  Washington, D.C.
• includes models available for download
• http//sunset.usc.edu/classes/cs599_99
• USC-CSE Software Process Modeling Course
  (includes other system dynamics links)

65
Backup Slides
66
A Software Process
67
Modeling Process Overview

• Iterative, cyclic

policy implementation
system understandings
policy analysis
problem definition
simulation
model conceptualization
model formulation
68
Modeling Stages and Concerns
context symptoms reference behavior
modes model purpose system boundary feedback
structure model representation model
behavior reference behavior modes
problem definition model conceptualization
model formulation simulation evaluation
69
The Continuous View

• Individual events are not tracked
• Entities are treated as aggregate quantities that
  flow through a system
• can be described through differential equations
• Discrete approaches usually lack feedback,
  internal dynamics

70
General System Behaviors

• Behaviors are representative of many known types
  of systems.
• Knowing how systems respond to given inputs is
  valuable intuition for the modeler
• Can be used during model assessment
• use test inputs to stimulate the system
  behavioral modes

71
System Order

• The order of a system refers to the number of
  levels contained.
• A single level system cannot oscillate, but a
  system with at least two levels can oscillate
  because one part of the system can be in
  disequilibrium.

72
Example System Behaviors

• Delays
• Goal-seeking Negative Feedback
• First-order Negative Feedback
• Second-order Negative Feedback
• Positive Feedback Growth or Decline
• S-curves

73
Third Order Delay

• A series of 1st order delays
• Graphs show water levels over time in each tank

tank 1 starts full
74
Delay Summary
input output
Delay order Pulse input Step
input 1 2 3 Infinite (pipeline)
75
Orders of Negative Feedback

• First-order Negative Feedback
• Second-order Negative Feedback
• Oscillating behavior may start out with
  exponential growth and level out. It could
  represent the early sales growth of a software
  product that stagnates due to satisfied market
  demand, competition or declining product quality.

76
Sample Project Progress Trends

• From Madachy 94

1 cum tasks design
2 cum tasks coded
3 tasks tested
4 fraction done
5 actual completio
1
1
2
3
4
5
533.30
2
533.27
3
533.27
1
4
1.00
5
5
260.25
1
266.65
2
266.63
3
266.63
4
0.50
5
130.12
1
2
1
0.00
2
0.00
3
0.00
4
4
0.00
1
2
2
3
3
3
4
4
5
5
5
0.00
0.00
75.00
150.00
225.00
300.00
818 AM 11/3/28
Days
task graphs Page 1
77
Error Multiplication Effects
78
Risk Analysis

• A deterministic point estimate from a simulation
  run is only one of many actual possibilities
• Simulation models are ideal for exploring risk
• test the impact of input parameters
• test the impact of different policies
• Monte-Carlo analysis takes random samples from an
  input probability distribution

79
Monte-Carlo Example

• Results of varying inspection efficiency

80
Trying to Accelerate Software Development
software tasks
tasks to
develop
personnel
restricted channel flow
development rate
(partially adapted from Putnam 80)
completed
tasks
81
Limited Parallelism of Software Activities

• There are always sequential constraints
  independent of phase
• analysis and specification figure out what
  you're supposed to do
• development of something (architecture, design,
  code, test plan, etc.)
• assessment verify/validate/review/debug
• possible rework recycle of previous activities
• These can't be done totally in parallel with more
  applied people
• different people can perform the different
  activities with limited parallelism, but
  downstream activities will always have to follow
  some of the upstream

82
Lessons from Brooks in The Mythical Man-Month

• Sequential constraints imply tasks cannot be
  partitioned.
• applying more people has no effect on schedule
• Men and months are interchangeable only when
  tasks can be partitioned with no communication
  among them.

83
Internal Process Concurrence

• Internal process concurrence relationship shows
  how much work can be done based on the percent of
  work already done.
• The relationships represent the degree of
  sequentiality or concurrence of the tasks
  aggregated within a phase.

84
External Process Concurrence

• External process concurrence relationships
  describe constraints on amount of work that can
  be done in a downstream phase based on the
  percent of work released by an upstream phase.
• See examples on following slide
• More concurrent processes have curves near the
  upper left axes, and less concurrent processes
  have curves near the lower and right axes.
• Tasks can be considered to be the number of
  function points demonstrable in their
  phase-native form

85
Roles Have Different Mental Models

• Differing perceptions upstream and downstream
  (Ford-Sterman 97)
• Group visualization helps identify disparities to
  improve communication and reduce conflict.

86
RAD Modeling Example

• One way to achieve RAD is by having base software
  architectures tuned to application domains
  available for instantiation, standard database
  connectors and reuse.
• The next two slides contrast the concurrence of
  an HR portal development using two different
  development approaches 1) from scratch and 2)
  with an existing HR base architecture.

87
Example Development from Scratch
88
Rayleigh Curve Applicability

• Rayleigh curve was based on initial studies of
  hardware research and development
• projects resemble traditional waterfall
  development for unprecedented systems
• Rayleigh staffing assumptions dont hold well for
  COTS, reuse, architecture-first design patterns,
  4th generation languages or staff-constrained
  situations
• However an ideal staffing curve is proportional
  to the number of problems ready for solution
  (from a product perspective).

89
Process Concurrence Advantages

• Process concurrence can model more realistic
  situations than the Rayleigh curve and produce
  varying dynamic profiles
• Can be used to show when and why Rayleigh curve
  modeling doesnt apply
• Process concurrence provides a way of modeling
  constraints on making work available, and the
  work available to perform is the same dynamic
  that drives the Rayleigh curve
• since the staff level is proportional to the
  problems (or specifications) ready to implement