Indian Statistical Institute

1
Indian Statistical Institute
Controlling Software Project Through Metrics -An
Overview
Amitava Banerjee
2
Indian Statistical Institute
The evil that is in the world always comes of
ignorance and good intentions can do as much harm
as malevolence if they lack understanding. Albert
Camus, The Plague
Software metrics have such a vast potential !
They have amazingly many uses but (alas)
surprisingly few applications ! Not sure about
the source
3
Indian Statistical Institute

• Contents
• Definition of measurements and metrics
• Need for measurement
• Classification of metrics
• Typical software metrics and their
  classification
• Typical mistakes
• Controlling Software Projects through Metrics

4
Indian Statistical Institute
Measurement Measurement is an activity which
assigns a numerical value or symbol to a physical
attribute. Note Physical attribute is
something which we are trying to measure.
Sometimes, like measurement of complexity or
skill level, it may be just a symbol. Beware! In
many cases the physical attribute is forgotten!
5
Indian Statistical Institute
Metric A metric is a proposed measure. Note
Measurements should have properties like known
repeatability and reproducibility which may not
be there in metrics. The physical attribute also
may not be well understood! Caution If the above
properties are not satisfied the metric is still
in a developmental stage and should not be used.
6
Indian Statistical Institute
Those who forget history are condemned to repeat
it. George Santayana
7
Indian Statistical Institute

• Why Measure ?
• Understand the physical attribute
• Control the attribute
• Understand how and where to improve
• Compare values of attributes

8
Indian Statistical Institute
Explanation Understanding Being able to make
probabilistic statements. Understanding enables
one to plan and forecast. Control Measure
Compare Act Cycle. The compare stage consists
of understanding the trend and making short term
predictions. Action is taken accordingly. Control
enables one to meet commitments.
9
Indian Statistical Institute
Explanation Improve Understanding relationship
with other parameters and changing them to
achieve desired goals. Study of relationship also
enables one to validate assumptions and assess
risks.
10
Indian Statistical Institute
Explanation Compare Compare current value of the
attribute with past or a set value (often called
organisational goals). Values obtained from two
different sources may also be compared. Note
Except for the purpose of comparison,
measurements should be used as leading
indicators. The stress should ideally be less on
comparison and more on other three. Metrics are
classified into these four categories.
11
Indian Statistical Institute
Classification of Typical Software
Metrics Effort variance Comparison of the
estimation ability. Schedule variance Same as
above. Productivity Comparison of development
efficiency. Risk assessment Improve development
efficiency but physical attribute is not clear.
12
Indian Statistical Institute
Classification of Typical Software
Metrics Delivered defect density Comparison of
development / defect detection ability.

• Notes
• Software metrics are typically used for
  comparison.
• In most cases the main physical attribute is not
  considered.
• In many cases the physical attribute is not well
  defined.

13
Indian Statistical Institute

• Some Points on Comparison
• Use comparison to understand organisational
  capabilities.
• When using comparison for the purpose of root
  cause analysis use wider limits (which can be
  computed statistically from the distribution of
  the physical attribute). Make sure that root
  cause analysis is done rarely.

14
Indian Statistical Institute

• Some Points on Comparison
• Avoid using comparison on a project to
  project basis.
• Note that comparison typically leads to defence
  mechanism and consequently explanation. Remember
  the saying Never explain. Your friends dont
  need it, your enemies dont believe it!

15
Indian Statistical Institute

• Where is the life we have lost in living
• Where is the knowledge we have lost in
  information
• Where is the wisdom we have lost in knowledge.
• T S Eliot

16
Indian Statistical Institute

• Some Definitions And Physical Attributes
• Plan
• Activities to control the future.
• Risk
• Probability that some assumption which impacts
  the output will not be correct.
• Physical Attributes
• Actual Effort, Schedule, Number of defects in
  different phases (per unit effort) classified
  with respect to severity and technical
  classification.

17
Indian Statistical Institute

• Some Points On Physical Attributes
• All physical attributes of the real world vary.
• The variability could be large or small depending
  on the capability of the process.
• The variability (as well as the typical value)
  can be estimated from the data.
• In most cases the projects can be segregated on
  the basis of external parameters (eg on size,
  estimated effort, coding effort and so on).

18
Indian Statistical Institute

• Some Points On Physical Attributes
• The segregation is effective in case the
  average or variability of the segments are
  different.
• When averages are different the underlying
  processes are different which needs to be studied
  (improvement action needs to be taken).
• Segmentation leads to reduced variability.

19
Indian Statistical Institute

• Explanation of Segments
• Two different segments may have
• Equal mean unequal variability
• Unequal mean equal variability
• Unequal mean unequal variability

In 2nd and 3rd case processes are different. (The
development process including methods of
analysis, technique/models of design, methods of
reviews, approvals and testing are different.
This needs to be studied through detailed flow
charts and the process changed as necessary).
20
Indian Statistical Institute

• Exploring Relationships
• The identified variable (physical attribute)
  within a segment is typically related to other
  variables. These relationships need to be
  explored.
• Examples
• Remaining effort may depend on number of tasks
  remaining and the current variability.
• Defects may depend on injection rate of previous
  phases.

21
Indian Statistical Institute

• Controlling Projects Through Metrics
• Identify physical attributes to be controlled.
• Carry out segmentation to reduce variability.
• Study the process within segments and carry out
  necessary improvements (describe process through
  detailed flow charts).
• For each segment identify the controlling
  variables (typically the same variable measured
  at an earlier stage).

22
Indian Statistical Institute

• Controlling Projects Through Metrics
• Develop a prediction model for control.
• Assess the risks and calibrate your prediction
  (list the assumptions and give a 1 10 rating
  for subjective probability of not meeting the
  assumption. Calculate risk priority rating to
  assess risk).
• Act if your predicted value is not acceptable.

23
Indian Statistical Institute

• Summary
• Metrics should be used to control important
  physical attributes.
• Number of metrics should be limited to ensure
  proper analysis.
• Metrics should be used as leading indicators.
• To start with segmentation must be carried out to
  reduce variability.

24
Indian Statistical Institute

• Summary
• Improvement actions must be carried out on
  segments.
• Models for short term prediction and risk
  assessment must be developed. Future actions must
  be taken on the basis of these models.
• Event by event root cause analysis should be
  reduced.
• Metrics for comparison must be used for overall
  organisational planning only.

25
Indian Statistical Institute
Thank You