Measuring Software Quality

1
Measuring Software Quality

• Norm Goodkin
• Quality Matrix International, Inc.

2
Why is there never enough time to do it right,
but always enough time to do it over?
3
Triple Constraint?
Scope
Requirements Management RequisiteProtm, Doorstm,
Caliber-RMtm
Whatabout Quality?
Schedule
Earned Value MS Project Primavera
Cost
4
Would you tell me, please, which way I ought to
go from here? That depends a good deal on where
you want to get to, said the Cat. I dont much
care where-- said Alice. Then it doesnt matter
which way you go, said the Cat. --so long as I
get somewhere, Alice added as an
explanation. Oh, youre sure to do that, said
the Cat, If only you walk long enough.
5
You cant manage what you cant measure.
6
(No Transcript)
7
1D, 2D, 3D

• 1D Total Number of Bugs
• But some bugs are more equal than others
• 2D Bugs by level of Severity
• But one area can suffer while another is great
• 3D Levels of Severity Area Affected
• Severity is not the same as Urgency for Fix.

8
Severity 0-9

• 9 worst case scenario
• 8 pretty bad
• 7 service outage
• 6 old feature fails
• 5 new feature fails
• 0 spelling error in an internal operations
  message

9
Accumulation of Low Level Bugs

• Lots of Level 3 and Level 4 bugs can give you a
  tummy ache.
• Set limits at all levels, for all areas.
• Take appropriate action if it looks like you are
  going to exceed the limits.
• Fast Track or Crash to allow more fixing
• Prepare users to deal with the problems

10
Setting the Bar

• Set realistic goals
• Base goals on past performance
• Enjoy the success of achieving goals
• Gold Plating is as wrong for Quality as it is for
  Scope.

11
Zero Defects/Zero Downtime?

• Is ZD2 a realistic requirement?
• Goals vs. Objectives
• Continuous Improvement a form of Gold Plating?
• Set Quality Targets based on project history
• Achieve Acceptable Quality

12
Quality Review Board

• Represent Stakeholders
• Not too senior
• Subject Matter Experts
• 10-15 max
• Quantify Problem Severity
• Establish Quality Targets

13
Quality Review Board?
14
The Quality Matrix
15
3D Quality Matrix Sample
16
Apply Feedback
17
Weigh the Risk

• Which bug fixes will be the least risky?
• If two have equal severity, fix the one with less
  risk.
• Generally, the larger the change, the larger the
  risk.
• Fixing a low severity problem can be risky.
• Theres no such thing as zero risk with software.

18
Summary

• Measure Quality
• Set reasonable quality targets
• Represent Stakeholders
• Balance Schedule, Scope, Cost and Quality
• Use feedback to get it right