Software Prediction Models

1
Software Prediction Models

• Forecasting the costs of software development

2
Prediction Study Outcomes Vary

• Estimation-by-analogy beats regression
• Or not
• Classification and regression trees (CART) beats
  regression
• Or not
• Artificial neural networks beat regression
• Or not

3
Why Are The Results Conflicting?

• Poor data or research procedure
• Complex techniques may require expert users
  hence applications may vary
• Small sample size
• Measurement process that is flawed
• Selective use of differing parameters may result
  in different rankings

4
Key Terms

• Accuracy indicator
• Some measure of a process
• A summary statistic based on that measure
• Leave-one-out cross-validation
• Arbitrary function approximator taxonomy
• Many-data versus sparse-data
• Linear versus nonlinear
• Supervised versus unsupervised
• Reliability versus validity

5
Indicator 1 MMRE

• Mean magnitude of relative error (MMRE) is an
  average where the MREactual-prediction/actual
• Claimed advantages of MMRE
• Compare across data sets
• Independent of units
• Compare across differing prediction models
• Scale independence
• An hypothesis challenged by this paper

6
Indicator 2 MER

• Magnitude of the error relative to the estimate
  (MER) is defined asMER actual-prediction/pred
  iction

7
Indicator 3 AR

• The absolute residual (AR) is defined asAR
  actual-prediction

8
Other Measures

• Standard deviation (SD)
• Relative standard deviation (RSD)
• Log standard deviation (LSD)
• Balanced relative error (BRE)
• Inverted balanced relative error (IBRE)

9
Standard Deviation of Residuals, Denoted SD
10
Algebraic Simplification
11
Relative Standard Deviation (RSD)
12
Log Standard Deviation (LSD)
13
Balanced Relative Error (BRE)
14
Inverted Balanced Relative Error (IBRE)