Assigning Metrics for Optimization

1
Assigning Metrics for Optimization
2
Evaluation Measures

• Each evaluation measure (EM) is a category by
  which an option is ranked/graded
• Example A car can be ranked by mileage, top
  speed, number of gears, and seating capacity.
• A metric (or value function) must be established
  for each EM

3
Value Functions

• A single value metric must be specified for each
  EM
• Sometimes, lower ratings are preferred
• Example lower cost
• Sometimes, higher ratings are preferred
• Example higher mileage
• Choose the one that is applicable to the case.
• The scale of each metric must then be normalized
  to a range of 1.

4
Choosing a Value Function for an Evaluation
Measure

• There are two types of value functions
• Piecewise linear
• Used for arbitrary scales of performance,
  attractiveness, etc.
• Few data points are available
• Good for expressing discontinuity
• Exponential
• Used for stress, deflection, and other physical
  or concrete factors.
• Many data points can be obtained
• Suitable for incorporating a risk factor

5
Example Piecewise Linear Value Function

• The x-axis shows the grading scale for
  productivity (chosen arbitrarily)
• The y-axis shows the relative value of each grade
• In this case, an improvement from a grade of -1
  to 0 is as much as an improvement from 0 to 2.

6
Risk Tolerance

• How daring is the decision maker?
• Risk tolerant report a better score than is
  calculated by the metric (positive r)
• Risk averse report a worse score than is
  calculated (negative r)
• Risk neutral report the actual score (r?)
• The risk level r can be
• Calculated using the method in the next slide
• Estimated by looking at the different graphs
  shown in the following slides or created by a
  simulation.

7
Technical Method for Solving ?

• ? gt 0.1 Range of Measurement
• Find mid-value score (i.e. value 0.5)
• Assign value of 0.5 to a specific score
• Solve numerically or use a table
• Calculate normalized mid-value (range of scores ?
  1)
• Find normalized ?
• De-normalize ? by multiplying by the range

8
Exponential Value Function

• Higher scores are better
• Risk tolerance r
• r gt 0 risk tolerant
• r lt 0 risk averse

9
Exponential Value Function (cont.)

• Lower scores are better

10
Final Evaluation

• Combine value function grades with their
  respective weights to calculate the final
  grade/value

11
Bicycle Example

• In our example, the EMs are
• Cost
• Measured in dollars
• Lower score is preferred
• Weight
• Measured in pounds
• Lower score is preferred
• Lifetime
• Measured in months
• Higher score is preferred

12
Bicycle Example (cont).

• We use the exponential value function.
• Use a risk-averse outlook for personal safety,
  e.g. set ? -5.
• The exponential value function has been built
  into an MS Excel module and will be
  explained/utilized in later lectures.