Antti Punkka and Ahti Salo

1
Incomplete Holistic Comparisons inValue Tree
Analysis

• Antti Punkka and Ahti Salo
• Systems Analysis Laboratory
• Helsinki University of Technology
• P.O. Box 1100, 02015 HUT, Finland
• http//www.sal.hut.fi/

2
Value tree analysis
Overall goal (a0)
Attribute 1(a1)
Attribute 5(a5)
Attribute 4(a4)
Attribute 3(a3)
Attribute 2(a2)
Alternative 1 (x1)
Alternative 3 (x3)
Alternative 2 (x2)
3
Value tree analysis

• m alternatives, Xx1,,xm , n attributes,
  Aa1,,an
• Additive value function
• Least and most preferred achievement levels
• all attributes relevant
• attribute weight wi represents the improvement in
  overall value when an alternatives achievement
  with regard to attribute ai changes from the
  least to the most preferred level

4
Weight elicitation

• Complete information
• captured by point estimates
• e.g., SMART (Edwards 1977)
• Incomplete information
• weight and weight ratio intervals
• e.g.,
• e.g., PAIRS (Salo and Hämäläinen 1992), PRIME
  (Salo and Hämäläinen 2001)
• Ordinal information
• ask the DM to rank the attributes in terms of
  importance
• e.g., rank sum weights (Stillwell et al. 1981)
• incomplete ordinal information (RICH Salo and
  Punkka 2003)

5
Incomplete information

• Complete information may be hard to acquire
• alternatives and their impacts?
• relative importance of attributes?
• e.g.,
• Alternatives overall values can be represented
  as intervals
• e.g., the smallest and the largest possible value
  can be solved through LP

where S is the feasible region for the attribute
weights based on the DMs preference statements
6
Pairwise dominance

• Alternative xk dominates xj in the sense of
  pairwise dominance
• dominated alternative is non-optimal whenever the
  DMs preference statements are fulfilled gt it
  can be discarded
• e.g., a problem with two attributes,
• Alternatives may remain non-dominated, however
• decision rules assist the DM in selection of the
  most preferred one

where S is the feasible region for scores and
weights
7
Decision rules

• Maximax
• alternative with greatest maximum overall value
• Maximin
• alternative with greatest minimum overall value
• Minimax regret
• alternative for which the greatest possible loss
  of value against some other alternative is the
  smallest
• Central values
• alternative with greatest sum of maximum and
  minimum overall value

8
Use of ordinal preference statements

• Complete ordinal information
• ask the DM to rank the attributes in terms of
  importance
• derive a representative weight vector from the
  ranking
• rank sum weights (Stillwell et al. 1981)
• rank reciprocal (Stillwell et al. 1981)
• rank-order centroid weights (Barron 1992)
• Incomplete ordinal preference information
• the DM may be unable to rank the attributes
• which is more important - economy or
  environmental impacts

9
Rank Inclusion in Criteria Hierarchies (RICH)

• Associate possible rankings with sets of
  attributes
• e.g., economy and environmental impacts are
  among the three most important attributes
• presumably easier and faster to give than
  numerical statements
• easy to understand
• statements define possibly non-convex feasible
  regions
• Supported by the decision support tool RICH
  Decisions http//www.rich.hut.fi

The most important of the three attributes is
either attribute 1 or 2
10
Ordinal information in evaluation of the
alternatives

• Numerical evaluation may be difficult
• may lead to erroneous approximations on
  alternatives properties (Payne et al. 1993)
• allow the DM to use incomplete ordinal
  information
• Score elicitation
• associate sets of rankings with sets of
  alternatives
• e.g., alternatives 1 and 2 are the two least
  preferred with regard to environmental impacts
• e.g., alternatives 3 and 4 are the two most
  preferred with regard to environmental impacts
  and cost together
• rank two alternatives in relative terms
• e.g., alternative 1 is better than alternative 2
  with regard to environmental impacts
• can be subjected to
• all attributes (holistic comparisons)
• a (sub)set of attributes or a single attribute

11
Incomplete holistic comparisons

• Evaluate some alternatives without decomposition
  into subproblems
• comparisons interpreted as (pairwise) dominance
  relations
• e.g., alternative x1 is better than alternative
  x2
• e.g., alternative x4 is not the most preferred
  one
• Constraints on the feasible region
• e.g., normalized scores known
• three attributes
• alternative x1 is preferred to alternative x2

12
Different forms of incomplete ordinal information
(Incomplete) ordinal information about
the importance of attributes (RICH)
(Incomplete) ordinal information about
alternatives, score information in form of
intervals
(Incomplete) holistic comparisons
LPs for 1) overall value intervals and 2)
pairwise dominance relations
Constraints on the feasible region
Decision recommendations
13
Rank-orderings (1/2)

• Rank the alternatives subject to their properties
• the most preferred alternative has the ranking
  one, etc.
• e.g., alternatives x1, x2 and x3 ranked with
  regard to cost r(r(x1), r(x2), r(x3))(1,2,3)
• alternative x1 is the preferred to x2 which is
  preferred to x3
• the alternative with a smaller rank with regard
  to some attributes has greater sum of scores with
  regard to these attributes
• mathematically rAX is a bijection from X?X
  onto 1,,m, Xm
• Compatible rank-orderings
• I?X is a set of alternatives, J?1,,m a set
  of rankings
• if IltJ, the rankings of alternatives in I are
  in J
• if I?J , the rankings in J are attained by
  alternatives in I
• many compatible rank-orderings
• e.g., if m3, Ix1, J1 for Aa1, a2, then
  compatible rank-orderings are rAX(1,2,3) and
  (1,3,2).

14
Rank-orderings (2/2)

• Feasible region associated with a rank-ordering
  rAX convex
• can be used as an elementary set
• R(I,J) contains the rank-orderings that are
  compatible with the sets I and J
• feasible regions defined by R(I,J) not
  necessarily convex
• Express statements as pairs of Ii, Ji, i1,,k
• feasible region is the intersection of the
  corresponding S(Ii,Ji)s

15
Efficiency of preference statements

• Monte Carlo study
• randomly generated problem instances (e.g.,
  Barron and Barret 1994)
• statements are based on them
• e.g., weight vector w(0.32, 0.60, 0.08)
  approximated through the rank-ordering r(2,1,3)
• correct choice, xC(i) at round i, (i.e., the
  alternative with the highest overall value) can
  be obtained
• xe(i) is the alternative recommended by a
  decision rule at round i
• Measures
• expected loss of value (ELV)
• percentage of correct choices (PCC)
• average number of non-dominated alternatives

ns is the number of simulation rounds
nC is the number of problems where xe(i) xC(i)
16
Efficiency of holistic comparisons

• Questions
• how effective are holistic comparisons?
• differences between strategies in choosing the
  compared alternatives
• Randomly generated problems
• n5,7,10 attributes m5,7,10,15,50 alternatives
• each weight vector has the same probability
• scores completely known, randomly generated
• uniform distribution, Uni0,1
• triangular distribution, Tri(0,1/2,1)
• Three strategies for choosing the alternatives
  for pairwise comparisons
• each applied in two different ways
• disconnected comparisons, x1 vs. x2, x3 vs. x4,
  etc.
• chained comparisons, x1 vs. x2, x2 vs. x3, etc.

17
Simulation layout

• Elicitation strategies
• A. arrange the alternatives in a descending order
  by the sum of the scores (strategies SoS1
  (disconnected) and SoS2 (chained))
• B. arrange the alternatives in a descending order
  by the score of the most important attribute
  (strategies MIA1 and MIA2)
• C. arrange the alternatives randomly (strategies
  Rnd1 and Rnd2)
• ELV and PCC was studied using central values,
  maximax, maximin and minimax regret decision
  rules
• 100 problem instances (simulation rounds)
• several linear programs are needed
• results indicative
• parameter variation (m,n and the number of
  comparisons) leads to 114 combinations,
  experiments

18
Simulation results (1/2)

• Sum of Scores is the best strategy
• SoS1 outperforms MIA1 in 113 of 114 experiments
  in terms of ELV
• in 82 of these the difference in loss of value
  significant
• risk level at 2.5 for a 1-tailed t-test
• SoS1 outperforms Rnd1 in every of the experiments
  in terms of ELV
• in 92 of these the difference in loss of value
  significant
• no clear difference between MIA1 and Rnd1
• Chained comparisons are better than disconnected
  comparisons in terms of ELV and percentage of
  correct choices

19
Simulation results (2/2)

• Holistic comparisons reduce the number of
  non-dominated alternatives efficiently
• e.g., m50, n5, the average number of
  non-dominated alternatives was between 3.92 and
  9.17 with 10 comparisons, depending on the
  strategy
• e.g., m50, n5, with only one comparison the
  average number of non-dominated alternatives was
  between 20.57 and 23.69
• by discarding one alternative, an average of
  almost 30 were eliminated
• Triangularity assumption increases efficiency

20
Conclusion

• Incomplete ordinal information enhances
  possibilities in preference elicitation
• presumably easier and faster to give than
  numerical statements
• easy to understand
• Screening of alternatives
• holistic comparisons efficient in discarding
  non-optimal alternatives
• useful especially in problems with many
  alternatives
• consequences of alternatives may be
  time-consuming to obtain
• constraints on the feasible region
• Further research directions
• efficient computational procedures
• simulation study on the efficiency of incomplete
  holistic comparisons
• implementation of a decision support system
• case studies

21
Related references

• Barron, F. H., Selecting a Best Multiattribute
  Alternative with Partial Information about
  Attribute Weights, Acta Psychologica 80 (1992)
  91-103.
• Barron, F. H. and Barron, B. E., Decision
  Quality using Ranked Attribute Weights,
  Management Science 42 (1996) 1515-1523.
• Edwards, W., How to Use Multiattribute Utility
  Measurement for Social Decision Making, IEEE
  Transactions on Systems, Man, and Cybernetics 7
  (1977) 326-340.
• Payne, J. W., Bettman, J. R. and Johnson, E. J.,
  The Adaptive Decision Maker, Cambridge
  University Press, New York (1993).
• Salo, A. ja R. P. Hämäläinen, "Preference
  Assessment by Imprecise Ratio Statements,
  Operations Research 40 (1992) 1053-1061.
• Salo, A. and Hämäläinen, R. P., Preference
  Ratios in Multiattribute Evaluation (PRIME) -
  Elicitation and Decision Procedures under
  Incomplete Information, IEEE Transactions on
  Systems, Man, and Cybernetics 31 (2001) 533-545.
• Salo, A. and Punkka, A., Rank Inclusion in
  Criteria Hierarchies, (submitted manuscript
  2003).
• Stillwell, W. G., Seaver, D. A. and Edwards, W.,
  A Comparison of Weight Approximation Techniques
  in Multiattribute Utility Decision Making,
  Organizational Behavior and Human Performance 28
  (1981) 62-77.