Multicriteria evaluation

1
Multi-criteria evaluation
2
Roadmap

• Outline
• Introduction
• Definitions
• Multi-criteria evaluation (MCE)
• Example
• Multi-objective land allocation (MOLA)
• Example

3
Introduction

• Land is a scarce resource
• essential to make best possible use
• identifying suitability for
• agriculture
• forestry
• recreation
• housing
• etc.

4
Sieve mapping

• Early methods
• Ian McHarg (1969) Design with Nature
• tracing paper overlays
• landscape architecture and facilities location
• Bibby Mackney (1969) Land use capability
  classification
• tracing paper overlays
• optimal agricultural land use mapping

5
GIS approaches

• Sieve mapping using
• polygon overlay (Boolean logic)
• cartographic modelling
• Example uses
• nuclear waste disposal site location
• highway routing
• land suitability mapping
• etc.

6
Question

• What problems or limitations are there with the
  sieve mapping approach?

7
Definitions

• Decision a choice between alternatives
• Decision frame the set of all possible
  alternative
• Parks Forestry
• Candidate set the set of all locations / pixels
  that are being considered
• all Crown lands
• Decision set the areas assigned to a decision
  (one alternative)
• all pixels identified as Park

8

• Criterion some basis for a decision. Two main
  classes
• Factor enhances or detracts from the suitability
  of a land use alternative (OIR) e.g., distance
  from a road
• Constraint limits the alternatives (N) e.g.,
  crown/private lands
• Can be a continuum from crisp decision
  rules(constraints) to fuzzy decision rules
  (factors)
• Goal or target some characteristic that the
  solution must possess (a positive constraint)

9
Definitions

• Decision rule the procedure by which criteria
  are combined to make a decision. Can be
• Functions numerical, exact decision rules
• Heuristics approximate procedures for finding
  solutions that are good enough
• Objective the measure by which the decision rule
  operates (e.g., minimize pollution)
• Evaluation the actual process of applying the
  decision rule

10
Kinds of evaluations

• Single-criterion evaluation
• Multi-criteria evaluation to meet one objective,
  several criteria must be considered
• Multi-objective evaluations
• Complementary objectives non-conflicting
  objectives (e.g., extensive grazing and
  recreational hiking)
• Conflicting objectives both cannot exist at the
  same place, same time (e.g., ecological reserves
  and timber licenses)

11
Multi-criteria evaluation

• Basic MCE theory
• Investigate a number of choice possibilities in
  the light of multiple criteria and conflicting
  objectives (Voogd, 1983)
• generate rankings of choice alternatives
• Two basic methodologies
• Boolean overlays (polygon-based methods)
• Weighted linear combinations (WLC) (raster-based
  methods)

12

• Multicriteria analysis appeared in the 1960s as a
  decision-making tool. It is used to make a
  comparative assessment of alternative projects or
  heterogeneous measures. With this technique,
  several criteria can be taken into account
  simultaneously in a complex situation. The method
  is designed to help decision-makers to integrate
  the different options, reflecting the opinions of
  the actors concerned, into a prospective or
  retrospective framework. Participation of the
  decision-makers in the process is a central part
  of the approach. The results are usually directed
  at providing operational advice or
  recommendations for future activities.

13

• Multicriteria evaluation be organised with a view
  to producing a single synthetic conclusion at the
  end of the evaluation or, on the contrary, with a
  view to producing conclusions adapted to the
  preferences and priorities of several different
  partners.
• Multicriteria analysis is a tool for comparison
  in which several points of view are taken into
  account, and therefore is particularly useful
  during the formulation of a judgement on complex
  problems. The analysis can be used with
  contradictory judgement criteria (for example,
  comparing jobs with the environment) or when a
  choice between the criteria is difficult.

14
MCE

• Non-monetary decision making tool
• Developed for complex problems,where uncertainty
  can arise if a logical, well-structured
  decision-making process is not followed
• Reaching consensus in a (multidisciplinary) group
  is difficult to achieve.

15
MCE techniques

• Many techniques (decision rules)
• Most developed for evaluating small problem sets
  (few criteria, limited candidate sets)
• Some are suitable for large (GIS) matrices
• layers criteria
• cells or polygons choice alternatives
• Incorporation of levels of importance (weights
  WLC methods)
• Incorporation of constraints (binary maps)

16
MCE pros and cons

• Cons
• Dynamic problems strongly simplified into a
  linear model
• Static, lacks the time dimension
• Controversial method too subjective?

• Pros
• Gives a structured and traceable analysis
• Possibility to use different evaluation factors
  makes it a good tool for discussion
• Copes with large amounts of information
• It works!

17
MCE pros and cons

• MCE is not perfectquick and dirty-option,
  unattractive for real analysts
• but what are the alternatives? - system
  dynamics modelling impossible for huge
  socio-technical problems - BOGSAT is not
  satisfactory (Bunch of Old Guys/Gals Sitting
  Around a table Talking)
• MCE is good for complex spatial problems
• Emphasis on selecting good criteria, data
  collection and sensitivity analysis

18
Principles of MCE

• Methodology
• Determine criteria (factors / constraints) to be
  included
• Standardization (normalization) of factors /
  criterion scores
• Determining the weights for each factor
• Evaluation using MCE algorithms
• Sensitivity analysis of results

19
Determine the criteria to be included

• Oversimplification of the decision problem could
  lead to too few criteria being used
• Using a large number of criteria reduces the
  influence of any one criteria
• They should be comprehensive, measurable,
  operational, non-redundant, and minimal
• Often proxies must be used since the criteria of
  interest may not be determinable (e.g., slope
  is used to represent slope stability)
• A multistep, iterative process that considers the
  literature, analytical studies and, possibly,
  opinions

20
Factor normalization

• Standardization of the criteria to a common scale
  (commensuration)
• Need to compare apples to apples, not apples to
  oranges to walnuts. For example
• Distance from a road (km)
• Slope ()
• Wind speed
• Consider
• Range (convert all
• to a common range)
• Meaning
• (which end of the
• scale good)

Good 255
255
Output
Output
Poor 0
0
low
high
low
high
Input
Input
21
Fuzzy membership functions
Used to standardize the criterion
scores Linguistic concepts are inherently
fuzzy (hot/cold short/tall)
Graphs of the Fuzzy Memberships within
IDRISI (Based on Eastman 1999)
22
Factor normalization example
Cholera Health Risk Prediction in Southern
Africathe relation between temperature and risk
23
Determine the weights

• A decision is the result of a comparison of one
  or more alternatives with respect to one or more
  criteria that we consider relevant for the task
  at hand. Among the relevant criteria we consider
  some as more important and some as less
  important this is equivalent to assigning
  weights to the criterion according to their
  relative importance.
• By normalizing the factors we make the choice of
  the weights an explicit process.

24
Analytical hierarchy process

• One of the more commonly-used methods to
  calculate the weights.

Refer to description of ArcGIS extension ext_ahp.
25
MCE Algorithms

• The most commonly used decision rule is the
  weighted linear combination
• where
• S is the composite suitability score
• x factor scores (cells)
• w weights assigned to each factor
• c constraints (or boolean factors)
• ? -- sum of weighted factors
• ? -- product of constraints (1-suitable,
  0-unsuitable)

S ?wixi x ?cj
26
MCE

• A major difference between boolean (sieve
  methods) and MCE is that for boolean and
  methods every condition must be met before an
  area is included in the decision set. There is
  no distinction between those areas that fully
  meet the criteria and those that are at the
  edges of the criteria.
• There is also no room for weighting the factors
  differentially.

27
Example weighted linear summation
Example
28
Sensitivity analysis

• Choice for criteria (e.g., why included?)
• Reliability data
• Choice for weighing factors is subjective
• Will the overall solution change if you use other
  weighing factors?
• How stable is the final conclusion?
• sensitivity analysis vary the scores / weights
  of the factors to determine the sensitivity of
  the solution to minor changes

29
Sensitivity analysis

• Only addresses one of the sources of uncertainty
  involved in making a decision (i.e., the validity
  of the information used)
• A second source of uncertainty concerns future
  events that might lead to differentially
  preferred outcomes for a particular decision
  alternative.
• Decision rule uncertainty should also be
  considered (? MCE itself)

30
Example MCEWind Farm Siting

• Dennis Scanlin
• (Department of Technology)
• Xingong Li
• Chris Larson
• (Department of Geography Planning)
• Appalachian State University

31
Spatial Analytical Hierarchy Process

• Wind farm siting
• Find the best wind farm sites based on siting
  factors
• Alternatives
• Locationinfinite
• Divide the space into squares/cells (200m 200m)
• Evaluate each cell based on the siting factors

32
Preliminary Siting Factors

• Accessibility to roads
• Distance to primary roads
• Distance to secondary roads
• Distance to rural roads
• Accessibility to transmission lines
• Distance to 100K lines
• Distance to 250K lines
• Distance to above250K lines
• Wind power (or wind speed)
• Visibility
• Viewshed size
• of people in viewshed

33
Siting Steps (MCE)

• Factor generation
• Distance calculation
• Visibility calculation
• Factor standardization (0 100)
• Each factor is a map layer
• Factor weights determination by AHP
• Final score
• Weighted combination of factors
• Exclusion areas

34
AHP
35
Factor Layers
36
Wind Turbine visibility--Viewshed
37
Wind Turbine Viewshed Size

• Red505km2
• Greed--805km2
• Blue--365km2
• Software tool developed to calculate viewshed
  size for each cell

38
Visibility FactorViewshed Size

• Computational expensive
• About 700,000 cells
• Each cell requires 10 seconds
• About 76 days
• Parallel computing
• 12 computers
• Each computer runs two counties
• About 55000 cells
• 6 days
• Succeed with 3000 cells but failed with 55,000
  cells

39
Visibility Factor-- of People in Viewshed
40
Final Score Layer
41
Candidate Sites
42
Constraints (binary)
43
Sites
44
Multi-objective land allocation

• Basic MOLA theory
• procedure for solving multi-objective land
  allocation problems for cases with conflicting
  objectives
• based on information from set of suitability maps
• one map for each objective
• relative weights assigned to objectives
• amount of area to be assigned to each land use
• determines compromise solution that attempts to
  maximize suitability of lands for each objective
  given weights assigned

45
Principles of MOLA

• Methodology
• construct ranked suitability maps for each
  objective using MCE
• decide on relative objective weights and area
  tolerances
• evaluate conflict demands on limited land via
  iterative process

46
MOLA decision space
255
Non-conflicting choices
Conflicting choices
Objective 2
Non-conflicting choices
Unsuitable choices
0
0
255
Objective 1
47
Example protected areas

• Multi-layered system in Britain
• National Parks, Areas of Outstanding Natural
  Beauty, Heritage Coasts, Special Areas of
  Conservation, Special Protection Areas, Sites of
  Special Scientific Interest, Nature Reserves,
  Ramsar Sites, National and Community Forests,
  Environmentally Sensitive Areas, National Scenic
  Areas, Regional Parks, Common Land, and Less
  Favoured Areas

48
Protected areas in Britain
49
Identifying wilderness areas

• Wilderness Britain?
• continuum of environmental modification from
  paved to the primeval (Nash, 1981)
• the Wilderness Continuum concept
• measurable and mappable?
• remoteness from settlement
• remoteness from mechanised access
• apparent naturalness (lack of human artefacts)
• biophysical naturalness (ecological integrity)

50
Factor maps
Apparent naturalness
Biophysical naturalness
Remoteness from mechanised access
Remoteness from settlement
51
Possible solutions
Stressing naturalness
Stressing remoteness
Equally weighted
52
Wild and recreational (city) parks output
Wild parks with without existing protected
areas constraint
City parks with without existing protected
areas constraint
53
MOLA Results wild parks vs city parks
Suitability for wild parks
Suitability for city parks
MOLA results (yellow wild parks, red city
parks, blue constraints)
54
Conclusions

• Few GIS packages provide MCE functionality (e.g.
  Idrisi)
• Most GIS provide facilities for building MCE
  analyses (e.g. ArcGIS modelbuilder)
• Important method for
• Site and route selection
• land suitability modelling