Normalisation

1
Normalisation

• Stefano Tarantola

2
Purpose of normalisation

• Scope to bring indicators to the same
  measurement unit
• Indicators can be originally incommensurate or
  (patents granted per population in thousands and
  high-tech exports as of total exports)
• Can be commensurate yet expressed in different
  units (temperature in Celsius or Kelvin or
  Fahrenheit)

3
Several methods exist

• Ranking
• Standardisation
• Re-scaling
• Distance to reference country
• Etc.

4
Select the proper method

• Take into account properties of data
• Whether hard or soft data are available
• Whether exceptional behaviour needs to be
  rewarded/penalised
• Objectives of the composite indicator
• Whether info on absolute levels matters
• Whether benchmarking against a reference country
  is requested
• Whether variance in the indicators needs to be
  accounted for

5
Select the proper method

• For example in presence of extreme values we
  should prefer normalisation methods that are
  based on standard deviation or distance from the
  mean
• Special care in the choice of the normalisation
  when the composite indicator needs to be
  comparable over time

6
Scale of measurement

• An indicator can be expressed in several
  measurement scales (interval, ratio)
• Attenion must be paid to this because it may
  interfere with normalisation.
• We must assure that the normalisation method
  selected is invariant to the measurement scale
  for all the indicators considered

7
Scale of measurement

• Normalisation methods are said to be invariant to
  changes in measurement unit of the indicator
  when they provide the same normalised values
  whatever the measurement unit of the indicator
  is.
• Using a normalisation method that is not
  invariant to changes in the measurement unit
  could result in different outcomes for the
  composite indicator.

8
example

• Two indicators (temperature and humidity) for two
  countries A and B and for two years 2003 and 2004.

9
example

• If we use the normalisation called re-scaling
  with respect to the country leader and we
  aggregate using equal weights we see that the
  composite for country A has a positive trend

10
example

• Now if we repeat the same exercise using
  temperature in Fahrenheit, the composite for A
  now has a negative trend

11
example

• This would not occur if we used standardisation
  or ranking across countries.
• The variable temperature used here is a case of
  interval scale

12
example

• Another common change of measurement scale is
  ratio scale, which is a subset of interval
  scale

13
Scale transformations

• Two types of transformations are sometimes
  applied to raw data prior to normalisation
  truncation and functional transformation
• Truncation of the tails ? to avoid having extreme
  values overly dominate the results, and also to
  correct for data quality problems

14
Scale transformations

• Functional transformation ? to modify the
  importance of marginal changes in the level of a
  given indicator
• Linear functional form (changes in the indicator
  values are important in the same way, regardless
  to the level)

15
Scale transformations

• Log or n-th root (when changes are more
  significant at lower levels of the indicator,
  e.g. the level of internet connection is less
  important in advanced countries (95) than in
  developing countries (20)).

• In other words the log shirnks the range on its
  right-hand side. As values approach zero they are
  penalised because after transformation they
  become negative.

16
Log-transformation

• When we perform linear aggregation of indicators
  expressed in logs, this is equivalent to a
  geometric aggregation of the raw indicators.

17
Log-transformation

• The ratio of two weights indicates the of
  improvement in one indicator that would
  compensate for 1 decline in another indicator.

18
Log-transformation

• With this transformation a unit improvement
  starting from a low level is much more important
  of the same unit improvement starting from a high
  level of performance

19
Ranking across countries

• Simplest method
• Independence on outliers
• Loss of information on absolute levels
  (impossible to draw any conclusion about
  difference in performance)

20
Ranking across countries (2)

• For time-dependent studies
• Ranking carried out for each point in time
• We cannot follow the performance of one country
  across time
• perhaps the country improves yet its ranking
  deteriorates as other countries improve faster

21
Standardisation (z-scores)

• An indicator with extreme values will have a
  greater effect on the composite indicator
• This can be desirable if the intention is to
  reward exceptional behaviour (e.g. if an
  extremely good result in few indicators is
  thought to be better than a lot of average
  scores)
• Important to assure that extreme values are not
  outliers

22
Re-scaling

• All normalised indicators have identical range
  (01)
• Example innovation scoreboard (-0.5 0.5)
• Extreme values could be unreliable outliers
  (distortion effect)
• Indicators lying within an interval with very
  small range, this latter is widened applying
  re-scaling,thus increasing intrinsically the
  weight of that indicator
• Example e-business readiness index

23
Re-scaling (2)

• For time dependent studies there might be a
  drawback if

24
Re-scaling (3)

• The solution would be

• The transformation is not stable when data for a
  new time point become available
• Solution recalculate the composite on all time
  points to maintain comparability

25
Distance to a reference country

• The reference could be a target to be reached in
  a given time frame
• Or an external benchmarking country
• Or the average country
• Or the group leader
• beware that the reference value is not an
  unreliable outlier

26
Distance to a reference country (2)

• The reference could be the country itself at an
  initial time point

27
Categorical scales

• Categories are selected
• They can be numerical (, , )
• Or qualitative (fully achieved, partially
  achieved, etc.)
• Each category is assigned a score (arbitrary)

28
Categorical scales (2)

• Scores can be based on the distribution of the
  indicator across countries eg 100 points to the
  upper 5th percentile,down to 0 points to the
  lower 5th percentile
• With this discretisation we omit a large amount
  of information about the variance between units
  in the transformed indicators
• however any small variation in the definition of
  the indicator that could occur with years will
  not affect the transformed variable

29
Categorical scales (3)

• Example Nicoletti et al., 2003 (OECD)
• Index of product market regulation based on the
  OECD Regulatory Indicators Questionnaire
• Both quantitative and qualitative data
• Qualitative information is coded by assigning a
  numerical value to each of its possible
  modalities (ranging from a negative to a positive
  answer)
• Quantitative information is sub-divided into
  classes
• The resulting coded information is normalised by
  ranking it on a common 0-6 scale, reflecting
  increasing restrictiveness of the regulatory
  provisions

30
Above or below the mean

• Arbitrarily defined threshold p
• The threshold builds a neutral region around the
  mean.
• Pro Not affected by outliers
• Con Omission of absolute level information

31
Cyclical indicators

• Used by institutes conducting business tendency
  surveys
• To reduce the risk of false signals and to better
  forecast cycles in economic activities
• Example OECD or EC leading indicators

32
Cyclical indicators (2)

• The higher the oscillation, the higher the
  denominator
• This minimises the influence of series with
  marked cyclical amplitude (OECD)

33
Cyclical indicators (3)

• Another normalisation is based on balance of
  opinions
• managers are asked if the firms have improved or
  deteriorated wrt the previous survey
• The method gives implicitly less weight to the
  more irregular series
• This minimises the influence of series with
  unstable behaviour (EC - economic sentiment index)

34
of annual differences over consecutive years

• growth with respect to the previous year
  instead of the absolute level
• Internal Market Index 2001

35
Robustness tests

• Different normalisation methods can be found
  eligible for use.
• They will provide different results for the
  composite indicator.
• Are these results robust or not?
• Robustness tests must be carried out to check
  this.