Benchmarking for shortterm economic statistics

1
Benchmarking for short-term economic statistics

• Richard McKenzie David Brackfield OECD

2
Definition

• Benchmarking is the process of aligning estimates
  of an economic variable at high frequency (e.g.
  monthly / quarterly) with independent estimates
  of the same economic variable produced at low
  frequency (e.g. annual)
• Assumes the low frequency source is more accurate
  than the high frequency source

3
Background

• Presentation on methods at 2002 STESEG
• Presentation on specific conditions for
  benchmarking STES at 2003 STESEG
• OECD / Eurostat workshop on benchmarking in
  November 2003
• Collection of relevant technical documents
  brought together on OECD website also part of
  the STES Timeliness Framework
• Availability of the ECOTRIM software

4
Use of benchmarking techniques

• Common practice for quarterly and annual national
  accounts
• Not common practice amongst NSOs for other
  short-term economic statistics
• Advantages of using benchmarking techniques for
  STES emphasised in EU-US comparison study from
  2001

5
Advantages of using benchmarking techniques for
short-term economic statistics

• Consistency in high and low frequency data for
  the same economic variable
• Procedure to review discrepancies in high and low
  frequency estimates improve estimation
  methods
• Improved accuracy lower sample
  sizes, lower costs or opportunity to improve
  timeliness
• Produces an accurate long time series of high
  frequency data important for
  empirical analysis

6
Advantages of using benchmarking techniques for
short-term economic statistics . cont

• Improved quality of input series to quarterly and
  annual national accounts helps
  minimise discrepancies for distinguishable
  components
• BEWARE
• Need to ensure less frequent (e.g. annual)
  estimates are consistent between years to be
  suitable for benchmarking

7
Is benchmarking for STES a priority?

• Judging from countries comments on the
  benchmarking paper ..
• 40 high, 50 medium, 10 low
• Some significant country comments
• Very important due to the push towards using more
  administrative data for STES
• Needs to be considered in conjunction with the
  design of structural surveys
• More discussion needed on the series requiring
  benchmarking . (Recent Eurostat study)
• Data confrontation at unit record level for
  annual and short-term surveys is more important

8
Achieving international comparabilityECOTRIM

• Software package developed by Eurostat
• Made freely available after the OECD / Eurostat
  workshop on benchmarking
• 14 organisations have requested the software to
  use on a a variety of topics
• Contains internationally recognised techniques
  and there are plans for further development
• Country comments on the benchmarking paper
• 50 interested in trialling, 50 needed more
  information

9
ECOTRIM Software

• Software is freely available and can be installed
  onto the desktop
• A User Manual that includes case studies and
  examples
• Program is windows based and follows the normal
  Windows layout and feel

10
Simple Example (Univariate)

• In this example a monthly OECD MEI Production
  Index (industry excluding construction) will be
  benchmarked against the equivalent annual Gross
  Domestic Product in industry index for the same
  country. The result is a benchmarked monthly IIP
  estimate.
• First the aggregated series is loaded into
  ECOTRIM.
• Next the related series is loaded into ECOTRIM.

11
Univariate Methods

• The next step is to decide on a univariate
  method. All possible combinations can be tried
  (simultaneously if wanted).
• Possible methods are
• AR (1) MIN SSR or AR (1) MAX LOG
• Fernandez
• Litterman MIN SSR or Litterman MAX LOG

12
Results

• The results can be displayed in the system
  graphically
• The results can be exported into Excel with or
  without full information
• Software can benchmark using multivariate methods
  with contemporaneous constraints and preliminary
  estimates. Software can also run in batch mode.

13
Possible Future Work

• Issues for discussion
• Possible area of future work if countries want to
  trial benchmarking and present results next year?
• OECD could coordinate this work and provide
  comparative analysis of lessons learnt.
• Further discussion?