Automated ShortRun Economic Forecasts ASEF

1
Automated Short-Run Economic Forecasts (ASEF)

• Bank of Canada
• October 25-26, 2007
• Nicolas Stoffels
• nicolas.stoffels_at_snb.ch

2
Introduction

• Motivation
• Global forecasts at the SNB
• ASEF Framework
• Application to U.S. GDP growth
• Summary

3
Motivation

•  policymakers are well advised to follow two
  principles familiar to navigators throughout the
  ages First, determine your position frequently.
  Second, use as many guides or landmarks as are
  available. 
• B. Bernanke, March 2006
• Releases of the official value of GDP and other
  national accounts data are published late
• ? Global economic developments have to be
  assessed from numerous and more timely indicators
• Incorporate higher-freqency indicators into
  statistical, quantitative forecasting models?
  e.g. Pain and Sédillot (2003, 2005), Ingenito and
  Trehan (1996)
• Two main Goals 1. Exploit the considerable
  amount of conjunctural information available
  before release of official national accounts
  data2. Build an automated process for an
  efficient day-to-day use
• ? SNB Project Automated procedure to select and
  run optimal, indicator models for short-term
  forecasting of the international economy

4
SNBs global forecasts cycle
Quarterly cycle 13 weeks
Board meeting
Interim assessment of Base scenario
Defining mid-term scenarios for the global
economy Base scenario 2-3 risk scenarios
(requiring Board approval)
Weekly review of incoming indicators
Update of short-term forecasts
Update of short and medium-term forecasts
Update of short and medium-term forecasts
5
Lexicon

• Hard indicatorsQuantitative data (e.g.
  industrial production, retail sales )
• Soft indicatorsQualitative data (e.g. Business
  / consumer surveys)
• Main-Bridge equationForecast equation for the
  target variable (quarterly frequency)
• Auxiliary variablesHigh-frequency indicators
  that help to forecast regressors in the
  Main-Bridge equation
• Mini-Bridge equationForecast equation for
  monthly indicators used as regressors in the
  Main-Bridge equation

6
Procedure
Representation
Universe of Indicators (W)
Stage 1 Indicator selection
Subset of Indicators (S ? W)
Stage 2 Specification and estimation of the
Main-Bridge equation
Main-Bridge equation
7
Representation
Process
Stage 3 Evaluation of Main-Bridge equation
Main-Bridge equation
Incoming data
Stage 4 Automated forecasts
Forecasts
8
Structure
Softwares - Eviews - Perl
Main Folder
Database
Model
Subroutines
crunching.prg ranking.prg tests.prgforecasts.prg
dataimport.prg monthly.wf1 quarterly.wf1 db.edb
Extensions
Programs
Results
results1_m/q.wf1 results2_m/q.wf1 results3_m/q.wf1
results4_m/q.wf1
combination3.prg combination4.prg markup3.prg
aggregation.prg
Stage4
Stage3
Stage2
Stage1
selection1.prg settings1_m.prgsettings1_q.prg se
tpath.prg
selection2.prg settings2_m.prgsettings2_q.prg se
tpath.prg
evaluation.prg settings3_m.prg settings3_q.prg se
tpath.prg
forecast.prg settings4_m.prg settings4_q.prg setp
ath.prg
9
ASEF Stage 1 Indicator selection

• Finds the dominant predictors
• Ranks high-frequency indicators according to
  their in-sample fit
• Tests for potentially useful combination of
  forecasts (encompassing). Modified
  Diebold-Mariano t-test.

10
ASEF Stage 1 Indicator selection

• Estimation of K bivariate distributed lags OLS
  regressions with lagged dependent variable
• ? Optimal lag specification for every
  (stationary) indicator xi,t
• 2. Ranking of the K best models according to a
  specified criterion (AIC, SIC or adj. R2)
• 3. Encompassing bivariate tests (Modified
  Diebold-Mariano t-test (Harvey, Leybourne and
  Newbold, 1997) )
• ? Tests whether the K-1 variables contain
  additional information to the best single
  variable equation

11
ASEF Stage 1Application

• Target variable Quarterly US GDP growth
  (saar)
• Data set 189 hard indicators
• Max of lags 2 (for exogenous) 4 (for
  target)
• of specifications 24192 (189 x 27)
• Criterion adj. R2
• Comp. time 80 sec.

12
ASEF Stage 1 Output
13
ASEF Stage 1 Additional feature

• Possibility to use Stage 1 also for a monthly
  target variable.

14
ASEF Stage 2 Selection of the Main-Bridge
equation

• Find the optimal specifications regarding lags
  and variables (e.g. selected from Stage 1)
• Ranks specifications according to in-sample
  criteria (SIC, AIC)

15
ASEF Stage 2 Main-Bridge equation

• Multivariate distributed lag regression with
  lagged dependent variable? Best Main-Bridge
  equation
• where regressors xi,t a subset of S indicators

16
ASEF Stage 2 Application

• Indicators IP manuf, PCE and Private
  constr.
• Sample 1987q2-2007q2
• Criterion SIC
• Max. of lags 2 (exogenous) 4 (target)
• of specifications 8192 (213)
• Comput. time 37 sec.

17
ASEF Stage 2 Output
18
ASEF Stage 2 Additional features

• Possibility to find equations for variables in
  monthly frequency

19
ASEF Stage 3 Main-Bridge evaluation

• Evaluates the Main-bridge equation in pseudo
  out-of-sample forecasting exercise against 1 or 2
  benchmarks
• Input Main-Bridge (e.g. from Stage 2) and
  benchmark equation (e.g. univariate)
• Output 4 tests of forecasts properties

20
ASEF Stage 3 4 accuracy tests

• 1. Forecast accuracy
• Test of Mean Squared Errors equality. Modified
  Diebold-Mariano t-test (Harvey, Leybourne and
  Newbold, 1997)
• 2. Directional accuracyTests if the direction
  forecast is significantly different from a random
  draw. Independence Chi-Squared-test. (Diebold and
  Lopez, 1996)
• 3. Forecast bias
• Tests the null of unbiased forecasts. t- and
  F-test
• 4. Normality of residualsJarque-Berra
  (Chi-Squared) -test

21
ASEF Stage 3 Main features

• Possibility to simulate different degree of
  information inside the quarter (1 month, 2
  months, full quarter)
• -gt Values of missing predictors forecasted with a
  VAR or the mean of the previous month(s)

22
ASEF Stage 3 Expanding forecast(with 1 month
of information)
VAR estimation (monthly)
M1
M2
M3
M4
M5
M6
Forecast (monthly)
Main-Bridge forecast (quarterly)
For Qt
For Qt1
Etc
23
ASEF Stage 3 Application

• Main-Bridge (SNB) _at_pca(gdp) c _at_pca(gdp(-1))
  _at_pca(pce) _at_pca(ipman) _at_pca(privconst)
• Benchmark 1 (FRSF-Model, Ingenito and Trehan,
  1996) _at_pca(gdp) c _at_pca(gdp(-1)) _at_pca(gdp(-2))
  _at_pca(gdp(-3)) _at_pca(pce) _at_pca(emp)
• Benchmark 2 Best univariate for _at_pca(gdp) (up to
  4 lags)
• Start of estimation period 1990q2
• Out-of-Sample forecast period 1999q1-2007q2
• Computing time 11 sec.

24
ASEF Stage 3 Output
25
ASEF Stage 3 Additional features

• Lag specification of VAR is automatically chosen
  (AIC, SIC, Hanna-Quinn) or can be imposed
• May also be used with equations containing
  quarterly regressors
• Evaluation of n-steps ahead forecasts
• Test whether rolling optimization (for
  coefficients, variables and lags) improves
  forecasts
• Any time series can be used as a benchmark (e.g.
  from external sources/models)
• Forecasts of several equations can be computed
  and weighted (mean, median..) and then compared
  to a benchmark

26
ASEF Stage 4 Automated forecasts

• Automated forecasting tool for day-to-day use
• Exploit as much information as possible by
  considering all monthly regressors available

27
ASEF Stage 4 Automated forecasts (contd)

• Problem of incomplete information Monthly
  predictors are often only partially available
  within a given quarter
• ? Dealing with staggerred data Mini-Bridge
  equations to forecast missing data of monthly
  indicators
• zh denote H auxiliary variables that wont be
  used in the Main-Bridge equation but are useful
  for forcasting the predictors x
• xj are the variables to be forecasted

28
ASEF Stage 4 Automated forecasts

• In order to find good auxiliary variables, Stage
  1 may be used with monthly dependent variables
• To find the optimal Mini-Bridge equation, Stage 2
  may also be used

29
ASEF Stage 4 Automated forecasts

• Once all predictors have the same end date, a VAR
  is estimated to compute a forecast up to the
  specified horizon
• Estimating then the best Main-Bridge equation and
  computing a forecast
• Optionally Skipping the Mini-Bridge and VAR
  forecasts and filling up the missing values with
  the mean of the previous month(s)

30
ASEF Stage 4 Automated forecasts
Mini-Bridge Forecast (monthly)
M1
M2
M3
M4
M5
M6
Var 1
VAR Forecast (monthly)
Var 2
Main-Bridge Forecast (quarterly)
Var 3
Target
31
ASEF Stage 4 Application

• Main-Bridge _at_pca(gdp) c _at_pca(gdp(-1)) _at_pca(pce)
  _at_pca(ipman) _at_pca(privconst)
• PCE and construction data ? Mini-Bridge equations
  to forecast regressors
• Auxiliary variables Retail sales, Hours worked
  in construction
• Number of tested Mini-Bridge specifications
  2304
• VAR to forecast monthly data up to 2010q1
• Computation time 10 sec.

32
ASEF Stage 4 Output
33
ASEF Stage 4 Output (contd)
34
ASEF Stage 4 Additional features

• Forecasts with equations based on quarterly data
• Forecasts where missing monthly predictors is
  completed with the mean of the previous month(s)
• Combination of forecasts can run a large set of
  forecasting equations and weight the results
  (mean, median) useful assessment of the balance
  of risks

35
Results

• Figure 1
• Forecast accuracy significantly above benchmarks
• First month of data particularly important
• Figure 2
• Hit-ratio above 50 much higher than the
  univariate forecast
• Hit-ratio not monotonously increasing with the
  information available

36
Figure 1
37
Figure 2
38
Results (contd)

• Figure 3
• Gain of using a survey-based model only until
  hard data arrive for the 1st month in the quarter

39
Figure 3
40
Results (contd)

• Figure 4
• Survey-based forecasts useful during the first
  half of current quarter
• Aferwards, better to incorporate incoming
  information from the hard data
• Significant jump in accuracy as soon as hard data
  become available
• Magnitude of errors very similar to the ones
  reported in recent OECD interim forecasts
  document

41
Figure 4
42
Summary ASEF

• Potent tool to select variables with high
  informational content (Stage 1 and 2)
• Convenient way to evaluate out-of-sample
  forecasts based on various informational
  assumptions (Stage 3)
• Fast computing and summary of forecasts for daily
  use (Stage 4)
• Can compute and combine the results of a large
  set of forecasting equations

43
Further potential developments

• Running a fully automated predictors subset
  selection (elimination and replacement
  algorithms)
• Combining model forecasts (mean,
  regression-based, Bayesian coaxing)
• Assessing risks associated with forecasts (hist.
  errors, fan-chart)