Multicollinearity

1
Multicollinearity

• Definition (wikipedia)
• -refers to any linear relationship amongst
  explanatory variables in a regression model. It
  can affect two or more of them. The original
  definition referred to an exact linear
  relationship, but later it was extended to mean a
  nearly perfect relationship. The correlation can
  be negative or positive.

2
Multicollinearity

• Estimation of regression model is an attempt to
  isolate the influence of each exogenous variable
  on the value of the endogenous variable
• Given that we as economists cannot control our
  experimental design, explanatory variables may be
  to some degree be related to one another
• Estimation of consumption function where
  consumption is determined by income and wealth
• Cross-section ? wealth and income usually
  strongly positively related
• Wealth and Income are said to be collinear

3
Multicollinearity
Klein-Goldberger Data
Cons a ßWWage ßPPension ßFFarm_Inc e
4
Multicollinearity

• CRM parameter estimates
• Note the effects of non-wage and farm income are
  not statistically significant although from
  aggregate perspective they should be.
• Very high R2 value and F-stat

5
Multicollinearity

• If there is perfect collinearity then X is said
  to be linear dependent.
• e.g.
• Xcx1c1x2c2xKcK 0 where ci are constants
  and not all are zero
• Example
• If this is the case we cannot take the inverse of
  XX
• If near collinearity we then have
• Xc x1c1x2c2xKcK 0
• Overall effect of collinearity the higher the
  collinearity, the greater the variance, and
  hence, less chance of being found significant.
  (Greene pg 56-57 talks about this and shows it
  mathematically)

6
Multicollinearity

• Symptoms
• -High R2 but low t-stat on some or all
  coefficients.
• -Unreasonable parameter estimates or unexpected
  signs.
• -Estimates are very sensitive to addition or
  deletion of a few observations, or the deletion
  of what appears to be a statistically
  insignificant variable.
• -Significant joint test but insignificant t-stat
  for individual coefficients
• -High correlation between two or more of the
  regressions

7
Multicollinearity

• Diagnosis
• -Examine pair wise correlation
• Rule of Thumb if simple correlation coefficient
  between 2 regressors is greater than 0.8 or 0.9
  then multicollinearity is a serious problem
• In Klein-Goldberger example correlations between
  wages and pensions and wages and farm income are
  0.72 and 0.92, respectively
• -Use F-stat to test for joint significance

8
Multicollinearity

• -estimate auxiliary regression, i.e.
• Want low R2, if is close to 1 then have high
  collinearity.
• Using the above R2 we can calculate the variation
  inflation factor (VIF)
• Want VIF close to 1 (i.e just look to see if R2
  is close to 0).
• If VIF gt 30 then semi collinearity.

9
Multicollinearity

• What to do if present
• If non severe, nothing
• If severe
• Do nothing (spurious correlation)
• Obtain more data
• Transform data of express the variables as ratios
  (this may cause problems if the original model
  satisfies assumptions of CRM -autocorrelation)
• Incorporate estimates from other studies
• Ridge regression (pg 878 Judge et al.)
• Overall there is usually no clear answer on how
  to solve this problem

10
Questions

• Questions about coding?
• Questions about Homework?