Program Evaluation 2

1
Program Evaluation (2)
Feng ,Jin Fudan University
2
Method of Instrumental Variables
3
The Problem

• Whether an individual participates in a program
  is usually determined at least in part by
  unobserved variables that also affect outcomes of
  interest.
• These unobserved variables affect the outcome and
  also are correlated with whether an individual
  participates in the program.

4
Selection Bias

• Participants and non-participants outcomes
  differ even if the program is ineffective.
• If we can describe the selection process, we can
  control for pre-existing differences in
  outcomes between the two groups.
• Selection on observables

5
Selection Bias

• Unobserved attributes likely lead to differences
  between the two groups.
• Selection on unobservables

6
Solution

• Consider the participation equation
• Di ?0 ? 1Zi ui
• The variable (vector) Zi is an instrumental
  variable

7
Instrumental Variable

• Find some variable(s) Zi that is
• (1) Correlated with whether an individual
  participates in the program.
• (2) Uncorrelated with the unobserved variables in
  the outcome equation.
• More formally, we express these ideas as follows
• E(Di, Zi ) is not equal to zero.
• E(Zi, ?i) 0

8
Example Workforce Development Policy

• Some individuals receive training and others do
  not.
• Consider the variable indicating whether a person
  is randomly offered the opportunity to
  participate in a government training program.
• The random offer of training is an instrument
• Offer is highly correlated with receipt of
  training.
• Offer is uncorrelated with the determinants of
  earnings and other outcomes of interest.

9
Example The Effects of Military Service

• For different U.S. generations military service
  is associate with either higher or lower lifetime
  earnings.
• Problem A persons veteran status is correlated
  with other determinants of earnings.
• Instrument Randomly drawn draft lottery
  number.
• Why is this variable a good instrumental variable?

10
Wald estimates

• Regress Di ?0 ? 1Zi ui
• Compute Di or the predicted value of Di
• Regress Yi b0 ?1Di ?i

Wald estimator is an important and
easily-analyzed IV estimator
11
Why wald estimator works

• Consider the results of the first stage
  regression
• Predicted participation, Di, is given by
• Di ?0 ?1Zi.
• The residual is given by
• ui Di - ? 0 - ? 1Zi Di - Di.
• We now can express the participation variable in
  terms of the correlated and uncorrelated
  parts.

12
Wald estimates

• We want to use only movements in the
  participation variable that are uncorrelated with
  the unobserved determinants of the outcomes of
  interest.
• Express the participation variable as
• Di Di ui.
• Regress the outcome on the predicted
  participation
• Yi b0 ?1 Di (? 1ui ei ).

13

• The spirit of OLS is to compare outcomes of Y for
  high D vs low D
• The spirit of IV is to compare outcomes of Y for
  high Z vs low Z
• The regression of outcome on the instruments Z is
  called the reduced form
• The reason for a causal interpretation is

14
(No Transcript)
15
(No Transcript)
16
Angrist (1990)

• The Wald estimator is based solely on earnings
  differences by draft-eligibility status
• A more efficient estimator exploits all the
  information using observations on mean earnings
  for each group of five consecutive numbers

17
More efficient estimator
18
2SLS

• First stage
• To run regression, to obtain the fitted value of
  D, which is D
• Second stage
• OLS regression on D and X

19
(No Transcript)
20
IV example

• Levitt (1997) what is the effect of increasing
  the police force
• on the crime rate?
• This is a classic case of simultaneous causality
  (high crime areas
• tend to need large police forces) resulting in an
  incorrectly-
• signed (positive) coefficient
• To address this problem, Levitt uses the timing
  of mayoral and
• gubernatorial elections as an instrumental
  variable
• Is this instrument valid?
• Relevance police force increases in election
  years
• Exogeneity election cycles are pre-determined

21
IV example

• Two-stage least squares
• Stage 1 Decompose police hires into the
  component that can
• be predicted by the electoral cycle and the
  problematic
• component
• policei ?0 ?1 electioni ?i
• Stage 2 Use the predicted value of policei from
  the first-stage
• regression to estimate its effect on crimei
• crimei ?0 ?1 police-hati ?i
• Finding an increased police force reduces
  violent crime
• (but has little effect on property crime)

22
IV number of instruments

• There must be at least as many instruments as
  endogenous
• regressors
• Let k number of endogenous regressors
• m number of instruments
• The regression coefficients are
• exactly identified if mk (OK)
• overidentified if mgtk (OK)
• underidentified if mltk (not OK)

23
Angrist Krueger, 1991Does compulsory schooling
attendance affect schooling and earnings
24
Angrist Krueger, 1991
25
Angrist Krueger, 1991
26
(No Transcript)
27
IV testing instrument relevance

• How do we know if our instruments are valid?
• Recall our first condition for a valid
  instrument
• 1. Relevance corr (Zi , Di) ? 0
• Stock and Watsons rule of thumb the first-stage
  F-statistic
• testing the hypothesis that the coefficients on
  the instruments
• are jointly zero should be at least 10 (for a
  single endogenous
• regressor)
• A small F-statistic means the instruments are
  weak (they
• explain little of the variation in D) and the
  estimator is biased

28
IV testing instrument exogeneity

• Recall our second condition for a valid
  instrument
• 2. Exogeneity corr (Zi , ?i) 0
• If you have the same number of instruments and
  endogenous
• regressors, it is impossible to test for
  instrument exogeneity
• But if you have more instruments than regressors
• Overidentifying restrictions test regress the
  residuals from
• the 2SLS regression on the instruments (and any
  exogenous
• control variables) and test whether the
  coefficients on the
• instruments are all zero

29
Over-identification test

• When there are more instruments more than the
  number of endogenous variables, there are two
  tests

30
IV pitfalls

• the validity of instruments
• The possibility that ?i and Zi are correlated
• Weak IV
• a. IVs are said to be weak when the endogenous
  regressor that depends on the IVs is small
• b. Measured by the first stage R2 and F
  statistics
• c. IVs can be weak and F-statistic small either
  because coefficients of IV are close to zero or
  variability of IVs is low

31
Example of weak IVs

• Angust and Krueger (1991)

32
The effect of weak IVs

• The bias is more than OLS
• The effect of weak IV depends considerably on the
  degree of endogeneity and the concentration
  parameter
• Other things equal, more instruments, smaller
  samples, and weaker instruments each mean more
  bias

33
Average Treatment EffectLocal average treatment
effect

• Average Treatment Effect (ATE)
• E(Y1-Y0 X)
• Treatment on Treated (TT)
• E(Y1-Y0 X, D1)
• Treatment on un-Treated (TUT)
• E(Y1-Y0 X, D0)
• Local Average Treatment Effect (LATE)
• E(Y1-Y0 X, D (z) 1, D (z)0)

Y1 is value of Y (outcome--say earnings) for
treated (e.g. job training) Y0 is value of Y
(outcome) for those who do not receive
treatment D1 if person receives treatment D0 if
person does not receive treatment