G89.2247 Lecture 4

1
G89.2247Lecture 4

• Panel Designs
• Panel Path Analyses
• Examples
• Lords Paradox
• Extensions

2
A Panel Design

• In many cases we have only two time points, a
  baseline and a followup
• Between the two time points some treatment,
  growth or other change is assumed to happen
• The two time points allow certain causal features
  of the relationships to be examined
• Temporal ordering of assumed cause to effect
• Establishing the contiguity of cause to effect
• Ruling out alternative causes, such as selection
  effects

3
A Panel Analysis

• Path/Regression methods are often proposed for
  panel analyses

Y2 b0 b1Y1 b2X e2
4
Features of the Panel Path Analysis

• The effect of X, b2, is a semi-partial effect
  that takes into account the level Y1
• "Holding constant Y1, what is the effect of X?"
• b2 is sometimes called regressed change
• Restate Y2b0b1Y1b2Xe, as
• Y2 - b1Y1 b0b2Xe2
• When b1 is close to 1, then simple change and
  regressed change analyses give similar answers

5
An example

• Suppose we want to show that the bar exam group
  has a different change in anxiety than the
  comparison group, using time 1 and time 4
  data.
• First we regress Anxiety(week4) on Group and
  Anxiety(week1)

6
Example, continued

• We proceed with a two step approach, first with
  no adjustment for week1 baseline, and then with
  an adjustment. Sample 0 Examinees, 1 Comparison

• When week 1 is adjusted, the group difference
  increases. The initial tendency for the
  comparison group to be more anxious is
  statistically adjusted.

7
In Path Terms
X1
e2
-.772
r.28
Y1
Y2
.796
8
Example, continued

• As Rogosa and others note, the results vary if we
  adjust for a different baseline

• The one week lag produces an effect which is less
  than a half the effect of the three week lag

9
Interpreting Statistical Adjustment for Baseline

• The panel analysis allows X1 and Y1 to be
  correlated
• If X1 is a treatment, this correlation may
  reflect some selection effect
• Including Y1 in the equation greatly strengthens
  the apparent causal claims regarding X1 gtY2
  compared to a simple cross-sectional analysis.
• The statistical adjustment for Y1 may not be
  enough to eliminate selection effects.
• If Y1 is measured with error, it will tend to
  underadjust
• If Y1 does not perfectly describe the selection
  bias, the analysis also will underadjust.
• E.g. Cohen's "premature covariate"

10
Regression Estimates

• If X and the Y measures are standardized, then
  the estimate for b2 is
• If, however, Y1, is measured with error, such
  that its reliability is R1lt1, then

11
A numerical example of underadjustment
X
Y1
Y2
X
1
Y1
0.6
1
Y2
0.5
0.7
1
Standardized coefficient for effect of X on Y
b2
0.125
Assuming Measurement error for Y1
R
0.5
b2star
0.354
12
Unreliability is not the only basis for
underadjustment

• Consider "Lord's Paradox" (Lord, 1967)
• Weight change in two dorms, Sept-May
• Is there an effect of food service?

13
The two perspectives

• The person who analyses raw change finds no
  difference On the average the two dorms have no
  weight gain
• The person who uses regressed change finds that a
  dorm effect
• Holding constant September weight, persons from
  one dorm are likely to be heavier than persons in
  the other dorm

14
The Paradox Explained

• The regressed change analysis focuses on May
  Weight holding constant September weight
• Suppose we found that women were more likely to
  be in Dorm B and men in Dorm A
• When we compare a man and a woman who are the
  same weight in September, we expect the man to
  gain weight, and the woman to lose weight.
• Even though it is reliable and valid, September
  weight is not a perfect proxy for selection
  effects

15
Combining Raw Change and Regression

• Suppose we defined D Y2 Y1 to be raw change
• D will be negatively correlated with level of Y1
• Higher scores on Y1 are more likely to go down
  and lower scores are more likely to go up
• Consider adjusting for Y1
• D a0 a1Y1 a2X e(Y2-Y1) a0 a1Y1 a2X
  e
• This implies
• Y2 a0 (1a1)Y1 a2X e

16
Return to Example

• Let D41difference between Anx 4 and Anx 1
• The group difference in this case is larger
  before adjustment, but the unstandardized
  adjusted results are exactly what we had before.

17
The Closer Time Point

• Now consider the anxiety difference between
  Times 4 and 3
• In this case the raw effect is diminished by
  adjustment.
• Note that while the unstandardized coefficient
  does not change, the standardized value does
  change.

18
Extensions of Path Approach