Regression as Moment Structure

1
Regression as Moment Structure
2
Regression Equation Y b X v
Observable Variables Y
z X Moment matrix sYY
sYX S sYX
sXX Moment structure S S(q) b2sXX
svv bsXX S
bsXX sXX Parameter vector q
(b, sXX, svv )
3

• Sample z1, z2, ..., zn n iid
• Sample Moments
• S n-1 S zi zi
• syy syx
• S
• syx sxx
• Fitting S to S S(q)
• Estimator q S close to S S(q)
• 3 moment equations
• syy b2sXX svv
• syx bsXX
• sxx sXX
• with 3 (unknown) parameters

b is the same as the usual OLS estimate of b
!


4
Regression Equation Y b x v
X x u Observable
Variables Y z X Moment
structure S S(q) b2sXX svv
bsXX S
bsXX sXX suu
Parameter vector q (b, sXX, svv , suu )
new parameter
5

• Sample z1, z2, ..., zn n iid
• Sample Moments
• S n-1 ? zi zi
• syy syx
• S
• syx sxx
• Fitting S to S S(q)
• Estimator q S close to S S(q )
• 3 moment equations
• syy b2sxx svv
• syx bsxx
• sxx sxx suu

b is the same as the usual OLS estimate of b
!

6
The effect of measurement error in regression
v
Y
b
x
X
u
Y b (X -u) v bX (v - bu) cX w,
where w v - bu Note that w is correlated
with X, unless u or b equals zero So, the
classical LS estimate b of b is neither ubiased,
neither consistent. In fact, b ---gt
sYX/sXX b (sxx/sXX ) kb k is the so called
Fiability coefficient (reliability of X). Since
0 ?k ? 1 b suffers from downward bias
7
In multiple regression
Regression Equation Y b1x1 b2x2... b
pxp v Xk xk
uk Observable Variables b SXX-1SXY
does not converge to b b (SXX - Quu)-1
SXY
Examples with EQS of regression with error in
variables Using suplementary information to
assessing the magnitude of variances of errors
in variables.
8
Path analysis covariance structure

• Example with ROS data

9
Sample covariance matrix
ROS92 ROS93 ROS94 ROS95 ROS92 72.07 ROS93 29.56
36.21 ROS94 30.21 31.09 46.51 ROS95 27.63 24.04 3
5.19 46.62 Mean 6.27 7.35 10.02 8.80 n
70
SEM bj ? It is a valid model ?
F
b1
b2
b3
ROS92 ROS93 ROS94
10
Calculations
b1b2 29.56 b1b3 30.21 b2b3 31.09 b1b2/b1b3
b2/b3 29.56/30.21--gt b2 .978b3 31.09
b2b3 b3 (.978b3) --gt b32 31.09/.978
b3 5.64 In the same way, we obtain
b15.34 b25.52 Model test in this case is
CHI2 0, df 0
11
Fitted Model
1
F
5.34
5.52
5.64
R92
R94
R93
43.34
5.80
14.74
CHI2 0, df 0
12
/TITLE FACTOR ANALYSIS MODEL (EXAMPLE
ROS) /SPECIFICATIONS CAS70 VAR4 /LABEL
V1ROS92 V2ROS93 V3ROS94 V4ROS95 /EQUATION
S V1 F1 E1 V2 F1 E2 V3
F1 E3 /VARIANCES F1 1.0 E1 TO E3
/COVARIANCES /MATRIX 72.07 29.56 36.21 30.21
31.09 46.51 27.63 24.04 35.19 46.62 /END
13
ROS92 V1 5.359F1 1.000 E1

.974
5.504

ROS93 V2 5.516F1 1.000 E2

.650
8.482

ROS94 V3 5.637F1 1.000 E3

.753
7.482 VARIANCES
OF INDEPENDENT VARIABLES -----------------------
----------- E
D ---
--- E1 -ROS92
43.347I I
8.205 I
I
5.283 I
I
I I
E2 -ROS93 5.789I
I
3.924 I I
1.475 I
I
I
I E3 -ROS94
14.736I I
4.693 I
I
3.140 I
I
I I
14
with the help of EQS
RESIDUAL COVARIANCE MATRIX (S-SIGMA)    
ROS92 ROS93 ROS94
V 1 V 2 V
3 ROS92 V 1 0.000 ROS93 V
2 0.000 0.000 ROS94 V 3
0.000 0.000 0.000 CHI-SQUARE
0.000 BASED ON 0 DEGREES OF
FREEDOM STANDARDIZED SOLUTION    ROS92 V1
.631F1 .776 E1
ROS93 V2 .917F1
.400 E2
ROS94 V3 .827F1 .563 E3

15
one - factor four- indicators model
F




R93
R95
R94
R92




CHI2 ?, df ? p-value ?
16
with the help of EQS
/TITLE FACTOR ANALYSIS MODEL (EXAMPLE ROS) !
This line is not read /SPECIFICATIONS CAS70
VAR4 /LABEL V1ROS92 V2ROS93 V3ROS94
V4ROS95 /EQUATIONS V1 F1 E1 V2
F1 E2 V3 F1 E3 V4 F1
E4 /VARIANCES F1 1.0 E1 TO E4
/COVARIANCES /MATRIX 72.07 29.56 36.21 30.21
31.09 46.51 27.63 24.04 35.19 46.62 /END
17
with the help of EQS
ROS92 V1 4.998F1 1.000 E1

.966
5.175
ROS93
V2 4.837F1 1.000 E2
.622

7.779
ROS94
V3 6.417F1 1.000 E3
.653

9.833
ROS95 V4
5.393F1 1.000 E4
.710

7.590
VARIANCES OF
INDEPENDENT VARIABLES --------------------------
-------- E
D ---
--- E1 -ROS92 47.090I
I
8.437 I
I
5.581 I I
I
I E2
-ROS93 12.810I
I
2.775 I I
4.616 I
I
I
I E3 -ROS94
5.332I I
3.017 I
I
1.767 I
I
I I
E4 -ROS95 17.536I
I
3.682 I I
4.763 I
I
18
RESIDUAL COVARIANCE MATRIX (S-SIGMA)    
RESIDUAL COVARIANCE MATRIX (S-SIGMA)    
ROS92 ROS93 ROS94
ROS95 V 1 V 2
V 3 V 4 ROS92 V 1
0.000 ROS93 V 2 5.383 0.000
ROS94 V 3 -1.862 0.049
0.000 ROS95 V 4 0.676 -2.048
0.583 0.000 CHI-SQUARE 6.271
BASED ON 2 DEGREES OF FREEDOM PROBABILITY
VALUE FOR THE CHI-SQUARE STATISTIC IS 0.04347
STANDARDIZED SOLUTION    ROS92 V1
.631F1 .776 E1
ROS93 V2 .917F1 .400
E2 ROS94
V3 .827F1 .563 E3

19
Fitted Model
F
4.84
6.42
5.40
4.99
R93
R95
R94
R92
17.54
5.33
12.81
47.10
CHI2 6.27, df 2 p-value .043
20
/TITLE FACTOR ANALYSIS MODEL (EXAMPLE
ROS) /SPECIFICATIONS CAS70 VAR4 /LABEL
V1ROS92 V2ROS93 V3ROS94 V4ROS95 /EQUATION
S V1 F1 E1 V2 F1 E2 V3
F1 E3 V4 F1 E4 /VARIANCES F1
1.0 E1 TO E4 /COVARIANCES /CONSTRAINTS

(V1,F1)(V2,F1)(V3,F1)(V4,F1)
/MATRIX 72.07 29.56 36.21 30.21 31.09
46.51 27.63 24.04 35.19 46.62 /END
21
estimation results
ROS92 V1 5.521F1 1.000 E1

.528
10.450

ROS93 V2 5.521F1 1.000 E2

.528
10.450

ROS94 V3 5.521F1 1.000 E3

.528
10.450

ROS95 V4 5.521F1 1.000 E4

.528
10.450

CHI-SQUARE 12.425 BASED ON 5 DEGREES OF
FREEDOM PROBABILITY VALUE FOR THE CHI-SQUARE
STATISTIC IS 0.02941
22
... EQS use an iterative optimization method
  ITERATIVE SUMMARY  
PARAMETER ITERATION
ABS CHANGE ALPHA FUNCTION
1 21.878996 1.00000
1.39447 2 5.741889
1.00000 0.43985 3
2.309283 1.00000
0.19638 4 0.477505
1.00000 0.18079 5
0.147232 1.00000 0.18014
6 0.056361 1.00000
0.18008 7 0.014530
1.00000 0.18007 8
0.005784 1.00000
0.18007 9 0.001423
1.00000 0.18007 10
0.000598 1.00000 0.18007
23
Exercise a) Write the covariance structure for
the one - factor four- indicators modelb)
From the ML estimates of this model, shown in
previous slides, compute the fitted covariance
matrix.c) In relation with b), compute
the residual covariance matrix
Note For c), use the following sample
moments ROS92 ROS93 ROS94 ROS95 ROS92 72.07 RO
S93 29.56 36.21 ROS94 30.21 31.09 46.51 ROS95 27.6
3 24.04 35.19 46.62 Mean 6.27 7.35 10.02
8.80 n 70
24
one - factor four- indicators model with means
F
1








R93
R95
R94
R92




CHI2 ?, df ? p-value ?
25
/TITLE FACTOR ANALYSIS MODEL (EXAMPLE ROS
data) /SPECIFICATIONS CAS70 VAR4 ANALYSIS
MOMENT /LABEL V1ROS92 V2ROS93
V3ROS94 V4ROS95 /EQUATIONS V1 V999
F1 E1 V2 V999 F1 E2 V3
V999 F1 E3 V4 V999 F1
E4 /VARIANCES F1 1.0 E1 TO E4
/COVARIANCES /CONSTRAINTS !
(V1,F1)(V2,F1)(V3,F1)(V4,F1) /MATRIX 72.07 29.
56 36.21 30.21 31.09 46.51 27.63 24.04 35.19
46.62 /MEANS 6.27 7.35 10.02 8.80 /END
26
ROS92 V1 6.270V999 4.998F1
1.000 E1
1.022 .966
6.135
5.175
ROS93 V2 7.350V999 4.837F1
1.000 E2
.724 .622
10.146
7.779
ROS94 V3 10.020V999 6.417F1
1.000 E3
.821 .653
12.204
9.833
ROS95 V4 8.800V999 5.393F1
1.000 E4
.822 .710
10.706
7.591
VARIANCES OF INDEPENDENT VARIABLES
----------------------------------
E D
--- --- E1
-ROS92 47.092I
I
8.437 I I
5.582 I
I
I
I E2 -ROS93
12.810I I
2.775 I
I
4.616 I
I
I I
E3 -ROS94 5.332I
I
3.017 I I
1.767 I
I
I
I E4 -ROS95
17.535I I
3.682 I
I
4.763 I
I