SEM I Lecture 3

1
WorkshopFactorial Invariance Revisted Are We
There Yet?Todd D. LittleUniversity of Kansas
2
Comparing Across Groups or Across Time

• In order to compare constructs across two or more
  groups OR across two or more time points, the
  equivalence of measurement must be established.
• This need is at the heart of the concept of
  Factorial Invariance.
• Factorial Invariance is assumed in any
  cross-group or cross-time comparison
• SEM is an ideal procedure to test this
  assumption.

3
Comparing Across Groups or Across Time

• Meredith provides the definitive rationale for
  the conditions under which invariance will hold
  (OR not)Selection Theorem
• Note, Pearson originated selection theorem at the
  turn of the century

4
Which posits if the selection process effects
only the true score variances of a set of
indicators, invariance will hold
5
Classical Measurement Theorem
Xi Ti Si ei Where, Xi is a persons
observed score on an item, Ti is the 'true' score
(i.e., what we hope to measure), Si is the
item-specific, yet reliable, component, and ei is
random error, or noise. Note that Si and ei are
assumed to be normally distributed (with mean of
zero) and uncorrelated with each other. And,
across all items in a domain, the Sis are
uncorrelated with each other, as are the eis.
6
Selection Theorem on Measurement Theorem
X1 T1 S1 e1 X2 T2 S2 e2 X3 T3 S3
e3
Selection Process
7
Levels Of Invariance

• There are four levels of invariance
• 1) Configural invariance - the pattern of fixed
  free parameters is the same.
• 2) Weak factorial invariance - the relative
  factor loadings are proportionally equal across
  groups.
• 3) Strong factorial invariance - the relative
  indicator means are proportionally equal across
  groups.
• 4) Strict factorial invariance - the indicator
  residuals are exactly equal across groups
• (this level is not recommended).

8
The Covariance Structures Model

• where...
• S matrix of model-implied indicator variances
  and covariances
• L matrix of factor loadings
• F matrix of latent variables / common factor
  variances and covariances
• Qd matrix of unique factor variances (i.e., S
  e and all covariances are usually 0)
• This model is fit to the data because it contains
  fewer parameters to estimate, yet contains
  everything we want to know.

9
The Mean Structures Model

• where...
• mx vector of model-implied indicator means
• tx vector of indicator intercepts
• L matrix of factor loadings
• a vector of factor means

10
Factorial Invariance

• An ideal method for investigating the degree of
  invariance characterizing an instrument is
  multiple-group (or multiple-occasion)
  confirmatory factor analysis or mean and
  covariance structures (MACS) models
• MACS models involve specifying the same factor
  model in multiple groups (occasions)
  simultaneously and sequentially imposing a series
  of cross-group (or occasion) constraints.

11
Some Equations
Configural invariance Same factor loading
pattern across groups, no constraints. Weak
(metric) invariance Factor loadings
proportionally equal across groups. Strong
(scalar) invariance Loadings intercepts
proportionally equal across groups. Strict
invariance Add unique variances to be exactly
equal across groups.
12
Models and Invariance

• It is useful to remember that all models are,
  strictly speaking, incorrect. Invariance models
  are no exception.
• "...invariance is a convenient fiction created to
  help insecure humans make sense out of a universe
  in which there may be no sense."
• (Horn, McArdle, Mason, 1983, p. 186).

13
Measured vs. Latent Variables

• Measured (Manifest) Variables
• Observable
• Directly Measurable
• A proxy for intended construct
• Latent Variables
• The construct of interest
• Invisible
• Must be inferred from measured variables
• Usually Causes the measured variables
  (cf. reflective indicators vs. formative
  indicators)
• What you wish you could measure directly

14
Manifest vs. Latent Variables

• Indicators are our worldly window into the
  latent space
• John R. Nesselroade

15
Manifest vs. Latent Variables
?11
?1
?11
?21
?31
X1
X3
X2
?22
?11
?33
16
Selection Theorem
Selection Influence
?11
?11
Group (Time) 1
Group (Time) 2
?11
?21
?31
?11
?21
?31
X1
X3
X2
X1
X3
X2
?22
?11
?33
?22
?11
?33
17
Estimating Latent Variables
?11
?11
?21
?31
To solve for the parameters of a latent
construct, it is necessary to set a scale (and
make sure the parameters are identified)
X1
X3
X2
?22
?11
?33
17
18
Scale Setting and Identification

• Three methods of scale-setting
• (part of identification process)
• Arbitrary metric methods
• Fix the latent variance at 1.0 latent mean at 0
• (reference-group method)
• Fix a loading at 1.0 an indicators intercept at
  0
• (marker-variable method)
• Non-Arbitrary metric method
• Constrain the average of loadings to be 1 and the
  average of intercepts at 0
• (effects-coding method Little, Slegers, Card,
  2006)

18
19

• Fix the Latent Variance to 1.0
• and Latent mean to 0.0)

1.0
?11
?21
?31
Three methods of setting scale 1) Fix latent
variance (?11)
X1
X3
X2
?22
?11
?33
19
20
2. Fix a Marker Variable to 1.0 (and its
intercept to 0.0)
?11
1.0
?21
?31
X1
X3
X2
?22
?11
?33
20
21
3. Constrain Loadings to Average 1.0 (and the
intercepts to average 0.0)
?11
?21
?31
?11 3-?21-?31
X1
X3
X2
?22
?33
?11
21
22
Configural invariance
xx
1
2
1
1
Group 1
.57
.61
.63
.63
.59
.60
1
6
5
4
3
2
.10
.12
.10
.11
.10
.07
xx
1
2
1
1
Group 2
.64
.66
.71
.59
.55
.57
1
6
5
4
3
2
.11
.09
.07
.11
.07
.06
23
Configural invariance
xx
1
2
1
1
Group 1
.57
.51
.63
.63
.59
.60
1
6
5
4
3
2
.10
.12
.10
.11
.10
.07
xx
1
2
1
1
Group 2
.64
.76
.71
.59
.55
.57
1
6
5
4
3
2
.11
.09
.07
.11
.07
.06
24
Configural invariance
-.07
1
2
1
1
Group 1
.57
.61
.63
.63
.59
.60
1
6
5
4
3
2
.10
.12
.10
.11
.10
.07
-.32
1
2
1
1
Group 2
.64
.64
.71
.56
.55
.57
1
6
5
4
3
2
.11
.09
.07
.11
.07
.06
25
Weak factorial invariance (equate ?s across
groups)
PS(2,1)
1
2
PS(1,1)
1
1
PS(2,2)
Group 1
LY(1,1)
LY(2,1)
LY(3,1)
LY(4,2)
LY(5,2)
LY(6,2)
1
6
5
4
3
2
TE(2,2)
TE(1,1)
TE(3,3)
TE(4,4)
TE(5,5)
TE(6,6)
Note Variances are now Freed in group 2
PS(2,1)
1
2
e
e
PS(1,1)
PS(2,2)
Group 2
LY(1,1)
LY(2,1)
LY(3,1)
LY(4,2)
LY(5,2)
LY(6,2)
1
6
5
4
3
2
TE(2,2)
TE(1,1)
TE(3,3)
TE(4,4)
TE(5,5)
TE(6,6)
26
F Test of Weak Factorial Invariance
(9.2.1.TwoGroup.Loadings.FactorID)
-.07
Positive
Negative
-.33
1.22
.85
.58
.59
.64
.62
.59
.61
Great Glad
Unhappy Bad
Down Blue
Terrible Sad
Happy Super
Cheerful Good
.10
.12
.11
.10
.07
.11
.07
.09
.10
.07
.06
.11
Model Fit ?2(20, n759)49.0 RMSEA.062(.040-.08
4) CFI.99 NNFI.99
27
M Test of Weak Factorial Invariance
(9.2.1.TwoGroup.Loadings.MarkerID)
-.03
Positive
Negative
-.12
.33
.39
.41
.33
1
1.02
1.11
1
.95
.97
Great Glad
Unhappy Bad
Down Blue
Terrible Sad
Happy Super
Cheerful Good
.10
.12
.11
.10
.07
.11
.07
.09
.10
.07
.06
.11
Model Fit ?2(20, n759)49.0 RMSEA.062(.040-.08
4) CFI.99 NNFI.99
28
EF Test of Weak Factorial Invariance
(9.2.1.TwoGroup.Loadings.EffectsID)
-.03
Positive
Negative
-.12
.36
.37
.44
.31
.98
1.06
1.00
.96
1.03
.97
Great Glad
Unhappy Bad
Down Blue
Terrible Sad
Happy Super
Cheerful Good
.10
.12
.11
.10
.07
.11
.07
.09
.10
.07
.06
.11
Model Fit ?2(20, n759)49.0 RMSEA.062(.040-.08
4) CFI.99 NNFI.99
29
Results Test of Weak Factorial Invariance

• The results of the two-group model with equality
  constraints on the corresponding loadings
  provides a test of proportional equivalence of
  the loadings

Nested significance test (?2(20, n759) 49.0)
- (?2(16, n759) 46.0) ??2(4, n759) 3.0,
p gt .50 The difference in ?2 is non-significant
and therefore the constraints are supported. The
loadings are invariant across the two age
groups. Reasonableness tests RMSEA weak
invariance .062(.040-.084) versus configural
.069(.046-.093) The two RMSEAs fall within one
anothers confidence intervals. CFI weak
invariance .99 versus configural .99 The
CFIs are virtually identical (one rule of thumb
is ?CFI lt .01 is acceptable).
(9.2.TwoGroup. Loadings)
30
Adding information about means

• When we regress indicators on to constructs we
  can also estimate the intercept of the indicator.
  
• This information can be used to estimate the
  Latent mean of a construct
• Equivalence of the loading intercepts across
  groups is, in fact, a critical criterion to pass
  in order to say that one has strong factorial
  invariance.

31
Adding information about means
1
2
1
1
AL(2)
AL(1)
1
6
5
4
3
2
TY(1)
TY(2)
TY(3)
TY(4)
TY(5)
TY(6)
X
32
Adding information about means
(9.3.0.TwoGroups.FreeMeans)
1
2
1
6
5
4
3
2
3.14
2.99
3.07
1.70
1.53
1.55
X
3.07
2.85
2.98
1.72
1.58
1.55
Model Fit ?2(20, n759) 49.0 (note that model
fit does not change)
33
Strong factorial invariance (aka. loading
invariance) Factor Identification Method
(9.3.1.TwoGroups.Intercepts.FactorID)
-.07
-.33
1
2
.58
.59
.64
.62
.59
.61
1
6
5
4
3
2
3.15
2.97
3.08
1.70
1.55
1.54
X
Model Fit ?2(24, n759) 58.4, RMSEA
.061(.041.081), NNFI .986, CFI .989
34
Strong factorial invariance (aka. loading
invariance) Marker Var. Identification Method
(9.3.1.TwoGroups.Intercepts.MarkerID)
-.03
-.12
1
2
1
1.03
1.11
1
.95
.97
1
6
5
4
3
2
0
-.28
-.43
0
-.06
-.12
X
Model Fit ?2(24, n759) 58.4, RMSEA
.061(.041.081), NNFI .986, CFI .989
35
Strong factorial invariance (aka. loading
invariance) Effects Identification Method
(9.3.1.TwoGroups.Intercepts.EffectsID)
-.03
-.12
1
2
.95
.98
1.06
1.03
.97
1.00
1
6
5
4
3
2
.23
-.05
-.18
.06
-.00
-.06
X
Model Fit ?2(24, n759) 58.4, RMSEA
.061(.041.081), NNFI .986, CFI .989
36
How Are the Means Reproduced?

• Indicator mean intercept loading(Latent Mean)
• i.e., Mean of Y intercept slope (X)
• For Positive Affect then
• Group 1 (7th grade) Group 2 (8th grade)
• Y t ? (a) Y t
  ? (a)
• 3.14 3.15 .58(0) 3.07 3.15
  .58(-.16) 3.06
• 2.99 2.97 .59(0) 2.85 2.97
  .59(-.16) 2.88
• 3.07 3.08 .64(0) 2.97 3.08 .64(-.16)
  2.98
• Note in the raw metric the observed difference
  would be -.10
• 3.14 vs. 3.07 -.07
• 2.99 vs. 2.85 -.14 gives an average of
  -.10 observed
• 3.07 vs. 2.97 -.10
• i.e. averaging 3.07 - 2.96 -.10

37
The complete model with means, stds, and rs
(9.7.1.Phantom variables.With Means.FactorID)
-.07
Positive 3
Negative 4
-.32
1.11 (in group 2)
.92 (in group 2)
1.0 (in group 1)
1.0 (in group 1)
Estimated only in group 2! Group 1 0
-.16 (z2.02)
.04 (z0.53)
.62
.59
.61
.64
.58
.59
3.15
2.97
3.08
1.70
1.54
1.54
Model Fit ?2(24, n759) 58.4, RMSEA
.061(.041.081), NNFI .986, CFI.989
38
The complete model with means, stds, and rs
(9.7.2.Phantom variables.With Means.MarkerID)
-.07
Positive 3
Negative 4
-.32
.57 (in group 2)
.64 (in group 2)
.58 (in group 1)
.62 (in group 1)
3.15 3.06
1.70 1.72
1
.95
1.11
.97
1
1.03
0
-.28
-.43
0
-.12
-.06
Model Fit ?2(24, n759) 58.4, RMSEA
.061(.041.081), NNFI .986, CFI.989
39
The complete model with means, stds, and rs
(9.7.3.Phantom variables.With Means.EffectsID)
-.07
Positive 3
Negative 4
-.32
.56 (in group 2)
.67 (in group 2)
.60 (in group 1)
.61 (in group 1)
3.07 2.97
1.59 1.62
1.03
.97
1.06
1.00
.96
.98
.23
-.05
-.18
.06
-.06
-.00
Model Fit ?2(24, n759) 58.4, RMSEA
.061(.041.081), NNFI .986, CFI.989
40
Effect size of latent mean differences

• Cohens d (M2 M1) / SDpooled
• where SDpooled v(n1Var1 n2Var2)/(n1n2)

41
Effect size of latent mean differences

• Cohens d (M2 M1) / SDpooled
• where SDpooled v(n1Var1 n2Var2)/(n1n2)
• Latent d (a2j a1j) / v?pooled
• where v?pooled v(n1 ?1jj n2
  ?2jj)/(n1n2)

42
Effect size of latent mean differences

• Cohens d (M2 M1) / SDpooled
• where SDpooled v(n1Var1 n2Var2)/(n1n2)
• Latent d (a2j a1j) / v?pooled
• where v?pooled v(n1 ?1jj n2
  ?2jj)/(n1n2)
• dpositive (-.16 0) / 1.05
• where v?pooled v(3801
  3791.22)/(380379)
• -.152

43
Comparing parameters across groups
1. Configural Invariance Inter-occular/model fit
Test
2. Invariance of Loadings RMSEA/CFI difference
Test
3. Invariance of Intercepts RMSEA/CFI difference
Test
4. Invariance of Variance/ Covariance Matrix ?2
difference test
5. Invariance of Variances ?2 difference test
6. Invariance of Correlations/Covariances ?2
difference test
3b or 7. Invariance of Latent Means ?2 difference
test
44
The Null Model

• The standard null model assumes that all
  covariances are zero only variances are
  estimated
• In longitudinal research, a more appropriate
  null model is to assume that the variances of
  each corresponding indicator are equal at each
  time point and their means (intercepts) are also
  equal at each time point (see Widaman
  Thompson).
• In multiple-group settings, a more appropriate
  null model is to assume that the variances of
  each corresponding indicator are equal across
  groups and their means are also equal across
  groups.

44
45
Thanks for Listening!
46
References
Byrne, B. M., Shavelson, R. J., Muthén, B.
(1989). Testing for the equivalence of factor
covariance and mean structures The issue of
partial measurement invariance. Psychological
Bulletin, 105, 456-466. Cheung, G. W.,
Rensvold, R. B. (1999). Testing factorial
invariance across groups A reconceptualization
and proposed new method. Journal of Management,
25, 1-27. Gonzalez, R., Griffin, D. (2001).
Testing parameters in structural equation
modeling Every one matters. Psychological
Methods, 6, 258-269. Kaiser, H. F., Dickman,
K. (1962). Sample and population score matrices
and sample correlation matrices from an arbitrary
population correlation matrix. Psychometrika, 27,
179-182. Kaplan, D. (1989). Power of the
likelihood ratio test in multiple group
confirmatory factor analysis under partial
measurement invariance. Educational and
Psychological Measurement, 49, 579-586. Little,
T. D., Slegers, D. W., Card, N. A. (2006). A
non-arbitrary method of identifying and scaling
latent variables in SEM and MACS models.
Structural Equation Modeling, 13,
59-72. MacCallum, R. C., Roznowski, M.,
Necowitz, L. B. (1992). Model modification in
covariance structure analysis The problem of
capitalization on chance. Psychological Bulletin,
111, 490-504. Meredith, W. (1993). Measurement
invariance, factor analysis and factorial
invariance. Psychometrika, 58, 525-543. Steenkamp
, J.-B. E. M., Baumgartner, H. (1998).
Assessing measurement invariance in
cross-national consumer research. Journal of
Consumer Research, 25, 78-90.
46