The StateoftheScience on Factorial Invariance

1
The State-of-the-Science on Factorial Invariance

• Todd D. Little, University of Kansas
• Factorial Invariance What It Is and Why It Is So
  Damn Important
• Roger Milsap, Arizona State University
• Prediction Using Measures That Violate Factorial
  Invariance What Happens?
• Nilam Ram, Penn State University
• Time For Change? Redefining and Reconsidering
  the Role of Factorial Invariance in Psychological
  Inquiry
• Keith F. Widaman, University of California at
  Davis
• Discussant

2
Factorial Invariance What It Is and Why It Is So
Damn Important

• Todd D. Little
• Director, Quantitative Training Program
• University of Kansas
• www.Quant.KU.edu

3
Comparing Across Groups or Across Time

• In order to compare constructs across two or more
  groups OR across two or more time points, the
  equivalence of measurement must be established.
• This need is at the heart of the concept of
  Factorial Invariance.
• Factorial Invariance is assumed in any
  cross-group or cross-time comparison
• SEM is an ideal procedure to test this
  assumption.
• Without evidence of Factorial Invariance,
  conclusions are dubious at best or wrong at
  worst.

4
Comparing Across Groups or Across Time

• Meredith provides the definitive rationale for
  the conditions under which invariance will hold
  (OR not)Selection Theorem
• Note, Pearson originated selection theorem at the
  turn of the century

5
Which posits if the selection process effects
only the true score variances of a set of
indicators, invariance will hold
6
Classical Measurement Theorem
Xi Ti Si ei Where, Xi is a persons
observed score on an item, Ti is the 'true' score
(i.e., what we hope to measure), Si is the
item-specific, yet reliable, component, and ei is
random error, or noise. Note that Si and ei are
assumed to be normally distributed (with mean of
zero) and uncorrelated with each other. And,
across all items in a domain, the Sis are
uncorrelated with each other, as are the eis.
7
Selection Theorem on Measurement Theorem
X1 T1 S1 e1 X2 T2 S2 e2 X3 T3 S3
e3
Selection Process
8
Factorial Invariance

• An ideal method for investigating the degree of
  invariance characterizing an instrument is
  multiple-group (or multiple-occasion)
  confirmatory factor analysis or mean and
  covariance structures (MACS) models
• MACS models involve specifying the same factor
  model in multiple groups (occasions)
  simultaneously and sequentially imposing a series
  of cross-group (or occasion) constraints.

9
The Covariance Structures Model

• where...
• S matrix of model-implied indicator variances
  and covariances
• L matrix of factor loadings
• F matrix of latent variables / common factor
  variances and covariances
• Qd matrix of unique factor variances (i.e., S
  e and all covariances are usually 0)
• This model is fit to the data because it contains
  fewer parameters to estimate, yet contains
  everything we want to know.

10
The Mean Structures Model

• where...
• mx vector of model-implied indicator means
• tx vector of indicator intercepts
• L matrix of factor loadings
• a vector of factor means

11
Levels Of Invariance

• There are four levels of invariance
• 1) Configural invariance - the pattern of fixed
  free parameters is the same.
• 2) Weak factorial invariance - the relative
  factor loadings are proportionally equal across
  groups.
• 3) Strong factorial invariance - the relative
  indicator means are proportionally equal across
  groups.
• 4) Strict factorial invariance - the indicator
  residuals are exactly equal across groups
• (this level is not recommended).

12
Defining Equations for Factorial Invariance
Configural invariance Same factor loading
pattern across groups, no constraints. Weak
(metric) invariance Factor loadings
proportionally equal across groups. Strong
(scalar) invariance Loadings intercepts
proportionally equal across groups. Strict
invariance Add unique variances to be exactly
equal across groups.
13
Comparing parameters across groups
1. Configural Invariance Inter-occular/model fit
Test
2. Invariance of Loadings RMSEA/CFI difference
Test
3. Invariance of Intercepts RMSEA/CFI difference
Test
4. Invariance of Variance/ Covariance Matrix ?2
difference test
5. Invariance of Variances ?2 difference test
6. Invariance of Correlations/Covariances ?2
difference test
3b or 7. Invariance of Latent Means ?2 difference
test
14
Got Factorial Invariance?
15
Effect size of latent mean differences

• Cohens d (M2 M1) / SDpooled
• where SDpooled v(n1Var1 n2Var2)/(n1n2)
• Latent d (a2j a1j) / v?pooled
• where v?pooled v(n1 ?1jj n2
  ?2jj)/(n1n2)
• dpositive (-.16 0) / 1.05
• where v?pooled v(3801
  3791.22)/(380379)
• -.152