Regularized Generalized Structured Component Analysis

1
Regularized Generalized Structured Component
Analysis 

• Heungsun Hwang
• Department of Psychology
• McGill University

2
Generalized Structured Component Analysis (Hwang
Takane, 2004)

• GSCA is a component-based approach to structural
  equation modeling (SEM).
• Defines latent variables as components.
• Combines measurement and structural models into a
  single equation.
• provides a global optimization criterion.

3
Extensions of GSCA

• GSCA has been extended to improve data-analytic
  flexibility and generality. For example,
• Fuzzy clusterwise GSCA (Hwang et al., 2007)
• Multilevel GSCA (Hwang et al., 2007)
• Nonlinear GSCA (Hwang Takane, 2008)
• GSCA with latent interactions (Hwang et al.,
  2009)
• Regularized GSCA (Hwang, 2009)

4
(www.sem-gesca.org)
5
The GSCA Model Submodels
Structural/inner model
?
e3
z3
c1
c3
z1
e1
b
?1
?2
z4
e2
e4
z2
c4
c2
Measurement/outer model
6
The GSCA Model Submodels
In GSCA, latent variables are defined as
components or weighted sum of observed variables.
?
e3
w1
w3
z3
c1
c3
z1
e1
b
?1
?2
z4
e2
e4
z2
c4
w4
c2
w2
?1 z1w1 z2w2
?2 z3w3 z4w4
7
GSCA A Few Technical Points

• Measurement model

• Structural model

• Weighted relation

8
GSCA A Few Technical Points
9
GSCA A Few Technical Points

• The unknown parameters of GSCA (W, C B) are
  estimated such that the sum of squares of the
  residuals (ei) is as small as possible.
• This is equivalent to minimizing the following
  least-squares criterion

10
GSCA A Few Technical Points

• GSCA provides overall goodness of fit measures
• FIT 1 - SS(ZV ZWA )/SS(ZV)
• AFIT 1 (1 - FIT)(NJ)/(NJ P)

11
Multicollinearity in GSCA

• Multicollinearity generally represents high
  correlations among exogenous variables.
• results in inaccurate parameter estimates and
  large standard errors, leading to inference
  errors.

12
Multicollinearity in GSCA

• Two sources of multicollinearity
• High correlations among latent exogenous
  variables
• High correlations among observed exogenous
  variables for a single latent variable
• Formative indicators

13
High correlations among latent exogenous variables
Z3
Z2
Z11
Z12
Z1
LV1
LV4
Z13
Z4
LV6
Z14
LV2
Z5
Z15
LV5
Z6
LV3
Z7
Z8
Z9
Z10
14
High correlations among observed exogenous
variables
z1
z9
z2
z10
z3
z11
LV1
LV2
z4
z12
z5
z13
z6
z14
z7
z15
z8
15
Regularized GSCA

• Regularized GSCA is proposed to deal with
  potential multicollinearity.
• incorporates ridge-type regularization into GSCA.

16
GSCA A Few Technical Points

• Measurement model

• Structural model

• Weighted relation

17
Regularized GSCA

• Regularized GSCA seeks for minimizing the
  following regularized least-squares optimization
  criterion
• subject to diag(WZZW) I, where ?1 , ?2 and
  ?3 denote the prescribed, non-negative ridge
  parameters.

18
Regularized GSCA

• An alternating regularized least squares (ARLS)
  algorithm is developed to minimize the
  optimization criterion.
• This algorithm repeats three main steps, given
  the values of ?1 , ?2 and ?3.

19
The ARLS Algorithm

• Step 1 C is updated for fixed W and B.
• Let , where and
  .
• Then, the criterion can be re-expressed as

20
The ARLS Algorithm

• Minimization of this criterion with respect to C
  is equivalent to minimizing

21
The ARLS Algorithm

• Step 2 B is updated for fixed W and C.

22
The ARLS Algorithm

• Step 3 W is updated for fixed A (B C).

23
Regularized GSCA

• K-fold cross-validation is utilized to select the
  values of ?1 , ?2 and ?3.
• K 5 or 10 (e.g., Hastie, Tibshirani, Friedman,
  2001, p. 214)

24
Example The ACSI data

• The present example was company-level data from
  the American Customer Satisfaction Index (ACSI)
  (Fornell et al., 1996) database collected in
  2002.
• The sample size was of 152 companies in total.

25
The ACSI Model (Fornell et al., 1996)
Z12
Z3
Z2
Z1
CC
CE
-
Z13

-

Z4

CL
Z14


CS
PQ
Z5
Z15


Z6
PV
Z9
Z10
Z11
Z7
Z8
26
Example Non-regularized GSCA
Z12
FIT .95 AFIT .95
Z3
Z2
Z1
1
.97
.94
.96
CC
CE
-.00
-.44
-.46
.91
Z13
.98
Z4
-.05
.97
.49
CL
.75
CS
PQ
.98
Z14
Z5
.98
.27
.98
.98
.95
.87
.97
Z6
PV
Z9
Z10
Z11
.98
.99
Z7
Z8
27
Example Regularized GSCA (?1 0, ?2 0, ?3
0.1 )
Z12
FIT .95 AFIT .95
Z3
Z2
Z1
1
.97
.94
.96
CC
CE
.17
-.42
-.41
.83
Z13
.98
Z4
.18
.97
.46
CL
.50
CS
PQ
.98
Z14
Z5
.98
.31
.98
.98
.95
.59
.97
Z6
PV
Z9
Z10
Z11
.98
.99
Z7
Z8