Factor Analysis

1
Factor Analysis

• Dr. Milne

2
Correlation Matrix Pattern
V1 V2 V3 V4 V5 V6 V1 1 0 0 0 V2 1 0 0 0 V
3 1 0 0 0 V4 0 0 0 1 V5 0 0 0 1 V6 0 0
0 1
3
Factor Analysis
Factor 1 Factor2 Variable 1 .9 .2 Variable
2 .8 .3 Variable 3 .6 .1 Variable
4 .2 .7 Variable 5 .1 .8 Variable 6 .2 .9
4
Basic Model
X1 w11 F1 w12 F2 e1 X2 w21 F1
w22 F2 e2 X3 w31 F1 w32 F2 e3 X4
w41 F1 w42 F2 e4 X5 w51 F1 w52
F2 e5 X6 w61 F1 w62 F2 e6 X w
F E
5
Factor Analysis Objectives

• Identify the structure of relationship among
  variables or respondents
• Identify representative variables from a much
  larger set of variables
• Create an entirely new set of variables, much
  smaller in number, to partially or completely
  replace the original set of variables for
  inclusion is subsequent analyses.

6
Analysis Process
Research Problem Which variables? How many?
How measured? Sample size?
Type of Correlation Matrix? Q or R
Factor Model
Components
Common Factor Method
Extraction Method
Number of Factors Retained?
Rotation, Interpretation?
Try again?
Factor Scores?
7
Keep in Mind
Factor Analysis will produce factors..
Factor Analysis
Garbage
Garbage
8
Designing a Factor Analysis

• Variable Selection and Measurement Issues
• Metric, or 0/1 allowed.
• Include about 5 variables for each factor
• Sample Size
• gt50, over 100 preferred, 5 times as many
  observations as variables.
• Correlations Among Variables or Respondents
• Q factor vs. R factor analysis

9
Q versus R?
Variables
X1 X2 Xn
1 . . . N
Q factor groups observations
Observations
R Factor groups variables
10
Q versus Cluster?
Score
A
B
C
D
V1 V2 V3 V4
Variable
11
Determining Appropriateness

• Assumptions
• Multivariate normal distribution
• Sufficient interdependence among measurement

• Tests for Interdependence
• Measure of sample adequacy (MSA) - .80 upward
  (Affected by sample size, correlation among
  items, number of variables and smaller number of
  factors)
• Bartletts Test of Sphericity - produces a
  significance level, p. (Large sample size
  inflates statistic)

• In practice
• Start by looking at corr. matrix.
• High correlations go ahead.Multivariate normal
  distribution
• Low corrlations look at other tests

12
Assumptions

• Some underlying structure exists in the set of
  selected variables
• Visual inspection of .3 correlation among at
  least some variables
• Bartlett test of Sphericity
• Measure of sampling adequacy (MSA)
• Normality is needed to test significance of the
  factors.
• Deviations from normality, homoscedacity, and
  linearity may affect correlations among
  variables.
• shared variance extends across the entire sample.

13
Deriving Factors and Assessing Overall Fit

• Common Factor vs. Component Analysis

X w F e
F w X
h12 r12 r1p r21 h22 rn1

hp2
1 r12 r1p r21 1 rn1
1
14
Calculating Eigenvalues redistributes the
variance in the data matrix
x2
F1
F2
x1
15
Types of Variance
Diagonal Value
Variance
Unity
Total Variance
Communality
Common
Specific and error
Variance Extracted
Variance Lost
16
Criteria for the Number of Factors to Be Extracted

• Latent Root Criterion
• A Priori Criterion
• Percentage of Variance Extracted
• Scree Test Criterion

17
(No Transcript)
18
Interpreting the Factors

• First compute unrotated factor matrix to
  determine number of factors to extract.
• Loading Strength ( .40 strong loading)
• Rotate factors to achieve simpler and more
  interpretable factors
• Respecify model and run again as needed

19
Guidelines for identifying significant factor
loadings based on sample size

• Factor Loading
• .30
• .35
• .40
• .45
• .50
• .55
• .60

• Sample size for sig.
• 350
• 250
• 200
• 150
• 120
• 100
• 85

20
Rotating Factors

• Correlation matrix can be reproduced with more
  than one set of factor loadings.
• Goal is simple structure
• Maximize the loading of each item on their
  principle factor and minimize the loadings on the
  other factors.

• Orthogonal rotation
• Used most
• Changes the basis vectors (and loadings) so that
  factors are uncorrelated (orthogonal).
• Not very realistic

• Oblique rotation
• Does not force factors to be uncorrelated
• Resulting structure is more realistic.
• Gives better loadings in many cases.

21
Rotate Factors
1.0
Factor II rotated
Factor II unrotated
-1.0
1.0
Factor I unrotated
Factor I, rotated
-1.0
22
Types of Rotations

• Orthogonal
• Quartimax
• Varimax (Most Preferred)
• Equimax
• Oblique
• Oblimin (SPSS)

23
Interpreting a Factor Matrix

• Examine the Factor matrix of Loadings
• Identify the highest Loading for Each Variable
• Assess Communalities of the Variables
• Label the Factors

24
Varimax rotated component analysis factor matrix
Variables Factor 1
Factor 2 Communality
X1 Delivery Speed -.781 .194 .66 X2 Price
Level .724 .266 .58 X3 Price Flexibility -.804
-.011 .65 X4 Manufacturers image .102 .933 .88
X6 Sales forces image .025 .934 .87 X7 Product
Quality .764 .179 .62 Sum of squares
(eigenvalue) 2.38 1.87 4.25 Percentage of
trace 39.7 31.2 70.9
25
Validation of Factor Analysis

• Use split sample
• Use Confirmatory Factor Analysis (lisrel)

Factor 1
x1
e1
e2
x2
e3
x3
e4
x4
Factor 2
x5
e5
x6
e6
26
Other Uses of Factor Analysis Results

• Selecting Surrogate Variables for Subsequent
  Analysis
• Use Factor Scorescomposite measures for each
  factor representing each subject. Uses
  information from all variables.
• Summated Scoresa score created by summing
  variable values for only variables that load on a
  factor.

27
Creating New Variables

• Factor Scores
• Based on the factor loadings
• Include information from all items in analysis

• Composites
• Average of raw scores all items weighted equally
  (introduces bias)
• Can multiple raw score by factor loading (Similar
  to factor score but eliminates influence from
  other items found in factor scores).

• Reliability
• Look at internal consistency with Cronbachs
  alpha
• Look at construct reliability (based on the
  factor loadings)

28
SPSS --Correlation
Examine the correlation matrix. Look for
significant correlations over .30.
29
Bartlett Test of Sphericity and Measure of
Sampling Adequacy
30
Number of Factors
31
Interpret Loadings
The art of naming factors ..
32
Evaluating the Solution
33
Varimax rotated component analysis factor matrix
Variables Factor 1
Factor 2 Communality
X1 Delivery Speed -.781 .194 .66 X2 Price
Level .724 .266 .58 X3 Price Flexibility -.804
-.011 .65 X4 Manufacturers image .102 .933 .88
X6 Sales forces image .025 .934 .87 X7 Product
Quality .764 .179 .62 Sum of squares
(eigenvalue) 2.38 1.87 4.25 Percentage of
trace 39.7 31.2 70.9
34
Lab Assignment

• Using SPSS
• With Hatco Data Set Factor analyze V1-V7.
  Objective is to go through the steps presented
  and arrive at results shown in class.
• With consumer sentiment data set, select
  variables X1 - X25 and run a factor analysis.
• If you want more fun, you can work with New
  Zealand Survey data.