Multiple Discriminant Analysis

1
Multiple Discriminant Analysis

• Dr. Milne

2
What is discriminant analysis?

• The appropriate statistical technique when the
  dependent variable is categorical and the
  independent variables are metric
• Two or more (multiple) groupshence MDA
• Mathematically it is the reverse of MANOVA.

3
Groups based on Purchase Intention Durability
(x1) Performance (x2) Style (x3) Group 1 Would
purchase Subject 1 8 9 6 Subject
2 6 7 5 Subject 3 10 6 3 Subject
4 9 4 4 Subject 5 4 8 2 Mean 7.4 6.8
4.0 Group 2 Would Not Purchase Subject
6 5 4 7 Subject 7 3 7 2 Subject
8 4 5 5 Subject 9 2 4 3 Subject
10 2 2 3 Mean 3.2 4.4 3.8 Difference 4.
2 2.4 0.2
4
10 8 9 7 5 6
2 1 4 3
X1 Durability x2 Performance x3 Style
1 2 3 4 5 6 7
8 9 10
9
6 7
10 4 8 3 2 5
1
1 2 3 4 5 6 7
8 9 10
10 7 9 8
5 3 4 2 1 6
1 2 3 4 5 6 7
8 9 10
5

F1 zx1 F2 z x1 x2
F3z-4.53.476x1.359x2 Group 1 Would
purchase Subject 1 8 17 2.51 Subject
2 6 13 0.84 Subject 3 10 16 2.38 Subject
4 9 13 1.19 Subject 5 4 12 .25 Group
2 Would Not Purchase Subject 6 5 9 -.71 Subj
ect 7 3 10 -.59 Subject 8 4 9 -.83 Subje
ct 9 2 6 -2.14 Subject 10 2 4 -2.86 Cutt
ing Score 5.5 11 0.0
6
Classification Accuracy
Function 1 zx1 Predicted Group Actual
Group 1 2 1 Would Purchase 4 1 2 Would not
Purchase 0 5 Function 2 zx1x2 1 Would
Purchase 5 0 2 Would not Purchase 0 5 Function
3 z-4.53. 1 Would Purchase 5 0 2 Would not
Purchase 0 5
7
x1-Price x2- Service x1-Price x2-Service Group
1 Switchers Group 2 Undecided Subject
1 2 2 Subject 6 4 2 Subject 2 1 2 Subject
7 4 3 Subject 3 3 2 Subject 8 5 1 Subject
4 2 1 Subject 9 5 2 Subject 5 2 3 Subject
10 5 3 Group mean 2.0 2.0 Group
Mean 4.6 2.2 Group 3 No Switch Subject
11 2 6 Subject 12 3 6 Subject 13 4 6 Subject
14 5 6 Subject 15 5 7 Group mean 3.8 6.2
8
15
11 14 5
13 10 4 12
7 9 2 1 3 6 8
x1 x2
1 2 3 4 5 6 7

9 6
14 3 10
13 8 2 7 12 4
1 5 11 15
1 2 3 4 5 6 7

9
Discriminant Function 2 z 0x1 1.0x2
7 6 5 4 3 2 1
15 11
12 13 14 5 7
10 2 1 3 6 9 4
8
Discriminant Function 1 z1.0x1 0x2
1 2 3 4 5 6 7

10
Cutting Score ZCE
Group B
Group A
Decrease S (variance)
ZA
ZB
d2
Classify as B (Purchaser)
Classify as A (Nonpurchaser)

• Discrimination is more effective as
• the means distance between means increases
• the variance decreases for two distributions

11
Discriminant Analysis relies on finding linear
composite
x2
Bivariate Density Distributions
x1
x2
x1 x2
Could represent as linear composites
x1
This composite is the best due to the less
overlap.
x1-x2
12
Discriminant Analysis Decision Process
Research Problem
Interpretation of the Discriminant Function
Research Design
Assumptions
Evaluation of Single Function
Evaluation of Separate Functions
Estimation of Discriminant Functions
Evaluation of Combined Functions
Assess Predictive Accuracy with Classification
Matrices
Validation of Discriminant Results
13
Discriminant Function
Z W1X1 W2X2 W3X3 . WiXi
Z Discriminant Score Wi Discriminant weight
for variable i Xi Independent variable i
14
Objectives of Discriminant Analysis

• Inference
• Dimension reduction
• Prediction
• Interpretation

15

• INFERENCE
• Determine whether statistically significant
  differences exist between the average score
  profiles on a set of variables for two (or more)
  a priori defined groups.
• DIMENSION REDUCTION
• Determining which of the independent variables
  account for the most for the differences in the
  average score profiles of the two or more groups.
• PREDICTION
• Establishing procedures for classifying
  statistical units into groups on the basis of
  their scores on a set of independent variable
• INTERPRETATION
• Establishing the number and composition of the
  dimensions of discrimination between groups
  formed from the set of independent variables.

16
Research Design

• Selection of Variables
• Groups must be mutually exclusive and exhaustive
• Artificial groups?, polar extremes?
• Independent variables picked based on theory and
  intuition
• Sample Size
• 20 observations per predictor variable
• Each group should at least have 20 observations
• Division of the Sample
• Analysis and holdout groups (60/40 or 75/25)

17
Assumptions of Discriminant Analysis

• Multivariate normality of the independent
  variables and unknown (but equal) dispersion and
  covariance structure (matrices) for groups.
• Linearity among relationships
• Watch for multicollinearity among independent
  variables during stepwise regressions.

18
Estimation and Assessing Fit

• Computational Method
• Simultaneous versus stepwise
• Statistical Significance of Functions
• Wilks lamda, Hotellings trace, Pilliais
  criterion. Mahalanobis D2 and Raos V for
  stepwise.
• Assessing Overall Fit
• Calculate Discriminant Z-scores
• Evaluate Group Differences
• Classification Matrices
• Cutting Scores
• Specifying probabilities of classification
• Measures of predictive accuracy
• Statistically-based measures of classification
  accuracy relative to chance.

19
Classification or Confusion Matrices
Predicted
1
2
1
n11
n12
Actual
2
n22
n21
classified correctly ( n11 n22 )/ total n
This works if probability is 50/50 but not if
group sizes differ. If group 1 had 70 and group
2 had 30 then percent classified correctly would
have to be greater than 58 to beat chance since
.72 .32 .58
20
Cutting Score Determination For equal groups ZCE
( ZA ZB ) / 2 Where ZCE critical cutting
score for equal groups, ZA is centroid for group
A, ZB is centroid for group B. For unequal
groups ZCU (NBZA NAZB) /( NA NB) Where ZCU
critical cutting score for unequal groups, ZA is
centroid for group A, ZB is centroid for group B,
NA is number in group A, NB is number in group B
ZCE
ZA ZB
Optimally weighted cutting score
Unweighted cutting score
ZA ZB
21
Hit ratios and classification accuracy
predicted 1 predicted 2 Actual Percent Actual
1 22 3 25 88 Acutal 2 5 20 25 80 Predicted
total 27 23 50
P - .5 .5 (1.0 - .5) N
t
Where P proportion correctly classified, N
sample size
22
Proportional Chance Criterion p2 (1-p)2
Where p is proportion from group 1 and (1-p) is
proportion from group 2. Statistically-Based
measure of Classification Accuracy Relative to
Chance N (nK)2 Presss Q
N (K 1) Where N
is Total sample size, nNumber of observations
correctly classified, and Knumber of groups.
23
Interpretation of Results

• Discriminant Weights
• Discriminant Loadings
• Partial F Values
• Interpretation of Two or More Functions
• Rotation of Discriminant Functions
• Potency index
• Graphical Display of Group Centroids
• Grapical Display of Discriminant Loadings

24
Potency Index

• A relative measure among all variables that is
  indicative of each variables discriminating
  power.

25
Potency Index a relative measure among all
variables that is indicative of each variables
discriminating power

• Calculate potency value for each significant
  function
• Relative eigenvalue for function a Eigenvalue
  for function a
• 
  Sum of Eigenvalues for all functions
• Potency value for variable b (Discriminant
  loading)2 x Relative eigenvalue for function
• 2. Calculate potency index across all functions
• Potency index for variable b sum of potency
  values for all significant functions.

26
Validation of Results

• Split sample or Cross-Validation Procedures
• Profiling Group Differences
• Variables used within the analysis
• New variables

27
(No Transcript)
28
Graphing Loadings and Centroids in Discriminant
Space
Discriminant Function 2
Group 2 Centroid
Group 3 Centroid
x1
x2
Discriminant Function 1
x3
Group 1 Centroid
29
SPSS

• Classify Discriminant Analysis
• Grouping Variate
• Independents
• enter together or use step wise
• Statistics
• mean, ANOVAs Box M, Matrices,
• function coefficients (select unstandardized)
• Classify
• All groups equal / compute from groups
• Display
• casewise results, summary table, leave one-out
  classification

30
(No Transcript)
31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
Multiple Groups
37
Assignment

• 2 Group (Specification Buying/Total Value
  Analysis) by
• delivery speed, price level, price flexibility,
  manufacturer image, overall service, salesforce
  image, product quality.
• 3 Group (Buying situation X14) by same DVs.
• Factor scores of Consumer Sentiment predicting
  Males vs. Females.