Classification Discriminant Analysis - PowerPoint PPT Presentation

1 / 18
About This Presentation
Title:

Classification Discriminant Analysis

Description:

Title: Steven F. Ashby Center for Applied Scientific Computing Month DD, 1997 Author: Computations Last modified by: James Jeffry Howbert Created Date – PowerPoint PPT presentation

Number of Views:56
Avg rating:3.0/5.0
Slides: 19
Provided by: Comput73
Category:

less

Transcript and Presenter's Notes

Title: Classification Discriminant Analysis


1
ClassificationDiscriminant Analysis
slides thanks to Greg Shakhnarovich (CS195-5,
Brown Univ., 2006)
2
(No Transcript)
3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
  • We want to minimize overlap between projections
    of the two classes.
  • One approach make the class projections a)
    compact, b) far apart.
  • An obvious idea maximize separation between the
    projected means.

7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
Example of applying Fishers LDA
maximize separation of means
maximize Fishers LDA criterion ? better class
separation
12
Using LDA for classification in one dimension
  • Fishers LDA gives an optimal choice of w, the
    vector for projection down to one dimension.
  • For classification, we still need to select a
    threshold to compare projected values to. Two
    possibilities
  • No explicit probabilistic assumptions. Find
    threshold which minimizes empirical
    classification error.
  • Make assumptions about data distributions of the
    classes, and derive theoretically optimal
    decision boundary.
  • Usual choice for class distributions is
    multivariate Gaussian.
  • We also will need a bit of decision theory.

13
Decision theory
  • To minimize classification error

At a given point x in feature space, choose as
the predicted class the class that has the
greatest probability at x.
14
Decision theory
At a given point x in feature space, choose as
the predicted class the class that has the
greatest probability at x.
probability densities for classes C1 and C2
relative probabilities for classes C1 and C2
15
MATLAB interlude
  • Classification via discriminant analysis,using
    the classify() function.
  • Data for each class modeled as multivariate
    Gaussian.
  • matlab_demo_06.m
  • class classify( sample, training, group, type
    )

test data
predicted test labels
training labels
model for classcovariances
training data
16
MATLAB classify() function
  • Models for class covariances

linearall classes have same covariance
matrix? linear decision boundary
diaglinearall classes have same diagonal
covariance matrix? linear decision boundary
quadraticclasses have different covariance
matrices? quadratic decision boundary
diagquadraticclasses have different diagonal
covariance matrices? quadratic decision boundary
17
MATLAB classify() function
  • Example with quadratic model of class
    covariances

18
Relative class probabilities for LDA
linearall classes have same covariance
matrix? linear decision boundary
relative class probabilities have exactly same
sigmoidal form as in logistic regression
Write a Comment
User Comments (0)
About PowerShow.com