Generative vs. Discriminative Models

1
Generative vs. Discriminative Models

• Generative approach separately model
  class-conditional densities and priorsthen
  evaluate posterior probabilities using Bayes
  theoremDiscriminative approach directly model
  posterior probabilities(C. Bishop 04)

2
Generative vs. Discriminative
3
Generative Methods

• ? Relatively straightforward to characterize
  invariances
• ? They can handle partially labelled data
• ? They wastefully model variability which is
  unimportant for classification
• ? They scale badly with the number of classes and
  the number of invariant transformations (slow on
  test data)

(C. Bishop 04)
4
Discriminative Methods

• ? They use the flexibility of the model in
  relevant regions of input space
• ? They can be very fast once trained
• ? They interpolate between training examples, and
  hence can fail if novel inputs are presented
• ? They dont easily handle compositionality (e.g.
  faces can have glasses and/or moutaches and/or
  hats)

(C. Bishop 04)