Outline

1
Outline

• Classification

2
Pattern Recognition

• It is natural and desirable that we should seek
  to design and build machines that can recognize
  patterns
• Automated speech recognition
• Fingerprint identification
• Optical character recognition
• DNA sequence identification

3
An Example

• Fish sorting
• A fish-packing plant wants to automate the
  process of sorting incoming fishes on a conveyor
  belt according to species using optical sensing
• Separate sea bass from salmon
• Physical differences between sea bass and salmon
• Length, lightness, width, number and shape of
  fins, position of mouth .....
• Noise

4
Components of A Recognition System

• A Typical pattern recognition system
• Input
• Sensing
• Segmentation
• Feature extraction
• Classification
• Post-processing

5
Sensing

• The input to the recognition system
• Digital cameras
• Lasers
• Some kind of a transducer
• Characteristics and limitations of the transducer
• Its bandwidth
• Resolution
• Sensitivity
• Distortion
• Signal-to-noise ratio

6
Segmentation and Grouping

• Segmentation
• To segment out the object we are interested in
  from all other objects
• This is a very difficult problem
• Grouping
• Group pixels that correspond to an object
  together
• Perceptual organization
• Figure-ground segregation

7
Feature Extraction

• Features
• Some characteristics of the input that can
  separate objects in different types very
  effectively
• Invariant features
• Translation invariance
• Rotation invariance
• Scale invariance
• Occlusion
• Projective distortion

8
Feature Extraction cont.

• Deformation
• Domain specific highly complex transformations
• Feature extraction is domain specific
• That is, good features depend on what you want to
  do
• Feature selection
• Techniques to select the best features among a
  set of features

9
Training

• Design or training samples
• One needs to make measurements of each pattern
  class
• This is often done by specifying examples

10
Classification

• Definition
• Given a set of classes, represented by the
  corresponding feature values, assign the new
  input object to a category
• The degree of the difficulty depends on the
  variability in the feature values for objects in
  the same object with respect to the difference
  between feature values for objects in different
  categories

11
Post-Processing

• Post-processing makes recommendations or takes
  actions based on the output from the classifier
• Error rate
• Risk
• Cost of a mistake
• Context
• Multiple classifiers

12
Design Cycles

• Data collection
• Feature choice
• Prior knowledge
• Model choice
• Training
• Evaluation
• Over-fitting

13
Computational Complexity

• Pattern recognition problems can be solved
  using algorithms that are highly impractical
• Polynomial vs. exponential
• The computational resources needed and
  computational complexity are of practical
  importance
• The system may have to make a decision within a
  time interval

14
Learning and Adaptation

• Supervised learning
• There is a teacher which provides a category
  label for each pattern in a training set
• Unsupervised learning
• There is no explicit teacher
• The system forms clusters of the input patterns
• Reinforcement learning
• Some feedback information about the systems
  performance

15
Neural Networks

• Based on the connections in the brain

16
Neural Networks cont.
17
Statistical Pattern Recognition

• Given a set of features and cost associated with
  each decision, classification is to decide a
  decision boundary in the feature space or make a
  decision rule
• We want to minimize the total cost
• Generalization
• The classifier is designed to suggest actions for
  novel patterns

18
Pattern Theory

• Pattern theory proposed by Ulf Grenander
• The analysis of the patterns generated by the
  world in any modality, with all their naturally
  occurring complexity and ambiguity, with the goal
  of reconstructing the processes, objects and
  events that produced them and of predicting these
  patterns when they reoccur

19
Bayesian Decision Rule

• A two-class example
• ?1 for sea bass
• ?2 for salmon
• Prior probability
• P(?1)
• P(?2)

20
Bayesian Decision Rule cont.

• Class conditional probability density
• P(?1 x)
• P(?2 x)
• Bayes formula

21
Bayesian Decision Rule cont.

• Bayes decision rule
• Decide ?1 if P(?1 x) gt P(?2 x)
• Otherwise decide ?2
• The optimal decision rule
• Minimize the average error we make