Pictorial Structures for Object Recognition

1
Pictorial Structures for Object Recognition

2
Introduction

• Object recognition is distinguishing between
  various objects in an image.
• Appearance based approaches represent an object
  by attributes like view, shape or color.
• Find general rules for representing objects since
  they consist of several different parts.

3
Background - Pictorial Structures

• Introduced in 1973 by Fischler Elschlager.
• Split object up into separate parts, model each
  part then connect parts to form object.
• Felzenszwalb and Huttenlocher extended this idea.
  

4
Contributions Felzenschwab Huttenlocher

• Provide an efficient algorithm for solving energy
  minimization problem between acyclic parts.
• Introduce a method for learning these models
  from training examples.
• Develop a technique for finding multiple
  hypothesis for the location of an object.

5
Pictorial Structures

• Simultaneous use of appearance and spatial
  information to represent an object.

6
Definition of model

• Set of parts V v1,vn
• Location of parts L l1,ln
• Appearance parameters A (a1,,an)
• Edge eij, (vi,vj) E
• Connection parameters C cij eij E

7
Energy function for matching

• Given an Image
• mi(lj) Degree of mismatch when vi is
  placed at li.
• Dij(li,lj) Degree of deformation when vi is at
  li and vj is at lj.
• Optimal Location

8
Efficency

• Graph G be acyclic
• Requiring that the relationships between
  connected pairs of parts be expressed in a
  particular form

9
Statistical Formulation

• The energy minimization problem is equivalent to
  finding the maximum a posteriori estimate of the
  object configuration given an observed image
• The statistical formulation can be used to learn
  the parameters of a model from examples.

10
Related work

• M.C. Burl, M. Weber, and P. Perona.
• A probabilistic approach to object recognition
  using local photometry and global geometry

11
Statistical Framework

• Posterior
• Various configurations L given image I and model
• Likelihood
• Seeing image I given configuration and model
• Prior
• Obtaining configuration L given just the model

12
Statistical Framework

• The product of the individual likelihoods
• The prior distribution over object configurations
  
• The posterior

13
Estimating the Appearance model and dependencies
14
Matching algorithm
15
Example Face model

• A vector is created for each part corresponding
  to the its response to a gaussian filter at
  different orientations and scales.
• Pairs of parts constrained in terms of their
  relative position in image.
• Two models
• 5 parts eyes, tip of nose, corners of mouth
• 9 parts corners and center of eyes other parts
  above.

16
Face Model

• Appearance and structure learned from labeled
  frontal views.
• Structure captures pairs with most predictable
  location.
• Gaussian (covariance) model captures direction of
  spatial variations for each part

17
Results
18
Appearance Model for Human
Observing image I given configurations and
appearance model

• Pixels inside a part are foreground pixels with
  probability , and it is close to 1.
• Pixels in area 2 belong to the foreground with
  probability
• We assume that pixels outside both area are
  equally likely to be background or foregroud
  pixels (t total number of pixels in the image)

19
Appearance Model for Human

• Appearance parameters
• This model is reasonable for a
  single part, but for a configuration with
  overlapping parts, this measure over-counts
  evidence. ? sampling
• Over-count tends to create high peaks ?
  smoothing

20
Spatial model for Human

• The location of the joint is specified by two
  points. (in the coordinate
  frame of each part)
• The ideal configuration
• (x,y) is the center of the rectangle, s is the
  amount of foreshortening deciding length of
  rectangle, and ? is the orientation

21
Spatial model for Human

• The joint probability for the two locations is
  based on the deviation between their ideal
  relative values and the observed ones
• Where

22
Spatial model for Human

• Which allow us to write to the right form

23
Summary

• Efficient algorithm for finding the best global
  match of a large class of pictorial structure
  models to an image
• Statistical sampling techniques to identify
  multiple good matches of a model to an image
• Statistical formulation provides a natural way of
  learning pictorial structure models from labeled
  example images