The Pyramid Match Kernel: Discriminative Classification with Sets of Image Features

1
The Pyramid Match Kernel Discriminative
Classification with Sets of Image Features

• Kristen Grauman
• Trevor Darrell
• MIT

2
Sets of features
3
Sets of features
4
Problem

• How to build a discriminative classifier using
  the set representation?

• Kernel-based methods (e.g. SVM) are appealing for
  efficiency and generalization power
• But what is an appropriate kernel?
• Each instance is unordered set of vectors
• Varying number of vectors per instance

5
Existing set kernels

• Fit (parametric) model to each set, compare with
  distance over models
• Kondor Jebara, Moreno et al., Lafferty
  Lebanon, Cuturi Vert,
• Wolf Shashua

Restrictive assumptions
Ignoring set statistics
6
Partial matching for sets of features

• Compare sets by computing a partial matching
  between their features.

Robust to clutter, segmentation errors, occlusion
7
Pyramid match
8
Pyramid match overview
Pyramid match kernel measures similarity of a
partial matching between two sets

• Place multi-dimensional, multi-resolution grid
  over point sets
• Consider points matched at finest resolution
  where they fall into same grid cell
• Approximate similarity between matched points
  with worst case similarity at given level

No explicit search for matches!
9
Pyramid match kernel
Approximate partial match similarity
10
Feature extraction
11
Counting matches
Histogram intersection
12
Counting new matches
Histogram intersection
13
Pyramid match kernel

• Weights inversely proportional to bin size
• Normalize kernel values to avoid favoring large
  sets

14
Efficiency

• For sets with m features of dimension d, and
  pyramids with L levels, computational complexity
  of
• Pyramid match kernel
• Existing set kernel approaches
• or

15
Example pyramid match
Level 0
16
Example pyramid match
Level 1
17
Example pyramid match
Level 2
18
Example pyramid match
pyramid match
optimal match
19
(No Transcript)
20
Building a classifier

• Train SVM by computing kernel values between all
  labeled training examples
• Classify novel examples by computing kernel
  values against support vectors
• One-versus-all for multi-class classification

Convergence is guaranteed since pyramid match
kernel is positive-definite.
21
Object recognition results

• ETH-80 database 8 object classes
• Features
• Harris detector
• PCA-SIFT descriptor, d10

Eichhorn and Chapelle 2004
22
Object recognition results

• Caltech objects database 101 object classes
• Features
• SIFT detector
• PCA-SIFT descriptor, d10
• 30 training images / class
• 43 recognition rate
• (1 chance performance)
• 0.002 seconds per match

23
Localization

• Inspect intersections to obtain correspondences
  between features
• Higher confidence correspondences at finer
  resolution levels

observation
target
24
Pyramid match regression

• Pose estimation from contour features
• Train SVR with CG data
• Features shape context
  histograms

25
Summary Pyramid match kernel
optimal partial matching between sets of features
number of new matches at level i
difficulty of a match at level i
26
Summary Pyramid match kernel

• A new similarity measure based on implicit
  correspondences that approximates the optimal
  partial matching
• linear time complexity
• no independence assumption
• model-free
• insensitive to clutter
• positive-definite function
• fast, effective object recognition

27
Future work

• Geometric constraints
• Fast search of large databases with the pyramid
  match for image retrieval
• Use as a filter for a slower, explicit
  correspondence method
• Alternative feature types and classification
  domains