Li FeiFei, Princeton

1
CVPR 2007 Minneapolis, Short Course, June 17
Recognizing and Learning Object Categories Year
2007

• Li Fei-Fei, Princeton
• Rob Fergus, MIT
• Antonio Torralba, MIT

2
Agenda

• Introduction
• Bag-of-words models
• Part-based models
• Discriminative methods
• Segmentation and recognition
• Datasets Conclusions

3
(No Transcript)
4
(No Transcript)
5
Plato said

• Ordinary objects are classified together if they
  participate' in the same abstract Form, such as
  the Form of a Human or the Form of Quartz.
• Forms are proper subjects of philosophical
  investigation, for they have the highest degree
  of reality.
• Ordinary objects, such as humans, trees, and
  stones, have a lower degree of reality than the
  Forms.
• Fictions, shadows, and the like have a still
  lower degree of reality than ordinary objects and
  so are not proper subjects of philosophical
  enquiry.

6
Bruegel, 1564
7
How many object categories are there?
10,000 to 30,000
Biederman 1987
8
So what does object recognition involve?
9
Verification is that a lamp?
10
Detection are there people?
11
Identification is that Potala Palace?
12
Object categorization
mountain
tree
building
banner
street lamp
vendor
people
13
Scene and context categorization

• outdoor
• city

14
Computational photography
15
Assisted driving
Pedestrian and car detection
Lane detection

• Collision warning systems with adaptive cruise
  control,
• Lane departure warning systems,
• Rear object detection systems,

16
Improving online search
Query STREET
Organizing photo collections
17
Challenges 1 view point variation
Michelangelo 1475-1564
18
Challenges 2 illumination
slide credit S. Ullman
19
Challenges 3 occlusion
Magritte, 1957
20
Challenges 4 scale
21
Challenges 5 deformation
Xu, Beihong 1943
22
Challenges 6 background clutter
Klimt, 1913
23
History single object recognition
24
History single object recognition

• Lowe, et al. 1999, 2003
• Mahamud and Herbert, 2000
• Ferrari, Tuytelaars, and Van Gool, 2004
• Rothganger, Lazebnik, and Ponce, 2004
• Moreels and Perona, 2005

25
Challenges 7 intra-class variation
26
History early object categorization
27

• Turk and Pentland, 1991
• Belhumeur, Hespanha, Kriegman, 1997
• Schneiderman Kanade 2004
• Viola and Jones, 2000

• Amit and Geman, 1999
• LeCun et al. 1998
• Belongie and Malik, 2002

• Schneiderman Kanade, 2004
• Argawal and Roth, 2002
• Poggio et al. 1993

28
10,000 to 30,000
29
Object categorization the statistical viewpoint
30
Object categorization the statistical viewpoint
posterior ratio
likelihood ratio
prior ratio

• Discriminative methods model posterior

• Generative methods model likelihood and prior

31
Discriminative

• Direct modeling of

Decisionboundary
Zebra
Non-zebra
32
Generative

• Model and

33
Three main issues

• Representation
• How to represent an object category
• Learning
• How to form the classifier, given training data
• Recognition
• How the classifier is to be used on novel data

34
Representation

• Generative / discriminative / hybrid

35
Representation

• Generative / discriminative / hybrid
• Appearance only or location and appearance

36
Representation

• Generative / discriminative / hybrid
• Appearance only or location and appearance
• Invariances
• View point
• Illumination
• Occlusion
• Scale
• Deformation
• Clutter
• etc.

37
Representation

• Generative / discriminative / hybrid
• Appearance only or location and appearance
• invariances
• Part-based or global w/sub-window

38
Representation

• Generative / discriminative / hybrid
• Appearance only or location and appearance
• invariances
• Parts or global w/sub-window
• Use set of features or each pixel in image

39
Learning

• Unclear how to model categories, so we learn what
  distinguishes them rather than manually specify
  the difference -- hence current interest in
  machine learning

40
Learning

• Unclear how to model categories, so we learn what
  distinguishes them rather than manually specify
  the difference -- hence current interest in
  machine learning)
• Methods of training generative vs. discriminative

41
Learning

• Unclear how to model categories, so we learn what
  distinguishes them rather than manually specify
  the difference -- hence current interest in
  machine learning)
• What are you maximizing? Likelihood (Gen.) or
  performances on train/validation set (Disc.)
• Level of supervision
• Manual segmentation bounding box image labels
  noisy labels

Contains a motorbike
42
Learning

• Unclear how to model categories, so we learn what
  distinguishes them rather than manually specify
  the difference -- hence current interest in
  machine learning)
• What are you maximizing? Likelihood (Gen.) or
  performances on train/validation set (Disc.)
• Level of supervision
• Manual segmentation bounding box image labels
  noisy labels
• Batch/incremental (on category and image level
  user-feedback )

43
Learning

• Unclear how to model categories, so we learn what
  distinguishes them rather than manually specify
  the difference -- hence current interest in
  machine learning)
• What are you maximizing? Likelihood (Gen.) or
  performances on train/validation set (Disc.)
• Level of supervision
• Manual segmentation bounding box image labels
  noisy labels
• Batch/incremental (on category and image level
  user-feedback )
• Training images
• Issue of overfitting
• Negative images for discriminative methods Priors

44
Learning

• Unclear how to model categories, so we learn what
  distinguishes them rather than manually specify
  the difference -- hence current interest in
  machine learning)
• What are you maximizing? Likelihood (Gen.) or
  performances on train/validation set (Disc.)
• Level of supervision
• Manual segmentation bounding box image labels
  noisy labels
• Batch/incremental (on category and image level
  user-feedback )
• Training images
• Issue of overfitting
• Negative images for discriminative methods
• Priors

45
Recognition

• Scale / orientation range to search over
• Speed
• Context

46
Hoiem, Efros, Herbert, 2006
47
OBJECTS
INANIMATE
ANIMALS
PLANTS
MAN-MADE
NATURAL
..
VERTEBRATE
MAMMALS
BIRDS
GROUSE
BOAR
TAPIR
CAMERA