Unsupervised%20Learning%20for%20Recognition

1
Unsupervised Learning for Recognition

• Pietro Perona
• California Institute of Technology
• Universita di Padova
• 11th British Machine Vision Conference
  Manchester, September 2001

2
Representation and Learning for Visual Object
Recognition

• Pietro Perona
• California Institute of Technology
• Università di Padova
• First SIAM-EMS Conference Berlin, 6 Sept. 2001

3
Representation and Learning for Visual Object
Recognition

• Pietro Perona
• California Institute of Technology
• Università di Padova
• University of Plymouth, 10 Sept. 2001

4
(No Transcript)
5
OBJECTS
INANIMATE
ANIMALS
PLANTS
MAN-MADE
NATURAL
VERTEBRATE
..
MAMMALS
BIRDS
GROUSE
BOAR
TAPIR
CAMERA
6
S. Thorpe et al. Nature 1996 J. Braun et al. J.
Neurosci. 1998 Fei Fei Li et al. Unpublished
animal
not animal
7
(No Transcript)
8
Issues

• Representation
• Recognition
• Learning

9
Meet the xyz
10
Spot the xyz
11
Meet the Boletus Edulis
12
Object categories
individual objects
functional categories
visual categories

13
Variability within a category
Deformation
Intrinsic
14
Part similarity
15
Importance of mutual position
16
SVD
17
SVD (2)
18
Model constellation of Parts
Tanaka et al., 1993

• Fischler Elschlager, 1973

• Yuille, 91
• Brunelli Poggio, 93
• Lades, v.d. Malsburg et al. 93
• Cootes, Lanitis, Taylor et al. 95
• Amit Geman, 95, 99
• Perona et al. 95, 96, 98, 00

Perrett Oram, 1993
19
Deformations
C
20
Presence / Absence of Features
occlusion
21
Background clutter
22
Generative probabilistic model
Model (Parameters)
Object shape pdf
Detector specification and prob. of detection
Clutter pdf
Prob. of N detect.
Pdf of location
0.9
pPoisson(N1?1)
0.8
pPoisson(N2?2)
0.6
p(x)A-1 (uniform)
pPoisson(N3?3)
e.g. p(x)G(x? ,? )
Example
1. Object Part Positions
3a. N false detect
2. Part Absence
3b. Position f. detect
N1
N2
N3
23
Affine Shape

• Translation, rotation and scaling Euclidean
  Shape
• Add weak perspective projection Affine Shape
• What is the probability density for the affine
  shape variables?

Feature space
Euclidean shape
Affine shape
24
Affine Shape DensityLeung, Burl Perona 98

• Gaussian figure space density
• Affine Shape density

(1) exact if N is odd (2) good approximation if
probability that bases points flip sign is low.
good
Careful!
25
Example Affine Shape Densities
Shape density (ground truth)
Shape density (approximation)
Model points
26
Generative probabilistic model
Model (Parameters)
Foregrond pdf
Prob. of Detection
Background pdf
0.8
0.9
Prob. of N detect.
Pdf of location
pPoisson(N1?1)
pPoisson(N2?2)
0.9
p(x)A-1 (uniform)
pPoisson(N3?3)
e.g. p(x)G(x? ,? )
Example
1. Object Part Positions
3a. N false detect
2. Part Absence
3b. Position f. detect
N1
N2
N3
27
Detection by likelihood ratio
P(object data) vs. P(clutter data)




















From Burl et al. ICCV95, CVPR96
28
Learning Models Manually

• Obtain set of training images

29
Unsupervised learning
30
Unsupervised detector training - 1

• Highly textured neighborhoods are selected
  automatically
• produces 100-1000 patterns per image

31
Unsupervised detector training - 2
Pattern Space (100 dimensions)
32
Unsupervised detector training - 3
100 detectors
100-1000 images
33
Parameter Estimation

• Take training images. Consider set of detectors

34
Parameter Estimation

• Signal? Clutter? Correspondence?

optimize for representation (ML on generative
models)
35
ML using EM
1. Current estimate
2. Assign probabilities to constellations
Large P
...
pdf
Image i
Image 1
Image 2
Small P
3. Use probabilities as weights to reestimate
parameters. Example ?
Large P
x

Small P
x

new estimate of ?
36
Final Part Selection
Model 1
Choice 1
Parameter Estimation
Model 2
Choice 2
Parameter Estimation
Preselected Parts (?100)
Predict / measure model performance (validation
set or directly from model)
37
Frontal Views of Faces

• 200 Images (100 training, 100 testing)
• 30 people, different for training and testing

38
Learned face model
Preselected Parts
Test Error 6 (4 Parts)
Parts in Model
Model Foreground pdf
Sample Detection
39
Face images
40
Background images
41
Rear Views of Cars

• 200 Images (100 training, 100 testing)
• Only one image per car
• High-pass filtered

42
Learned Model
Preselected Parts
Test Error 13 (5 Parts)
Parts in Model
Model Foreground pdf
Sample Detection
43
Detections of Cars
44
Background Images
45
Wildcard Parts
46
Context
Parts
Shape
47
Dilbert
125 examples
77 examples
vs.
48
Dilbert Model
Model Foreground pdf
Preselected Parts
Parts in Model
Sample Detection
Test Error 15 (4 Parts)
49
Manual vs. Automatic Part Design Selection
Markus Weber move task up left color thicker
Automatic
TaskE vs. No E
Similar to manual
Used in best models
Manual
?16 Error
? 7 Error
50
Strictly Unsupervised Learning (Single Class)
Training Set
Test Error
100 Faces (so far)
...
6
66 Faces
...
10
50 Faces
...
12
51
Which Part Size and Scale?
Markus Weber Trade-off informativity occlusion
sensitivity
12
14
18
116
52
Multi-Scale Experiment
Preselected Parts
1
2
3
4
5
6
Gaussian Pyramid
53
Multi-Scale Detection Performance
Test Error
single scale 6 (4 parts) multi-scale 11
(5 parts)
54
Occlusion Experiment
Markus Weber Say what we do here. Occlusion in
TRAINING and TESTING. Is this possible? Fewer
Errors below.
Test Error
no occlusion 6 (4 parts) occlusion 18
(5 parts)
Are learning and detection possibleunder partial
occlusion?
55
View - Based 3D Model
56
Background Examples
57
Test Images with Faces
58
3D Orientation Tuning
Markus Weber Canonical views add axes info
Profile
Frontal
59
(No Transcript)
60
Johanssons experiments 70s
61
What is your brain doing?
Input
Output

• Xi(t)

• Combinatorial
• Missing features
• Noise

62
From trajectories to labels
Input
Output

• xi,vi

Li EL
i 1,,M
63
Representation dilemma
XWL(t)
???
64
What is this???
65
Probabilistic approach to learning

• learn joint p.d.f. Pr(data labels)
• labelling by maximizing likelihood
• Unfortunately
• High dimensional p.d.f.
• cumbersome (62 variables -gt 103 -104 param.)
• need lots of learning examples
• Search cost M! (try all labellings)
• E.g. M16 -gt 16!21013

66
Approximate decomposition

• Human body as kinematic chain
• Markov property
• Fewer parameters
• Find global max with dynamic programming
• polynomial cost

Pr(A, B, C, D, E) Pr(A, B, C)Pr(DB,
C)Pr(EC, D)
67
Triangulated decomposition (by hand)
H
3
N
4
LS
LS
5
6
LE
LE
2
7

• 102 - 103 parameters
• Markov property
• Solve in O(M4 )
• See also recent results on turbo-decoding and
  bayesian inference

8
LW
LH
LH
9
10
LK
LK
11
12
LA
LA
13
14
LF
LF
(a)
68
Training sequences
69
Unsupervised model
Means
Correlations
G
B
G
B
F
F
D
D
J
C
J
C
H
H
L
A
E
A
E
I
K
L
I
K
70
Positive example
71
Negative example 1
72
Negative example 2
73
Person walking left-to-right?
74
Learning for visual recognition

• Supervised
• Manual alignment/correspondence of training
  examples
• Unsupervised (1 class)
• Training images contain examples of 1 class
  clutter
• Unsupervised (multi-class)
• Turn your camera on, come back one year later

75
OBJECTS
INANIMATE
ANIMALS
PLANTS
MAN-MADE
NATURAL
VERTEBRATE
..
MAMMALS
BIRDS
GROUSE
BOAR
TAPIR
CAMERA
76
Discovering multiple classes

• Cars (rear and side view)
• Leaves (three species)
• Human Heads (90o viewing range)

77
Preselected Parts for Mixture Models
Heads
Cars
Leaves
78
Mixture Model of Heads
79
Tuning of Mixture Models
80
Tuning of Mixture Models
81
Summary

• Probabilistic constellation models
• Learning based on Maximum Likelihood
• Unsupervised learning of object categories
• 3D invariance
• Biological motion

82
Main accomplices
Markus Weber
Thomas Leung
Yang Song
Max Welling
Michael Burl
83
(No Transcript)
84
Referencesavailable from www.vision.caltech.edu

• CVPR98 (affine shape)
• FG00 (viewpoint invariance)
• ECCV00 (EM algor. for unsupervised learning)
• CVPR00 (learning of multiple classes)
• ECCV00, CVPR00, NIPS01, CVPR01 (biological
  motion)
• Funded by
• National Science Foundation
• Sloan Foundation
• INTEL