Invariant Large Margin Nearest Neighbour Classifier

1
Invariant Large Margin Nearest Neighbour
Classifier

• M. Pawan Kumar
• Philip Torr
• Andrew Zisserman

2
Aim

• To learn a distance metric for invariant
• nearest neighbour classification

3
Aim

• To learn a distance metric for invariant
• nearest neighbour classification

Target pairs
4
Aim

• To learn a distance metric for invariant
• nearest neighbour classification

Impostor pairs
Problem Euclidean distance may not provide
correct nearest neighbours
Solution Learn a mapping to new space
5
Aim

• To learn a distance metric for invariant
• nearest neighbour classification

6
Aim

• To learn a distance metric for invariant
• nearest neighbour classification

Euclidean Distance
Learnt Distance
7
Aim

• To learn a distance metric for invariant
• nearest neighbour classification

Euclidean Distance
Learnt Distance
8
Aim

• To learn a distance metric for invariant
• nearest neighbour classification

Euclidean Distance
Learnt Distance
9
Aim

• To learn a distance metric for invariant
• nearest neighbour classification

Euclidean Distance
Learnt Distance
10
Aim

• To learn a distance metric for invariant
• nearest neighbour classification

Euclidean Distance
Learnt Distance
11
Aim

• To learn a distance metric for invariant
• nearest neighbour classification

Learn a mapping to new space
12
Aim

• To learn a distance metric for invariant
• nearest neighbour classification

13
Aim

• To learn a distance metric for invariant
• nearest neighbour classification

Euclidean Distance
Learnt Distance
14
Motivation

• Face Recognition in TV Video

Euclidean distance may not give correct nearest
neighbours
Learn a distance metric
15
Motivation

• Face Recognition in TV Video

Invariance to changes in position of features
16
Outline

• Large Margin Nearest Neighbour (LMNN)
• Preventing Overfitting
• Polynomial Transformations
• Invariant LMNN (ILMNN)
• Experiments

17
LMNN Classifier

• Weinberger, Blitzer and Saul - NIPS 2005

Learns a distance metric for Nearest Neighbour
classification
18
LMNN Classifier

• Weinberger, Blitzer and Saul - NIPS 2005

Learns a distance metric for Nearest Neighbour
classification
Distance between xi and xj D(i,j) (xi-xj)T
LTL (xi-xj)
19
LMNN Classifier

• Weinberger, Blitzer and Saul - NIPS 2005

Learns a distance metric for Nearest Neighbour
classification
Distance between xi and xj D(i,j) (xi-xj)T
M (xi-xj)
M 0
Convex Semidefinite Program (SDP)
min Sij D(i,j) subject to M
0
Global minimum
20
LMNN Classifier

• Weinberger, Blitzer and Saul - NIPS 2005

Learns a distance metric for Nearest Neighbour
classification
D(i,k) D(i,j) 1
- eijk
eijk 0
min Sijk eijk subject to M
0
Convex SDP
21
LMNN Classifier

• Weinberger, Blitzer and Saul - NIPS 2005

Learns a distance metric for Nearest Neighbour
classification
min Sij D(i,j) ?H Sijk
eijk subject to M 0
D(i,k) D(i,j) 1- eijk
eijk 0
Solve to obtain optimum M
Complexity Polynomial in number of points
22
LMNN Classifier

• Weinberger, Blitzer and Saul - NIPS 2005

Advantages

• Trivial extension to multiple classes
• Efficient polynomial time solution

23
Outline

• Large Margin Nearest Neighbour (LMNN)
• Preventing Overfitting
• Polynomial Transformations
• Invariant LMNN (ILMNN)
• Experiments

24
L2 Regularized LMNN Classifier
Regularize Frobenius norm of L

• L2 S Mii

min Sij D(i,j) ?H Sijk eijk ?R
Si Mii subject to M 0
D(i,k) D(i,j) 1- eijk
eijk 0
L2-LMNN
25
Diagonal LMNN
Learn a diagonal L matrix gt Learn a diagonal M
matrix
min Sij D(i,j) ?H Sijk
eijk subject to M 0
D(i,k) D(i,j) 1- eijk
eijk 0
Mij 0, i ? j
Linear Program
D-LMNN
26
Diagonally Dominant LMNN
Minimize 1-norm of off-diagonal element of M
min Sij D(i,j) ?H Sijk eijk ?R
Sij tij subject to M 0
D(i,k) D(i,j) 1- eijk
eijk 0
tij Mij, tij -Mij , i ? j
DD-LMNN
27
LMNN Classifier
What about invariance to known transformations?
Append input data with transformed versions
Inefficient
Inaccurate
Can we add invariance to LMNN?

• No Not for a general transformation

• Yes - For some types of transformations

28
Outline

• Large Margin Nearest Neighbour (LMNN)
• Preventing Overfitting
• Polynomial Transformations
• Invariant LMNN (ILMNN)
• Experiments

29
Polynomial Transformations
a
x
Rotate x by an angle ?
b
Taylors Series
30
Polynomial Transformations
a
x
Rotate x by an angle ?
b
a
cos ?
-sin ?
cos ?
b
sin ?
1
a
b
-a/2
b/6
T(?,x) X ?
?
b
a
-b/2
-a/6
?2
?3
X
?
31
Why are Polynomials Special?
SD-Representability of Polynomials
Lasserre, 2001
Sum of squares of polynomials
32
Why are Polynomials Special?
?1
?2
D I S T A N C E
Sum of squares of polynomials
33
Outline

• Large Margin Nearest Neighbour (LMNN)
• Preventing Overfitting
• Polynomial Transformations
• Invariant LMNN (ILMNN)
• Experiments

34
ILMNN Classifier
Learns a distance metric for invariant Nearest
Neighbour classification
xi
xj
Polynomial trajectories
xk
35
ILMNN Classifier
Learns a distance metric for invariant Nearest
Neighbour classification
xi
xj
Polynomial trajectories
xk
M 0

• Bring target trajectories closer

Minimize maximum distance

• Move impostor trajectories away

Maximize minimum distance
36
ILMNN Classifier
Learns a distance metric for invariant Nearest
Neighbour classification
xi
xj
Polynomial trajectories
xk

• Use SD-Representability. One Semidefinite
  Constraint.

• Solve for M in polynomial time.

• Add regularizers to prevent overfitting.

37
Outline

• Large Margin Nearest Neighbour (LMNN)
• Preventing Overfitting
• Polynomial Transformations
• Invariant LMNN (ILMNN)
• Experiments

38
Dataset
Faces from an episode of Buffy The Vampire
Slayer
11 Characters
24,244 Faces (with ground truth labelling)
Thanks to Josef Sivic and Mark Everingham
39
Dataset Splits
Experiment 1

• Random permutation of dataset
• 30 training
• 30 validation (to estimate ?H and ?R)
• 40 testing

Suitable for Nearest Neighbour-type Classification
Experiment 2

• First 30 training
• Next 30 validation
• Last 40 testing

Not so suitable for Nearest Neighbour-type Classif
ication
40
Incorporating Invariance
Invariance of feature position to Euclidean
Transformation
Approximated to degree 2 polynomial using
Taylors series
Derivatives approximated as image differences
Rotated Image
Image
41
Incorporating Invariance
Invariance of feature position to Euclidean
Transformation
Approximated to degree 2 polynomial using
Taylors series
Derivatives approximated as image differences
-

Smooth Image
Derivative
Smooth Image
42
Training the Classifiers
Problem Euclidean distance provides 0 error
Solution Cluster.
43
Training the Classifiers
Problem Euclidean distance provides 0 error
Solution Cluster. Train using cluster centres.
Efficiently solve SDP using Alternative Projection
Bauschke and Borwein, 1996
44
Testing the Classifiers
Map all training points using L
Map the test point using L
Find nearest neighbours. Classify.
45
Timings
46
Accuracy
47
Accuracy
48
Accuracy
49
Accuracy
50
True Positives
51
Conclusions

• Regularizers for LMNN
• Adding invariance to LMNN
• More accurate than Nearest Neighbour
• More accurate than LMNN

52
Future Research

• D-LMNN and D-ILMNN for Chi-squared distance
• D-LMNN and D-ILMNN for dot product distance
• Handling missing data
• Sivaswamy, Bhattacharya, Smola, JMLR 2006
• Learning local mappings (adaptive kNN)

53
Questions ??
54
(No Transcript)
55
False Positives
56
Precision-Recall Curves
Experiment 1
57
Precision-Recall Curves
Experiment 1
58
Precision-Recall Curves
Experiment 1
59
Precision-Recall Curves
Experiment 2
60
Precision-Recall Curves
Experiment 2
61
Precision-Recall Curves
Experiment 2