Attribute and Simile Classifiers for Face Verification

1
Attribute and Simile Classifiersfor Face
Verification
Neeraj Kumar Alexander C. Berg Peter N.
Belhumeur Shree K. Nayar
Columbia University
2
Recognition using visual attributes
4-Legged
White
Male
Orange
Symmetric
Asian
Striped
Ionic columns
Beard
Furry
Classical
Smiling
3
Attributes can define categories
Female
Eyeglasses
Middle-aged
Dark hair
4
Attributes can define categories
Caucasian
Teeth showing
Outside
Tilted head
5
Are these images of the same person?
6
Prior approaches
Low-level features
Images
Verification
Different
7
Our approach attributes
Low-level features
Images
Verification
Attributes

-
Dark hair
Male
Round Jaw
Asian
Different

-
8
3,000,000 face images
MITCMU
Yale A
Yale B
FERET
CMU PIE
FRGC v2.0
9
(No Transcript)
10
Amazon Mechanical Turk
500,000 Attribute Labels 5,000 1 month
See also Deng, et al., 2009 Vijayanarasimhan
Grauman, 2009
11
Learning an attribute classifier
Feature selection
Train classifier
Training images
Low-level features
RGB, Nose
HoG, Eyes
Gender classifier
HSV, Hair
Males
Edges, Mouth

Male
0.87
Females
12
Using attributes to perform verification
Verification classifier
13
Attributes are intuitive
Female
Black hair
Young
Frontal pose
Attractive
Mouth closed
White
Eyes open
14
Describe faces using similes
Penelope Cruz
Angelina Jolie
15
Training simile classifiers
s eyes
Images of Penelope Cruz
Images of other people
s eyes
16
Using simile classifiers for verification
Verification classifier
17
Results
18
Labeled Faces in the Wild (LFW)
http//vis-www.cs.umass.edu/lfw
19
Experimental evaluation

• LFW Image-Restricted Benchmark
• 6,000 face pairs (3,000 same, 3,000 different)
• 10-fold cross-validation

20
Previous state-of-the-art on LFW
as of May 2009
21
Our performance on LFW
85.29 Accuracy (31.68 Drop in error rates)
as of May 2009
22
Human face verification performance
Original 99.20
Cropped 97.53
Inverse Cropped 94.27
23
PubFig dataset benchmark

• Public figures
• Politicians
• Celebrities
• Larger deeper
• 60,000 Images
• 200 People
• 300 Images per person
• Subsets
• Pose
• Illumination
• Expression

http//www.cs.columbia.edu/CAVE/databases/pubfig/
24
Describable visual attributes

• Attributes for recognition
• State-of-the-art performance on LFW
• Enormous set of labeled training images
• Automatic training of classifiers
• First human results on LFW
• New large face dataset PubFig

http//www.cs.columbia.edu/CAVE/projects/faceverif
ication
25
Questions?
http//www.cs.columbia.edu/CAVE/projects/faceverif
ication
26
(No Transcript)
27
Results on PubFig
28
Attribute Labeling Task
29
Human Face Verification Task
30
FaceTracer A Face Search Engine
N. Kumar et al., FaceTracer A Search Engine
for Large Collections of Images with Faces,
ECCV 2008
31
Reference People
Reference Person R1
Reference Person R2
32
Face Regions
(After Alignment)
33
Feature Types
34
Feature Types
RGB, Mean Normalization, No Aggregation
35
Feature Types
Edge Orientations, No Normalization, Histogram
36
Experimental evaluation

• Image-Restricted Benchmark (View 2)
• 6,000 face pairs (3,000 same, 3,000 different)
• 10-fold cross-validation
• Results
• ROC curves
• Average accuracy
• Separate Development Set (View 1)
• 3,200 face pairs (2,200 training, 1,000 testing)

37
Google smiling asian men with glasses 7/08
38
ECCV 2008, FaceTracer smiling asian men with
glasses