Face Categorization using SIFT features

1
Face Categorization using SIFT features

• Mei-Chen (Mei) Yeh
• ECE 281B
• 06/13/2006

2
Bush vs Schwarzenegger
3
Serena Williams vs Venus Williams
4
Main goal

• Face categorization problem
• Object class recognition techniques have seen
  progress in recent years (ex bag-of-words
  models)
• Different people are considered different object
  classes
• Integration of SIFT features
• Does SIFT help face recognition? To what degree?
• Target on a few people
• Applications
• Face annotation in family albums
• Name-based photo search

5
Method

• Features SIFT
• Bag-of-words representation for faces
• Each SIFT feature is considered a codeword
• Build a dictionary based on training samples
• Each face is represented as a histogram over
  codewords
• Learning Naïve Bayes Classifiers

6
codeword 2
codeword 1
SIFT feature vectors from training samples
codeword 3
128-d feature space
7
Datasets

• The BioID Face Database (simple)
• 1521 images with 23 people
• Variety of illumination, background and face size

8
22 categories, 1613 images 70 for training, 30
for testing
9
Measurement

• Confusion Matrix

Classifiers
Categories
Average Categorization Rate
10
1 2 3 4 5 6 7 8
9 10 11 12 13 14 15 16 17 18
19 20 21 bg
Average 89.14
11
Datasets

• http//www.cs.berkeley.edu/millert/faces/faceDict
  /NIPSdict/

• Faces in the Wild (challenging)
• 851 images, 10 people 1 non-faces
• Extracted from news videos

12
11 categories, 851 images 70 for training, 30
for testing
13
81.82 0.00 0.00 0.00 0.00 4.55
4.55 0.00 4.55 4.55 0.00 0.00
83.33 0.00 0.00 0.00 0.00 8.33
8.33 0.00 0.00 0.00 4.00 4.00
68.00 0.00 4.00 16.00 4.00 0.00
0.00 0.00 0.00 20.00 0.00 0.00
80.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 8.00 0.00 0.00 8.00
68.00 12.00 0.00 4.00 0.00 0.00
0.00 0.00 0.00 7.14 0.00 0.00
85.71 0.00 7.14 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00
100.00 0.00 0.00 0.00 0.00 0.00
9.09 9.09 0.00 0.00 4.55 0.00
77.27 0.00 0.00 0.00 0.00 4.76
0.00 0.00 0.00 0.00 0.00 0.00
85.71 9.52 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 8.33 8.33
83.33 0.00 0.00 2.44 2.44 2.44
2.44 0.00 0.00 0.00 2.44 0.00
87.80
Average 81.91
14
Conclusions

• SIFT features bag-of-words representation might
  work for face recognition
• Simple dataset good
• Challenging dataset may be improved
• Consider the spatial relations between features
  may be the next step to improve the performance