Soft%20Biometrics%20at%20CUBS

1
Soft Biometrics at CUBS

• Venu Govindaraju
• CUBS, University at Buffalo
• govind_at_buffalo.edu
• www.cubs.buffalo.edu

2
Background

• Traits of biometrics
• Universality
• Distinctiveness
• Permanence
• Collectability
• Acceptability
• Present perfect?
• No biometric is truly universal. It is estimated
  that 2-4 of the population have unusable
  fingerprints
• Each biometric has a lower bound for errors
  (constraint of algorithm individuality)
• Individual biometrics need to be augmented by
  other biometrics (multi-modal) or traits (soft
  biometrics)

3
Soft Biometrics

• Definition1
• Soft biometric traits are those characteristics
  that provide some information about the
  individual but are not distinctive enough to
  sufficiently differentiate any two individuals

1

• Soft Biometrics
• Not very distinctive
• Can be used to augment regular biometrics
• Not typically used during verification/identificat
  ion
• More intuitive than strong biometrics

1 A. K. Jain, S. Dass, K. Nandakumar, Soft
Biometrics for Personal Identification, SPIE
Defense and Security Symposium 2003
4
Soft Biometrics Examples

• Other classification
• Continuous Age, Height, Weight etc.
• Discrete Gender, Eye Color, Ethnicity etc.

5
Motivation

• Heckathorn3 have shown that a combination of
  personal attributes can be used to identify the
  individual reliably
• Binning and Indexing
• Hardening primary biometric
• Speech Recognition
• Can be used to tune individual biometrics
• Socially aware computing (call centers)?

6
Extracting Soft Biometric Traits

• Devices
• Color video
• Stereo images
• Challenges
• Controlled vs Uncontrolled environment
• Pose variations
• Illumination variation
• Complex backgrounds
• Feature selection and extraction
• Features used in traditional biometrics do not
  encode soft biometric traits
• Decision systems (soft thresholds)

7
Problems in Representation
Fuzzy class boundaries
Purely statistical features
8
Soft Biometrics Research at CUBS

• Speech
• Gender Identification
• Accent Identification
• Face
• Face Catalog Semantic Face Retrieval
• Gender Classification
• Skin
• Skin spectroscopy

9
Soft Biometric Traits in Speech

• Gender
• There exists a difference in the pitch period
  between genders
• This difference is fundamental in the
  discrimination between males and females
• Accent1
• Temporal features onset time, closure/voicing/wor
  d duration
• Prosodic/Intonation slope patterns
• Formant frequencies
• Age
• The average power measurement and speech rate
  are used as indicators for measurement of
  agedness in a speaker

1A Study of Temporal Features and Frequency
characteristics in American English Foreign
Accent L.M. Arslan, J.H.L. Hansen , Journal of
the Acoustical society of America, July 1997
10
Uses of Soft Biometrics in Speech
Soft Biometrics for binning
P(wx1)
P(wx1y)
Primary Biometric
Soft Biometric(s)
Soft Biometrics for improving accuracy
11
Loose Gender Classification (PITCH)
Results

• 3 Methods
• Fast Fourier Transform
• Linear Predictive Analysis
• Cepstral Analysis
• Data
• 75 files
• Males -41, Females -34

Male Low Male Medium Male High Female Low
Female Medium Female High 132Hz
156Hz 171Hz 205Hz
230Hz 287Hz
12
Definition of Accent (linguistics)

• An accent is the perceived peculiarities of
  pronunciation and intonation of a speaker or
  group of speakers
• A foreign accent is defined in a way that the
  phonology of the spoken language is modified by
  the phonology of another language, more familiar
  to the speaker
• 3 major language groups
• American
• Chinese
• Indian

13
Proposed Approach for Accent

• First identify the accent markers
• Determine the effect of gender and
  co-articulation
• Initially develop a text dependent model
• Accumulate evidence over time
• Features
• formants
• phoneme duration
• instantaneous (mel)cepstral slopes
• HMMs

14
Accent Markers

• A look at various non-native pronunciations of
  English
• CHINESE
• r read sometimes as l or w
• v read as w
• th read as d
• n and l often confused
• Often drop articles like the and a
• INDIAN SUBCONTINENT
• Use of the rhotic r
• Use of rolling l
• Fast speech tempo with choppy syllables
• Rhythmic variation of pitch

Websters Revised Unabridged
Dictionary Definition of non-native
pronunciations of English wordIQ.com
15
MALES PHONEME CONTAINING L
American - Indian -
F3
F3
PLEASE
STELLA
F2
F2
F3
F3
SLABS
PLASTIC
F2
F2
16
MALES PHONEMES CONTAINING R AND AA
American - Indian -
17
FEMALES SEGMENTED PHONEMES L, R, AA
American - Indian -
18
Soft Biometrics for Law Enforcement
Novel Forensic System
19
Law Enforcement Application Face Catalog

• User can select some facial feature to describe.
• System will prompt the user after each query
  with the best feature for the next query.

20
Related Work

• Identikit 1 composes faces by putting together
  transparencies of facial features.
• Evofit 2, automate the process of identikits.
• Phanthomas 3 face composition using elastic
  graph matching.
• CAFIIRIS 4 and Photobook 5 use PCA for face
  composition and matching.
• But general description of users are semantic!

1. V. Bruce, Recognizing Faces, Faces as Patterns,
   pp. 37-58, Lawrence Earlbaum Associates, 1988
2. Frowd, C.D., Hancock, P.J.B., Carson, D.
   (2004). EvoFIT A Holistic, Evolutionary Facial
   Imaging Technique for Creating Composites, ACM
   TAP, Vol. 1 (1)
3. Phantomas Elaborate Face Recognition .Product
   description http//www.global-security-solutions.
   com/FaceRecognition.htm 
4. J. K. Wu, Y. H. Ang, P. C. Lam, S. K. Moorthy, A.
   D. Narasimhalu, Facial Image Retrieval,
   Identification, and Inference System
5. A. Pentland, R. Picard, S. Sclaroff, Photobook
   tools for content based manipulation of image
   databases, Proc. SPIE Storage and Retrieval for
   Image and Video Databases II, vol. 2185

21
Face Catalog System Overview
Semantic Face Retrieval System
Input Image
Face Detection
Meta Database
Lip Location and parameterization
Face Image Database
Semantic Description
Eye Location
Parameterization of other Features
Sorted Images
Query Sub-System
Prompting Sub-System
user
22
Enrollment Sub-System

• Face Detection.
• Lips and eye detection.
• Locate and parameterize other features.

23
Query Sub-System

• Pruning images based on descriptions given?
• What if user makes a mistake in one of the
  description.
• Ranking images based on their probability of
  being the required person is a better idea.
• Bayesian learning can be used to update
  probability of each face being the required one.
• Prompting users the feature with highest entropy
  at each step.

24
Example Query
Query
Query Spectacles Yes
Query Spectacles Yes Mustache Yes
Query Spectacles Yes Mustache Yes
Nose Big
Probabilities of Faces
25
Results

• Results of Enrollment Sub-system (Database of 150
  images)
• Results of Query (25 users, 125 test cases)

 
26
Gender Classification in Images

• Gender classification
• Identifying male or female from facial image
• Existing approaches
• Geometric feature based 1-2
• Appearance feature based (raw data feature or PCA
  classifier) 3
• Approaches using other features, e.g., wrinkle
  and skin color 4

1 A. Burton, V. Bruce and N. Dench, Whats the
difference between men and women? Evidence from
facial measurements, Perception, vol. 22,
pp.153-176, 1993. 2R. Brunelli and T. Poggio,
Hyperbf network for gender classification,
DARPA Image Understanding Workshop, pp. 311-314,
1992. 3B.A. Golomb, D.T. Lawrence, T.J.
Sejnowski, Sexnet A Neural Network Identifies
Sex from Human Faces, Advances in Neural
Information Processing Systems3, R.P Lippmann,
J.E. Moody, D.S. Touretzky, eds. Pp. 572-577,
1991. 4 J. Hayashi, M. Yasumoto, H. Ito, H.
Koshimizu, Age and gender estimation based on
wrinkle texture and color of facial images,,
Proceedings of 16th International Conference on
Pattern Recognition, vol. 1, pp. 405 - 408, 11-15
Aug. 2002
27
Gabor Feature based gender classification system
28
Facial image Normalization

• Mapping feature points to fixed positions
• Feature points
• Centers of two pupils
• Tip of the nose
• Normalized image
• 64 by 64
• Convert from color to grayscale by averaging RGB
  components

29
Gabor feature

• Gabor filter and Gabor wavelet B.S. Manjunath,
  et al, PAMI, 1996

Gabor Filter
Fourier Transform of g(x, y)
Gabor Wavelet
30
Gabor feature (cont.)

• Redundancy reduction B.S. Manjunath, et al,
  PAMI, 1996
• Let and denote the lowest and highest
  frequencies of interest
• are determined by

31
Gabor feature (cont.)

• Characteristics of Gabor wavelet
• A powerful tool to capture changes of signals
• Selective on certain frequency and orientation by
  setting parameters m, n
• Gabor feature for gender classification
• Gabor WT at 4 scalses, 4 orientations (m 0, ..,
  3 n 0, , 3)
• Each output image of Gabor WT (64 by 64) is
  divided into non-overlapping blocks of the size
  2m2 by 2m2 (m the scale number).
• Average of magnitudes in each block as a feature
• 
  
• Total number of features

32
Gabor feature (cont.)
33
Classification

• Features
• 1360-dimensional training and testing vectors fed
  into SVM classifier
• Classifier
• SVM with Gaussian RBF kernel 6 (B. Moghaddam,
  et al, PAMI 2002)
• Adjust ? to minimize error rate
• 1360 features from Gabor WT (in 4 scales, 4
  orientations) of 6464 input image
• Training and testing vectors (of 1360 dimensions)
  normalized into unit vectors

34
Experimental Results

• Dataset AR face database A.M. Martinez and R.
  Benavente, The AR face database, CVC Tech.
  Report 24, 1998
• Overall 3265 frontal facial images including 136
  Caucasian people (768 by 576, color)
• Training 2246 samples including 91 individuals
• Testing 1019 samples including 45 individuals
• Test 1
• 393 regular samples. Accuracy 96.2
• Test 2
• 626 irregular samples (occluded by dark
  sun-glasses or masks) Accuracy 92.7

Method Accuracy of test 1 Accuracy of test 2
Gabor feature SVM with Gaussian RBF kernel 96.2 92.7
Raw data feature SVM with Gaussian RBF kernel 94.7 89.8
35
Skin Spectroscopy

• Measures the composition of the skin using
  IR(Deep tissue biometric)
• Based on spectroscopy
• Fool proof against fake fingers (Can detect
  liveness)
• Can be easily integrated into solid state devices
• Immune to surface degradations
• Currently implemented by only one Vendor
  (Lumidigm Inc)

Skin composition
36
Chromophores in skin

• Melanin
• Absorbs light at all wavelengths
• Absorbance decreases with increase in wavelength
• Hemoglobin
• Strongest absorption bands in 405 430 nm and
• 540 580 nm.
• Lowest absorption beyond 620 nm
• Can be used for liveness testing
• Collagen, Keratin, Carotene

37
Spectra of Melanin and Hemoglobin
38
Sample Skin Spectrum
39
Sample skin spectrum (contd.)
40
Sample skin spectrum (contd.)
41
Results so far

• Soft classification based on skin color
• Melanin index used as indicator of skin color
• Spectral difference noticed between different
  skin locations on the same individual

42
Thank You

• ssc5_at_cedar.buffalo.edu