Pattern Recognition Unit 1 Overview

1
Pattern RecognitionUnit 1 - Overview

• Jim Rehg
• School of Interactive ComputingGeorgia Institute
  of Technology
• August 18-25, 2008

2
Perspectives on Pattern Recognition

• Our goal is to uncover the underlying
  organization for what might otherwise look like a
  laundry list of methods
• Linear and Tree Classifiers
• Gaussian Mixture Classifiers
• Logistic Regression
• Neural Networks
• Support Vector Machines
• Gaussian Process Classifiers
• AdaBoost

3
Goal of Pattern Recognition

• Learn a function that maps features x to
  predictions C, given a dataset D Ck , xk
• Elements of the problem
• Knowledge about data-generating process and task
• Design of feature space for x based on data
• Decision rule f x C
• Loss function L(C,C) for measuring quality of
  prediction
• Learning algorithm for computing f from D
• Empirical measurement of classifier performance
• Visualization of classifier performance and data
  properties
• Computational cost of classification (and
  learning)

4
Example Skin Detection in Web Images

• Images containing people are interesting
• Most images with people in them contain visible
  skin
• Skin can be detected in images based on its
  color.
• Goal Automatic detection of adult images
• DEC Cambridge Research Lab, 1998

5
Physics of Skin Color

• Skin color is due to melanin and hemoglobin.
• Hue (normalized color) of skin is largely
  invariant across the human population.
• Saturation of skin color varies with
  concentration of melanin and hemoglobin (e.g.
  lips).
• Detailed color models exist for melanoma
  identification using calibrated illumination.
• But observed skin color will be effected by
  lighting, image acquisition device, etc.

6
Skin Classification Via Statistical Inference

• Joint work with Michael Jones at DEC CRL
• M. Jones and J. M. Rehg, Statistical Color
  Models with Application to Skin Detection, IJCV,
  2001.
• Model color distribution in skin and nonskin
  cases
• Estimate p(RGB skin) and p(RBG nonskin)
• Decision rule f RGB skin, nonskin
• Pixel is skin when p(skin RGB) gt p(nonskin
  RGB)
• Data set D
• 12,000 example photos sampled from a 2 million
  image set obtained from an AltaVista web crawl
• 1 billion hand-labeled pixels in training set

7
Some Example Photos
Example skin images
Example non-skin images
8
Manually Labeling Skin and Nonskin
Labeled skin pixels are segmented by hand
Labeled nonskin pixels are easily obtained from
images without people
9
Skin Color Modeling Using Histograms

• Feature space design
• Standard RGB color space - easily available,
  efficient
• Histogram probability model

P(RBG skin)
P(RBG nonskin)
10
Skin Color Histogram
Segmented skin regions produce a histogram in RGB
space showing the distribution of skin colors.
Three views of the same skin histogram are shown
11
Non-Skin Color Histogram
Three views of the same non-skin histogram
showing the distribution of non-skin colors
12
Decision Rule
Class labels skin C1 nonskin C0
Equivalently
13
Likelihood Ratio Test
14
Skin Classifier Architecture
P(RBG skin)
Output skin
Input Image
P(RBG nonskin)
15
Measuring Classifier Quality

• Given a testing set T Cj , xj that was not
  used for training, apply the classifier to obtain
  predictions
• Testing set partitioned into four categories

16
Measuring Classifier Quality
A standard convention is to report
Fraction of positive examples classified correctly
Fraction of negative examples classified
incorrectly
17
Trading Off Types of Errors

• Consider
• Classifier always outputs f 1 regardless of
  input
• All positive examples correct, all negative
  examples incorrect
• dR 1 and fR 1
• Consider
• Classifier always outputs f 0 regardless of
  input
• All positive examples incorrect, all negative
  examples correct
• dR 0 and fR 0

18
ROC Curve
1
0.75
Detection Rate dR
0.5
Generating ROC curve does not require
classifier retraining
0.25
0
0
1
0.5
0.25
0.75
False Positive Rate fR
19
ROC Curve
1
A fair way to com- pare two classifiers is to
show their ROC curves for the same T
0.75
Detection Rate dR
0.5
ROC stands for Receiver Oper-ating
Characteristic and was originally developed for
tuning radar receivers
0.25
0
0
1
0.5
0.25
0.75
False Positive Rate fR
20
Scalar Measures of Classifier Performance
1
0.75
Detection Rate dR
0.5
Area under the ROC curve
0.25
0
0
1
0.5
0.25
0.75
False Positive Rate fR
21
ROC Curve Summary

• ROC curve gives application independent measure
  of classifier performance
• Performance reports based on a single point on
  the ROC curve are generally meaningless
• Several possible scalar summaries
• Area under the ROC curve
• Equal error rate
• Compute ROC by iterating over the values of a
• Compute the detection and false positive rates on
  the testing set for each value of a and plot the
  resulting point.

22
Example Results
Skin examples
Nonskin examples
23
Skin Detector Performance
Extremely good results considering only color of
single pixel is being used. Best published
results (at the time) One of the largest datasets
used in a vision model (nearly 1 billion labeled
pixels).
Detection Rate dR
False Positive Rate fR
But why does it work so well ???
24
Analyzing the color distributions
Why does it work so well?
2D color histogram for photos on the
web projected onto a slice through the
3D histogram
Surface plot of the 2D histogram
25
Contour Plots
Full color model (includes skin and non-skin)
26
Contour Plots Continued
Non-skin model
Skin model
Skin color distribution is surprisingly
well-separated from the background distribution
of color in web images
27
Comparison to Mixture Models

• Both histogram and mixture models are examples of
  graphical models.
• Bin size controls generalization of histogram
• Size 32 gave the best performance
• Mixture models have often been used for skin
  color modeling in small sample size cases.
• We found histograms to give better accuracy
• They are also much faster to evaluate
• lt Show figures from CRL technical report gt

28
Adult Image Detection

• Observation Adult images usually contain large
  areas of skin
• Output of skin detector can be used to create
  feature vector for an image
• Adult image classifier trained on feature vectors
• Exploring joint image/text analysis

Image
Skin Features
Neural net Classifier
Skin Detector
Adult?
Text Features
HTML
Classifier
29
Examples
Classified as not adult
Incorrectly classified as adult - closups of
faces are a failure mode due to large amounts of
skin
Classified as not adult
Classified as not adult
30
Performance of Adult Image Detector
31
Adult Image Detection Results
Two sets of html pages collected. Crawl A Adult
sites (2365 pages, 11323 images). Crawl B
Non-adult sites (2692 pages, 13973 images).

image-based text-based combined OR
detector detector
detector
-----------------
------------- ------------------- of
adult images rated correctly
(set A) 85.8 84.9
93.9 of non-adult images rated
correctly (set B) 92.5
98.9 92.0
32
Computational Cost Analysis

• General image properties
• Average width 301 pixels
• Average height 269 pixels
• Time to read an image .078 sec
• Skin Color Based Adult Image Detector
• Time to classify .043 sec
• Implies 23 images/sec throughput

33
Person Detection From Skin Detection

• Skin detector gives evidence for the presence of
  people, but has false positives and negatives.
• Use skin detector output for person detection
• Construct feature vector from detected skin
  pixels.
• Classify image into person/non-person
• Features
• Percent of pixels in image detected as skin
• Average probability of skin pixels
• Largest connected component of skin

34
Person Detection Example Results
Person
Person
No Person
35
Person Detection Results Continued
No Person
No Person
Person
36
Person Detector Performance
Two classifiers were built using these measures
on 1400 training images. A test set of 456
images was used to evaluate the classifier.
Classifier Performance
Training Testing
examples
examples Neural network 76.2
74.3 Decision tree 75.8 72.1
37
Applications of Person Detection

• Person Detected tag for media search
• Skin and face analysis tag photos and video
  frames with people in them.
• Improved ranking of query returns Photos of
  people appear at top of list.
• Image similarity measure
• Photos with people in them are grouped together.
• Can be used during query refinement.