Spatial Histograms for Head Tracking

1
Spatial Histograms for Head Tracking

• Sriram Rangarajan
• Department of Electrical and Computer
  Engineering,
• Clemson University,
• Clemson, SC 29634

2
Overview of tracker
Intensity Gradients (works on the boundary of the
ellipse)

• Modules that are complementary to gradients
• Color histograms
• Spatiograms
• Co-occurrence matrices
• Log-Gabor histograms
• Haar histograms
• Edge-orientation histograms

Complementary module (works inside the ellipse)
3
Gradient module
Likelihood score
Gradient score
Normal to points on ellipse
Stan Birchfield, 1998
4
Overview of modules used
Target histogram (from current frame)
Model histogram (from first frame)
Similarity measure
Likelihood score from module
Convert to percentage score, combine with
intensity gradient module likelihood and update
state.
5
Similarity measure between model and target
histograms
Histogram intersection Swain Ballard1991
Likelihood normalization
6
Overview of modules
Color histograms Only color information (no spatial information)
Spatiograms Color information limited spatial information ( global)
Edge-orientation histograms Only spatial information
Co-occurrence matrices Color information limited spatial information ( local)
Log-Gabor histograms Only spatial information (no color)
Haar histograms Only spatial information (no color)
7
Color Histograms

• Ignore spatial information (most cases)
• Computationally efficient, simple, robust and
  invariant to any one-to-one spatial
  transformations

8
Computing color histograms
Pixels in a bin
Number of bins for channel C1
Index for color channel
Single color channel of image
9
Spatiograms

• Higher-order histograms that capture spatial
  information globally
• Captures both values of pixels and a limited
  amount of their spatial relationship
• Bins are weighted by mean and covariance of
  pixels contributing to it

Birchfield and Rangarajan, CVPR 2005
10
Spatiograms and histograms
A histogram (no spatial information)
A histogram (no spatial information)
A histogram (no spatial information)
S
S
S
A spatiogram (some spatial Information)
A spatiogram (some spatial Information)
A spatiogram (some spatial Information)
µ
µ
µ
11
An illustrative example
Three poses of a head
Image generated from histogram
Image generated from spatiogram
12
Co-occurrence matrices

• Used for texture analysis
• Captures the local spatial relationships between
  colors (or gray levels)
• Normally used for gray-level images

No. of pixel pairs with value (x,y)
13
Co-occurrence matrices
Local spatial relationships
13
10
11
(C)
10
11
10
10
13
10
10
13
10
11
11
13
10
11
13
Co-occurrence matrix
Image
Color values (C)
14
Texture histograms


Filter bank
Histogram
Image
(Haar Wavelets or Log-Gabor filters)
15
Haar histograms

• Histogram of image after convolving with 3-level
  Haar pyramid

Haar histogram (at scale S and orientation O.)
Image obtained by convolving with Haar pyramid at
scale S and orientation O
16
Log-Gabor histograms

• Similar to Haar histograms, but uses a bank of
  log-Gabor filters.

Log-Gabor histogram
Image obtained by convolving with filter bank at
scale S and orientation O
17
Edge-orientation histograms

• Obtained from gradient information
• Complete reliance on spatial information
• Histogram bin is decided by orientation of a pixel

18
Computing edge-orientation histograms
Difference of Gaussian kernel (DoG)


Image
Edge-orientation Histogram
19
Edge-orientation histograms

• Computed from gradient images obtained by
  convolving image with Difference of Gaussian
  (DoG) kernel in x and y
• Orientation for pixel along vertical direction is
  0

20
Results log-Gabor histograms
Legend
log-Gaborhistogram
colorhistogram
21
Results Haar histograms
Legend
Haarhistogram
colorhistogram
22
Results Edge-orientation histograms
Legend
Edge-orientationhistogram
colorhistogram
23
Results Spatiograms
Legend
spatiograms
colorhistogram
24
Results Co-occurrence matrices
Legend
Co-occurrence matrices
colorhistogram
25
Overview of results
Color histograms Distracted by skin-colored background
Spatiograms Tracks target in skin-colored background and clutter
Edge-orientation histograms Fails in a cluttered background
Co-occurrence matrices Tracks target in skin-colored background and clutter
Log-Gabor histograms Fails in a cluttered background
Haar histograms Fails in a cluttered background
26
Mean errors in x and y for Sequence 1
27
Mean errors in x and y for Sequence 2
28
Conclusion

• Limited amount of spatial information drastically
  improves tracking results
• Color information also important
• With only spatial information tracker is
  distracted by cluttered background
• With only color tracker is distracted by
  skin-colored background
• Global spatial information is the most effective
  (spatiograms)

29
Thank You!