Detecting Pedestrians by Learning Shapelet Features

1
Detecting Pedestrians by Learning Shapelet
Features

• Payam Sabzmeydani and Greg Mori
• Vision and Media Lab
• School of Computing Science
• Simon Fraser University

2
Problem

• Given a still image, we want to find and locate
  the pedestrians in the image
• Clothing (color, appearance)
• Body pose
• Applications
• Automated surveillance systems
• Image search and retrieval
• Robotics
• Intelligent vehicles

3
Problem
4
Problem

• Classification-based detection
• Classify a window as pedestrian or non-pedestrian
• Search exhaustively the scale-space image

• Different cues
• Wavelet coefficients (Mohan et al., PAMI 2001)
• Oriented gradients (Dalal and Triggs, CVPR 2005)
• SIFT features (Leibe et al., CVPR 2005)
• Edgelet features (Wu and Nevatia, ICCV 2005)
• Shapelet features (Sabzmeydani and Mori, CVPR
  2007)

5
Datasets

• MIT Standing pose, simple background, no
  occlusion
• INRIA Standing pose, complex background,
  partial occlusions

6
Previous Work

• Dalal Triggs (CVPR 2005)
• HOG features SVM

7
Previous Work

• Wu Nevatia (ICCV 2005)
• Edgelet features short line and curve segments
• AdaBoost

8
Our Method

• Compute low-level gradient features
• Oriented filter responses
• Learn mid-level features for detecting
  pedestrians
• Shapelet features
• Build final classifier from shapelet features

9
Low-level Features

• Filter responses
• Image gradient in different directions

10
Low-level Features

• Smoothed gradient responses in different
  directions

11
Shapelet Features

• A weighted set of low-level gradient features
  inside a sub-window of the detection window
• Characteristics
• Simple and low-dimensional
• Learned exclusively for our object classes
• Highly discriminative
• Local effective area useful to model separate
  parts instead of the whole body

12
Shapelet Features
13
Learning Shapelet Features

• Learned using AdaBoost (Viola and Jones, 2001)
• Extract low-level features in sub-window
• Select subset of features using AdaBoost
• Find those which discriminate between pedestrian
  and background classes

14
AdaBoost Algorithm
W
w
15
Low-level features as weak classifiers

• Each low-level feature can provide us many weak
  classifiers
• AdaBoost will combine weak classifiers to form a
  better classifier

16
Shapelet features

• Train classifiers in sub-windows
• Use the output of a classifier as the shapelet
  feature response

17
Shapelet Features
18
Shapelet Features
19
Final Classifier

• Take all shapelet features
• Learned at many sub-windows of detection window
• Run AdaBoost again to select weighted subset of
  shapelet features for final classifier

20
Final Classifier
21
Shapelet Feature Size

• Small, Medium, and Large features
• Capture different scales of information

22
Normalization

• Why normalize?
• Different lighting, shadows, different contrast,
  
• How to normalize?
• Per shapelet feature L2-norm

23
Normalization
24
Results on INRIA Dataset
25
Error examples

• Most non-pedestrian-like pedestrians (false
  negatives)
• Most pedestrian-like non-pedestrians (false
  positives)

26
Future work

• Detecting other objects
• Use image context or segmentation
• Pyramid of features

27
(No Transcript)
28
References

• N. Dalal and B. Triggs. Histograms of oriented
  gradients for human detection. CVPR 2005.
• B. Wu and R. Nevatia. Detection of multiple,
  partially occluded humans in a single image by
  bayesian combination of edgelet part detectors.
  ICCV 2005.
• P. Viola and M. Jones. Rapid object detection
  using a boosted cascade of simple features. SCTV
  2001.

29
Problem
30
Problem
31
Bootstrapping
32
Mid-level Features