Activity Analysis based on Human Pose Detection

1
Activity Analysis based on Human Pose Detection
Anima Mishra Sharma Y8104001 Mtech, EE Thesis
Supervisors Dr. Amitabha Mukerjee Dr. KS
Venkatesh
2
Outline

• Motivation
• Literature Survey
• Challenges
• Brief Overview of pose estimation
• Proposed Method for Activity Analysis
• Results
• Future Work

3
Motivation

• Having vast applications
• Video Surveillance
• Tracking
• Activity recognition
• Gait analysis
• Classification
• Segmentation etc.

4
Literature Survey

• A Cascade Of Feed-forward Classifier For Fast
  Pedestrian Detection, Chen, ACCV 2007
• No articulated bodies, holistic approach
• Detecting Pedestrians by Learning Shapelet
  Features, G Mori, CVPR 2007
• Stresses neck-head pattern, non-articulated
• Learning to parse images of articulated bodies,
  D Ramanan, NIPS 2006
• Part based, articulated poses

5
Datasets

• People dataset
• http//www.ics.uci.edu/dramanan/papers/parse/inde
  x.html
• Weizmann dataset
• http//www.wisdom.weizmann.ac.il/vision/SpaceTime
  Actions.html

6
Challenges

• Challenges for pose estimation
• Large variation in appearances
• Variety of clothing, skin color, shape
• Different illumination conditions
• Variation of posture
• Occlusions
• Challenges for activity analysis
• Variable action duration
• Variable actor speed and stride

7
Brief Overview

• Learning to parse images of articulated bodies,
  D Ramanan, NIPS 2006

8
Images of articulated bodies

• Pose estimation from a static image
• Iterative parsing process Sequentially learns
  better and better features tuned to a particular
  image, simultaneously estimates pose of the
  articulated object
• No skin or face detection
• No explicit foreground segmentation
• Employs part based model

9
Person Model
Tree structure of person model
Deformable model

• Model is parameterized by probability
  distributions capturing geometric arrangements of
  parts and local part appearances

10
Iterative algorithm

• Match an edge based deformable model to the image
  to obtain soft estimates of body part locations
• Use estimated body part position to build a rough
  region model for each body part and background
• Soft estimates of body from the new model are
  then used to build new region models

11
Part Model
First Row Conditional Random Field Model (Edge
based template) Second Row Conditional Random
Field Model (Region based template)
12
Figure1
Build a deformable pose model based on edges
13
Figure2
Use the initial parse from the previous example
to build a region model for each part
14
Figure3
15
Activity Analysis

• Extending Learning to parse images of
  articulated bodies to Videos

16
Proposed Method

• Obtain foreground information
• Estimate pose for each frame
• Regularize noisy pose estimates
• Use temporal collection of centroid and
  orientation of parts as raw features
• Convert raw features into equal length feature
  vectors
• Train a classifier by feeding these feature
  vectors

17
Propose Method
18
Pose Estimation

• Assumptions
• Video clips are captured by a static camera
• Extracted foregrounds are available
• The pose estimates are accurate
• Tools
• Dev Ramanans MATLAB code for pose
• Improved version for more iterations and color
  coded part boxes

19
Convergence of iterations
Normalized difference in consecutive parses
Number of iterations
20
Pose Regularization

• Videos have advantage of presence of previous and
  next frames
• Filter temporally to correct noisy locations
  across the frames
• Constrain resulting boxes to be rectangular

21
Feature Extraction

• Joint locations are difficult to calculate
• Obtain centre of box, and its orientation
• Stack centers and orientations temporally
• Obtain equal length feature vectors

22
Results Pose Estimation
23
Results Pose Estimation
24
Results Pose Estimation
25
Results Pose Estimation
26
Results Video eli_jump
27
Results Video eli_jack
28
Results Video eli_run
29
Results Video eli_walk
30
Future Work

• Improve pose estimation
• Generating pose based action features
• Training classifier for supervised learning
• Clustering for unsupervised learning

31
Thank You