Shape and Dynamics in Human Movement Analysis

1
Shape and Dynamics in Human Movement Analysis

• Ashok Veeraraghavan

2
Outline

• Motivation
• What do we want to do?
• Shape
• Shape based methods for recognition
• Dynamics based methods for recognition
• Results

3
Motivation

• Human Perception
• Shape or Dynamics (or is it Both??)

4
Laurel and Hardy
5
Laurel ??? Hardy ???
6
Who is this ? ? ?
7
Who is this ? ? ?
8
Introduction

• Psychophysics work indicates that dynamics is
  important for recognition in humans.
• Johansson Light Display Moving dots
• Murray(1964) 24 gait components
• Cutting FamiliarityStatic Vs Dynamic
• Kozlowski dynamics speed, bounciness,
  rhythm.
• Cutting Dynamic Invariant
• Gender Discrimination

9
Prior Work

• Image Correlation.
• Silhoutte Based Nearest Neighbour.
• Dynamic Time Warping
• Hidden Markov Model
• Model parts of human body and extract gait
  signature.(eg., Thigh)

10

• Most gait recognition algorithms are shape based
  !
• Relative importance of shape and dynamics

11
Definition of Shape

• Shape is all the geometric information that
  remains when location, scale and rotational
  effects are filtered out from the object.
• Kendalls Statistical Shape Theory used for the
  characterization of shape.
• Pre-shape accounts for location and scale
  invariance alone.

12
Pre-Shape

• k landmark points (complex vector)
• Translational Invariance Subtract mean
• Scale Invariance Normalize the scale

13
Feature Extraction

• Silhoutte
• Landmarks
• Centered Landmarks
• Pre-shape vector

14
Distance between shapes

• Shape lies on a spherical manifold.
• Shape distance must incorporate the non-Euclidean
  nature of the shape space.
• 1)Full Procrustes distance.
• 2)Partial Procrustes distance.
• 3)Procrustes distance.

15
Full Procrustes Distance

• Procrustes Fit
• Full Procrustes DistanceMinimum Procrustes Fit.

16
Other shape distances

• Partial procrustes distance
• Procrustes distance (?) distance on the Great
  circle.

17
Tangent Space

• Linearization of spherical shape space around a
  particular pole.
• The Procrustes mean shape is usually chosen as
  the pole.
• If the shapes in the data are very close to each
  other then Euclidean distance in tangent space
  approximates shape distances.

18
Shape based methods for Recognition

• Stance Correlation.
• Dynamic time warping in shape space.
• Hidden Markov Model in shape space.

19
Stance Correlation

• Exemplars for 6 stances for each individual.
• The correlation between exemplars is used as the
  matching criterion.
• Performance comparable to Baseline.

20
Dynamic time warping in shape space .

• Enforce end-point constraint.
• Obtain best warping path.
• Cumulative error is computed using the shape
  distances described.
• Performance is better than baseline.

21
Hidden Markov Model in shape-space

• Exemplars are regarded as states.
• HMM built for each person in the gallery.
• Identity established by maximizing the
  probability that the observation came from the
  model in the gallery.
• Performance is better than baseline and
  comparable to DTW.

22
Dynamical Models

• Stance based AR model.
• Linear Dynamical System

23
Stance based AR model

• Video sequence is clustered into 3 distinct
  stances. Each frame is identified as belonging to
  one of these three stances.
• Parameters of an AR model learnt for each stance.
• Model parameters used for recognition.
• Performance is below baseline.

24
Linear Dynamical System(ARMA)

• Parameters (A,C) of a dynamical system learnt for
  each individual.
• Distance between models used as score for
  recognition.

25
Learning the model
26
Distance between models

• Subspace angles (?i i1,2.n).
• Martin,Gap and Frobenius Distance.

27
Results on USF database

• Gallery 71 people.
• Probe varies from Gallery in view, shoe and
  surface.
• CMS curves shown.

28
Sample Sequences
29
Stance Correlation.
30
Dynamic time warping
31
Comparison of DTW with Baseline
32
Stance based AR model
33
Linear Dynamical system
34
Comparison of various methods on the USF database.
35
Results on the CMU database

• Gallery consists of 25 people.
• 3 different activities studied Slow walk, Fast
  walk and walk with ball.
• Recognition performed within and across
  activities.

36
Percentage of Recognition using Stance
correlation.
37
Similarity Matrix using Linear Dynamical
system(ARMA)
38
Percentage of Recognition using Linear Dynamical
system
39
Mocap Data
40
Mocap (Activity Recognition)
41
Mocap (Activity using ARMA)
42
Conclusions

• Shape is more important for recognition than
  dynamics. Shape also provides for speed change
  invariance.
• Dynamics can help to improve performance of shape
  based methods.
• Activity Recognition Dynamics plays a important
  role.
• Dynamical models like ARMA can perform
  recognition across activities.

43
Thank You.