Recognition of Human Gaits

1
Recognition of Human Gaits
Alessandro Chiuso Dipartimento di Elettronica e
Informatica Universita di Padova Stefano
Soatto Computer Science UCLA

• Alessandro Bissacco
• Computer Science
• UCLA
• Yi Ma
• Electrical and Computer Engineering
• University of Illinois at Urbana-Champaign

Presented by Peter Schwer
2
Summary

• Problem Description
• Related Work
• Overview of Gait Recognition Process
• Tracking
• Learning the Model
• Recognition
• Results

3
The Human Gait

• Observable from great distances
• Discernable independent of skew (relative
  direction of subjects gait)
• Communicates
• Identity
• Mood

4
Good Recognition Traits

• Invariant to
• Photometric Factors
• Illumination, clothing
• Geometric Factors
• Camera Position
• Length of subjects limbs
• Relative direction of gait

5
Good Recognition Traits

• Capable of discerning
• Different types of gaits
• Running
• Walking
• Skipping, etc.
• Mood
• Not addressed, but dynamical models can be
  extended to this area.

6
Related Work
University of Southampton Carnegie Mellon
University (CMU) Massachusetts Insitute of
Technology (MIT) Georgia Tech (GATech)University
of MarylandUniversity of South Florida

• Analyzing gait with spatiotemporal surfaces
  (1994) Sourabh A. Niyogi, Edward H. Adelson

7
The Recognition Problem

• The Sub-problems
• Tracking
• Learning the model
• Recognizing

8
Tracking

• The emphasis of our work is not tracking.
• Images ? Skeletons
• Results
• y(t), t 1 t
• (Sequence of joint positions)

9
Learning the Model
10
The Right Model

• Ideally,
• Unique model for each dataset
• Method for calculating similarity (distance)
  between models
• Catch
• The set of dynamical models in canonical form is
  not a linear space (even if the model itself is
  linear!)
• Distances between dynamical models are not
  trivially computed.
• Data, as usual, is inherently noisy.

11
Creating the Dynamical Model

• Input y(t) Sequence of Joint Positions
• What are some of the statistical properties of
  this sequence of joint positions?
• Stochastic Stationary Second-order process
• How is noise distributed in the data?
• We use a Gauss-Markov Model as a representative
  of each class.

12
The ARMA Model
x(t1) Ax(t) v(t), v(t) N(0,Q) x(0)
x0 y(t) C(x(t) w(t) w(t) N(0,R)
13
The ARMA Model

• Other Applications
• Speech Recognition
• Tracking
• Geophysics Forecasting and Simulation
• Pattern Recognition
• Dynamic Texture Recognition

14
The ARMA model

• x(t) is a process describing the state at time
  t (vector)
• y(t) is the measured joint positions (vector)
• Q,R are input and output noise covariances
  (matrices)
• x(t1) Ax(t) v(t), v(t) N(0,Q) x(0) x0
• y(t) C(x(t) w(t) w(t) N(0,R)
• (An n, Bm n)

PROBLEM Non-uniqueness of A, C, Q, R, S
15
Innovation/Canonical Representation

• Identify stochastic subspace models without
  forming the covariance matrix.
• Principal Angles and Directions
• Source
• P. Van Overschee and B. De Moor
• Subspace Algorithms for the stochastic
  identification problem. Automatica, 29649-660,
  1993.

16
A Canonical Representation

• x(t) is a process describing the state (vector)
• y(t) is the measured joint positions (vector)
• Q,R are input and output noise covariances
  (matrices)
• x(t1) Ax(t) v(t), v(t) N(0,Q) x(0) x0
• y(t) C(x(t) w(t) w(t) N(0,R)
• From Overschee and De Moor we have A,C,Q,R
• A and C describe the gait.

17
Recognizing
18
Our Dynamic Model

• A, C, Q, R provided
• What are they?
• How might they be useful?

19
How to Describe Gait

• Tracking produces a waveform that describes a
  series of joint positions/angles

What is the best way to compare these to
waveforms?
Gait 1
Gait 2
20
Distance Between Models

• Subspace/principal angles
• Start with O(M)- infinite observability matrix

21
Form O(m)

• O(m) is a subspace
• We want to compare the observability subspaces of
  two models M1 and M2.
• Form O(m) for each model

• Call the results H1 and H2, respectively.

22
Form Orthonormal Matrices

• Gram-Schmidt Orthogonalization on H1 and H2
• Described http//lagrange.la.asu.edu/VirtualClass
  /Algebra/GramSchm.html
• Applet http//www.mste.uiuc.edu/exner/ncsa/orthog
  onal/simulation
• Call the result orthonormal matrices QH1 and QH2

23
Calculate Principal Angles

• Compute n-ordered Singular Values of QTH1 QH2
• n-ordered singular values can be denoted
  cos2(?1), , cos2(?n)
• Principal Angles between the subspaces H1 and H2
  are denoted by the n-tuple
• (?1 , ?2 ,, ?n ) sorted in descending order.

24
Calculate the Distance

• First measure is not necessarily non-negative
  (less fit)
• Second Measure is the largest principal angle
  (more fit)

• The Martin and Finsler Distance, respectively.

25
Or

• Just use their matlab code

function theta subspace_angles(A1,K1,C1,A2,K2,C2
) n size(A1,1) m size(C1,1) A A1
zeros(n,3n) zeros(n) A2-K2C2
zeros(n,2n) zeros(n,2n) A2 zeros(n)
zeros(n,3n) A1-K1C1 C C1 -C2 C2 -C1 Q
dlyap(A,CC) E eig(zeros(2n)
pinv(Q(12n,12n))Q(12n,2n14n) pinv(Q(2
n14n,2n14n))Q(2n14n,12n) zeros(2n)
) E max(-ones(size(E)),E) E
min(ones(size(E)),E) theta acos(E(12n))
26
Results
27
Example Gaits
28
Name that Gait
Pensive and determined.
29
Obligatory Gait Video 2
Not in a rush.
30
Obligatory Gait Video 3
Sad walking.
31
Obligatory Gait Video 4
There is no stopping me.
32
Results
The pairwise distance between each sequence in
the dataset is displayed in this plot. Each
row/column of a matrix represents a sequence, and
sequences corresponding to similar gaits are
grouped in block rows/columns. Dark indicates a
small distance, light a large distance. The
minimum distance is of course along the diagonal,
and for each column the next closest sequence is
indicated by a circle, while the second nearest
is indicated by a cross.
33
Results

• How accurate are the results?
• How does this compare with human ability?

34
Questions?
35
From Discussion Board

• Diem How are the ARMA model and Kalman filters
  related?
• Mei-fang Is there any method to find the best
  size training set, and the best k for classifying
  the new sample?
• Matt Clothier What is the advantage of the
  Martin distance over the Finsler distance?
• Neil and Mike What do they mean by Transient
  Actions and why does the previous assumption not
  hold for them?