Perceptually%20Consistent%20Example-based%20Human%20Motion%20Retrieval

1
Perceptually Consistent Example-based Human
Motion Retrieval

• Zhigang Deng, Qin Gu, Qing Li
• University of Houston

2
Introduction

• Popularization of human motion capture data in
  animation and gaming applications
• Efficient retrieval of similar motions from a
  large data repository
• Fundamental basis for many motion data based
  applications

e.g. CMU motion capture library.
http//mocap.cs.cmu.edu. 2605 trials in 6
categories and 23 subcategories.
3
Related Work Motion retrieval

• Transform original high-dimensional human motion
  data to a reduced representation Agrawal et al.
  1993 Faloutsos et al. 1994 Chan and Fu 1999
  Liu et al. 2003 Chiu et al. 2004 Baciu 2006
  Lin 2006 .
• Match webs Kovar and Gleicher 2004
• Describe potential subsequence matches between
  any pair of motion sequences.
• Semantics-based motion retrieval Muller et al.
  2005 Muller and Roder 2006
• Users provide a query motion as a set of
  time-varying geometric feature relationships.

4
Our Approach Pipeline
Motion Data Preprocessing

• Human hierarchy construction
• Motion segmentation and normalization
• Motion pattern detection and indexing
• Hierarchical pattern matching
• Search result ranking

Runtime Motion Query
5
Data Preprocessing - Motion Hierarchy Construction

• Decompose human motion into a hierarchical
  structure Gu et al. 08
• Local control granularity
• Correlations among different human parts are
  embedded in different layers
• 4 layers, 18 parts are used in this work.

6
Data Preprocessing - Motion Segmentation and
Normalization

• Existing human motion segmentation techniques
• Angular acceleration Zhao 01, Fod et al. 02, Kim
  et al.03, SVM classifier Li et al. 07,
  weighted sum of marker velocities Gu et al. 08,
  PCA/PPCA Barbic et al.04.
• Probabilistic PCA Barbic et al. 04 is used to
  segment motion into short motion segments for
  each body part in the hierarchy.

Parts Head LHand LArm RArm
Ave Frm Var 18.32 2.32 8.43 6.43 11.39 6.54 12.75 7.34
Parts Torso RLeg LFoot RFoot
Ave Frm Var 13.43 5.65 11.24 5.12 6.75 5.35 6.05 5.88
Average Frame Information of segments
7
Data Preprocessing - Clustering

• Motion Pattern for each body part
• A representative motion segment for a node
  (i.e.,a body part) in the constructed human
  hierarchy
• Normalization of motion segments
• Adaptive K-Means clustering
• Increase K when the clustering error metric is
  larger than a threshold
• Resulting data structures
• (1) Motion Pattern Library, (2) Pattern Index
  Lists, (3) Pattern Dissimilarity Maps.

8
Review of Motion Preprocessing
9
Runtime Motion Query

• Query motion transformation
• Map the query motion into a motion pattern index
  list for each hierarchy node
• Fast (no clustering, just database matching)
• Motion similarity score computing
• Local motion similarity between two index lists
• Extended Knuth-Morris-Pratt (KMP) string matching
  algorithm Knuth et al. 77
• Global motion similarity computing and ranking
• Hierarchical propagation

10
Local Motion Similarity

• Similarity between two pattern index lists
• Different length of index lists
• Matching of two integer lists
• Extended KMP String match algorithm
• Introducing Quasi-Match based on the
  pre-constructed pattern dissimilarity maps
• Large numbers of different motion segments
• Distance is less than a threshold
• Update matching score
• If the number of consecutive quasi-matches is
  larger than a threshold, otherwise decrease..
• Score normalization based on the length of index
  lists

11
Global Motion Similarity

• Hierarchical Score Propagation
• High local motion similarity does not mean global
  motion similarity
• Nodes in the upper levels encode more global
  motion information
• From bottom to top
• Ranking of the final scores at the root node

12
Review of Runtime Motion Retrieval
13
Results and Evaluation

• Time and Storage
• Search Accuracy
• Search Quality
• Perceptual Consistency Experiment

14
Results and Evaluations Time and Storage

• We tested our method on four datasets with
  different sizes
• The test computer with a Intel Duo Core 2GHz CPU
  and 2GB memory.
• The average duration of used query motions is 10
  seconds.

56MB, 170 motions,68,293 frames 456MB, 396
motions, 556,097 frames 976MB, 542 motions,
1,190,243 frames 1452MB, 941 motions, 1,770,731
frames
15
Results and Evaluations Search Accuracy

• Accuracy evaluation scheme Kovar and Gleicher
  04
• Two different types of datasets single-type
  motion datasets (pre-labeled dataset with the
  same semantic category, walking) Ground truth,
  mixed motion dataset (unlabeled, mixed of various
  types).
• True-positive accuracy ratio is defined top N
  (20) results from mixed motion datasets are in
  the correct/expected single-type motion dataset.
• 56M test dataset 170 sequences, 68,293 frames,
  five categories walking, running, jumping,
  kicking, basket-playing.

16
Results and Evaluation Comparative User Studies

• Compare our approach with match-webs approach
  Kovar and Gleicher 04, piecewise linear space
  Liu et al. 05, weighted PCA Forbes and Fiume
  05.
• Semantic-based motion retrieval Muller et al.
  05 was not chosen, because of significant
  differences in input requirements.
• Two usability questions
• (a) Perceptual Consistency Retrieved results
  (motions) are ranked in a perceptually consistent
  order?
• (b) Search Quality Motion similarities of
  retrieved results?

17
Results and Evaluation Comparative User Studies

• Perceptual-consistency
• Computer algorithms rank motions in a certain
  order, C.
• Humans rank these (the same) motions in another
  order, H.
• Relationship/consistency between C and H?
• Study Experiments
• 3 query motions (walking, running,
  basketball-playing),Top-ranked N (6) results for
  query, 4 approaches, total 72 364 results.
• Side-by-side comparison and user rating (one is a
  searched motion, the other is the query motion),
  in a random order.
• Rating is from 1 (completely different) to 10
  (identical).
• 24 experiment participants

18
Results and Evaluation Comparative User Studies

• Quality of searched motions
• Compute average similar ratings and standard
  deviation
• Higher the average similar rating is, the better
  quality of search it achieves.

19
Results and Evaluation Comparative User Studies

• Perceptual-Consistency
• Plot human-rankings vs computer-rankings in a 2D
  space.
• Ideal consistency is shown as a straight line.
• Canonical Correlation Analysis
• Scale-invariant optimum linear framework

Walking
Running
CCA Coefficient results
Basketball-playing
20
Review of User Studies
21
Conclusions

• An efficient, example-based human motion
  retrieval technique
• Major distinctions of our approach
• Efficiency
• Linear to the size of query motion and database
  size
• Flexible search query
• A human motion subsequence, or a hybrid of
  multiple motion sequences
• Perceptually consistent search outcomes
• Comparative user studies to find out the
  correlations between the result-ranking by
  computer algorithms and the result-ranking by
  humans

22
Discussion and Limitations

• Current approach does not consider the
  path/motion trajectory of the root of the human
  in the retrieval algorithm. The search results
  may enclose different paths/trajectories.
• Current approach can only search for
  single-character motion sequences.

23
Future Work

• A number of empirical parameters of current
  approach may critically affect the search
  accuracy and outcomes.
• Establish quantitative correlations between
  parameter setting and search accuracy and
  outcomes.
• Graphics hardware accelerated, motion query
  processing.

24

• Thank You!
• Questions?