Hierarchical Indexing Structure for 3D Human Motions

1
Hierarchical Indexing Structure for 3D Human
Motions
Gaurav N. Pradhan, B. Prabhakaran Department of
Computer Science University of Texas at Dallas,
Richardson, TX 75083 gaurav, praba _at_
utdallas.edu
2
Introduction

• Content-based retrieval of 3D human motion
  capture data has significant impact in different
  fields such as physical medicine, rehabilitation,
  and animation.
• Several scientific applications, especially those
  in medical and security field, need to analyze
  and quantify the complex human body motions.

3
3D Motion Capture
4
Applications

• Gait Analysis
• Several orthopedic applications,
• Joint Mechanics
• Prosthetic Designs
• Sports Medicines
• Physical medicines and Rehabilitation
• Biomechanics
• Physiological Applications

5
Objective

• Our main objective is to find similar 3D human
  motions by constructing the indexing structure
  which supports queries on sub-body motions in
  addition to whole-body motions.
• We focus on content-based retrieval for the
  sub-body queries such as
• Find similar shoulder motions,
• Find similar leg motions,
• Find similar walking human motion.

6
Challenges

• 3D motions are multi-dimensional, multi-attribute
  and co-related in nature associated segments of
  one sub-body (e.g. hand) must be processed always
  together along every dimension.
• Human motions exhibit huge variations in local
  speed for similar motions as well as in
  directionality.
• It is difficult to come up with a similarity
  measure/metric on 3D human motions.
• The result/ranking of similar patterns to
  sub-body query is influenced by movements of
  other sub-body parts.

7
Feature Components
3D Trajectories of Foot Movement while walking
8
Identifying Hierarchical Structure of Human Body
for Indexing

• Each sub-index structure derived from generic
  index structure.

9
Problems in Constructing the index trees

• Non-metric
• DIST(M1,Q) lt DIST(M2,Q) does not imply M1 is
  more similar to Q than M2.
• Difficult to strictly rank similar motions.

10
Space Partitioning Problem

• Definition
• Due to different variations in performing similar
  motions, the corresponding mapped points may fall
  into different groups after clustering causing
  false dismissals in resulting output of similar
  motions for the query.
• Solution
• We capture the uncertainty of differences in
  similar motions for a joint in different feature
  dimension using standard deviation.

11
Space Partitioning Problem

• In a database of M motions, we have E sets of
  pre-determined similar motions. Let simDevc be
  the standard deviation of the differences between
  similar motions for the cth feature component.
• Using simDevc ,we get the threshold dc for the
  cth dimension of the feature space as follows,
• simTolerancec dc e simDevc
• e is an input parameter which varies in the range
  of 0.2-1
• simTolerancec (dc ) is ultimately a threshold
  distance used to retrieve the set of motions that
  lie within this distance along the cth dimension
  from the query motions feature point.

12
Effect of feature-space partitioning
The grouping of mapped feature points in feature
space for body joints and single-path query
resolution.
13
Dimension 2
Before G1 A, B G2 D, C
After G1 A, B, C, E G2 D, C, B
d1 d1 d2 d2
G2
d2 d2
d1
F
d1 d1
d1
D
C
E
d2
d2
B
d2
A
d1 d1
d1
d2 d2
G1
Dimension 1
d1
14
Dimension 2
0 d1 lt d1 0 d2 lt d2
G2
2d1
D
G1
C
2d2
d2
Q
Dimension 1
d1
15
Construction for hierarchical index tree for leg
segments
16
Tibia Joint
Foot Joint
Toe Joint
Hierarchical Index Tree for the Leg
17
Extraction of feature vectors from individual
joint matrices
18
(No Transcript)
19
Tibia Joint
Fsx Fsy Fsz
Fex Fey Fez

Fwx Fwy Fwz

Foot Joint
Toe Joint
20
Grouping inside Node

• Hierarchical Self-Organizing Clustering Approach
• Heterogeneity (Ht) is a measure to calculate the
  distribution of
• the points in a given group.

xj mapped point C centroid of the node D
Number of patterns
21
Grouping inside Node contd.

• Scattered points in group give high heterogeneity
  value and closely distributed points give low
  heterogeneity value.
• The threshold Th is determined by product of
  this measure and heterogeneity scaling factor a.
• Th Ht a

22
Vertical growing

• A node is spliced into two groups if
  heterogeneity is greater than a defined
  threshold.
• IF Ht(Ni) gt Th THEN Split node Ni into two
  groups.

Horizontal growing
Finds the proper number of groups in a node
using cluster validation criteria.
23
Illustration of hierarchical, self-organizing
clustering in a node N1.
24
Performance Analysis

• Pruning non-promising motions for a query.
• Pruning Efficiency is defined as þ
• þ (Npr / Nir) x 100
• Npr Number of irrelevant motions pruned by the
  index tree.
• Nir Total number of irrelevant motions in the
  database.

25
Experimental set-up and Platform

• Our test bed consists of 1000 different human
  motions, performed by different subjects,
  resulting into 1000 data matrices of 18(x 3)
  attributes for each motion.
• We tested the average computational time required
  for 1000 queries to traverse through the index
  tree and to prune the irrelevant motions.
• All experiments are performed on a computer with
  3.00GHz Linux Intel(R) Xeon(TM) CPU.

26
Pruning Efficiency
27
Computational Time
28
Recall
29
Precision
30
Performance Comparisons

• MUSE seems to be the most suitable indexing
  structure published so far for multi-attribute
  motion data.
• By querying 1000 3D motions, the pruning
  efficiency achieved was 5.5, as compared to 97
  of our index tree structure.
• The average computational time required per query
  was 0.3 seconds. In our case, for the same set of
  queries, the average computational time per query
  is 15 µsec.
• The reason MUSE has a poor performance on 3D
  motions is that the lower bound defined by MUSE
  for pruning irrelevant motions is not tight
  enough for 3D human motions.

31
Performance Comparisons

• Keogh et. al. introduced a technique for indexing
  time series with invariance to uniform or global
  scaling, based on bounding envelopes.
• Achieved pruning efficiency 90-95 .
• Limitation
• Query in form of single time series and the
  candidates as uni-variate time subsequences with
  fixed length.
• Apply searching algorithm sequentially to every
  degree of freedom and at the end evaluate the
  best match by measuring the weighted sum of
  Euclidean distance of the sequences, which is
  time consuming.
• Fastest average running time per query achieved
  in 10-20msec which is significantly lower than
  our approach, typically around 15 microsec.

32
Performance Comparisons

• Liu et. al. indexed the motion database by
  representing motion sequences by their residence
  in distinct subsets of clusters and transit among
  the clusters with distinctive trajectories.
• The authors claim that response time is
  interactive and results are accurate but the
  quantitative performance measures have not been
  presented.

33
Conclusions

• This work considers content-based motion querying
  on a repository of multi-attribute 3D motion
  capture data.
• We proposed a composite index structure that maps
  on to the hierarchical segment structure of the
  human body comprising five independent index
  trees.
• In each index trees, each level is assigned to
  one segment feature vector and similar feature
  points inside all nodes are grouped together to
  increase the pruning power of the index tree.
• Upto 96-97 irrelevant motions can be pruned for
  any kind of motion query while retrieving all
  similar motions.
• One traversal of the index structure through all
  index trees takes on an average 15 µsec.

34
References

• R. Agrawal, C. Faloutsos, and A. Swami,
  Efficient similarity search in sequence
  databases, In Proc. 4th Conference on
  Foundations of Data Organization and Algorithms,
  pages 69--84, October 1993.
• S.P. Chao, C.Y. Chiu, J.H. Chao, Y.C. Ruan, and
  S.N. Yang, Motion retrieval and synthesis based
  on posture features indexing, In Proc. Fifth
  International Conference Computational
  Intelligence and Multimedia Applications, pages
  266--271, September 2003.
• E. Keogh, T. Palpanas, V. B. Zordan, D.
  Gunopulos, and M. Cardle, Indexing large
  human-motion databases, In Proc. 30th VLDB
  Conference, pages 780--791, Toronto, Canada,
  2004.
• C. Li and B. Prabhakaran., A similarity measure
  for motion stream segmentation and recognition",
  In Proc. MDM/KDD, The sixth International
  Workshop on Multimedia Data Mining, Chicago, IL
  USA, August 2005.
• F. Liu, Y. Zhuang, F. Wu, and Y. Pan., 3D motion
  retrieval with motion index tree, Computer
  Vision and Image Understanding, 92265--284, June
  2003.
• F. Luo, L. Khan, F. Bastani, I.L. Yen, and J.
  Zhou, A dynamically growing self-organizing tree
  (dgsot) for hierarchical clustering gene
  expression profiles., Bioinformatics,
  202605--2617, May 2004.
• M. Muller, T. Roder, and M. Clausen, Efficient
  content-based retrieval of motion capture data,
  ACM Transactions on Graphics (TOG), 24677--685,
  2005.
• K. Yang and C. Shahabi, Multilevel
  distance-based index structure for multivariate
  time series, In TIME, Burlington, Vermont, USA,
  2005. IEEE Computer Society.

35
Questions ????