Visual Tracking of High DOF Articulated Structures

1
Visual Tracking of High DOF Articulated Structures
Chris DeCoro Presentation of an article by Rehg,
et. Al.
2
Talk outline

• Introduction
• Motivation
• Intended Applications
• State Model
• Feature Measurement
• State Estimation
• Conclusion

3
Motivation and Background

• Track the configuration of the hand.
• Generalizes to arbitrary kinematic skeletons
• Traditional approaches (feature tracking)
• Self-occlusion
• Fail due to high degrees of freedom
• New Approach
• Limit the degrees of freedom (impose skeleton)

4
Intended Applications

• Why do we care?
• Virtual Trackball
• Use the a trackball without a ball present
• Rotate hand to rotate ball
• Simply move finger to indicate clicks
• Human Computer Interaction
• Computer should be able to perceive user, and
  respond accordingly
• Should not require users to wear extra markers,
  mechanical devices

5
Talk outline

• Introduction
• State Model
• Kinematic Denavit-Hartenburg Model
• Features Links Central-axis line
• Feature Measurement
• State Estimation
• Conclusion

6
Kinematics Denavit-Hartenburg

• Palm
• Flat plane (shown to be reasonable)
• Quaternion rotation (4 DOF)
• Offset vector (3 DOF)
• Fingers
• Fixed 2d position in finger plane
• Rigid bone lengths for each joint
• Base joint rotates in 2 angles
• Other joints rotate in 1 angle
• 4 DOF total
• Thumb
• First and Second joint can rotate in 2 angles
• 5 DOF
• Overall 24 DOF

7
Feature Models Cylinders/Links

• Fingers/Thumb
• Conceptualized as a cylinder
• Hemispheric Cap on the last cylinder
• Link Tracking
• Occlusion boundaries of each cylindrical link
  form pair of lines
• Central-axis line is used to ignore radius of
  cylinder, relative slopes
• Find a, b, p ax by p 0 , in image plane
• Center of tip hemisphere is also tracked (radius
  not important)
• Find x, y center of sphere in image plane

8
Talk outline

• Introduction
• State Model
• Feature Measurement
• Link and Tip features
• Tracking
• State Estimation
• Conclusion

9
Link and Tip Features

• High Hz results in local tracking
• Use local trackers for image features
• Projections of spatial hand geometry into image
  plane
• Link features
• Represented as T
• Stem is projection of cylinder axis
• Tip features
• Shown by the

10
Tracking Features

• Central axis is positioned in image
• Determined from last frame
• Determine current features
• Search gradient on lines perpendicular to central
  axis
• Peak in gradient indicates edge of feature
  (occlusion boundary)
• High sampling rate allows us to treat as a local
  problem
• Will be used with kinematic model to determine
  pose

11
Talk outline

• Introduction
• State Model
• Feature Measurement
• State Estimation
• Link and Tip Image Alignment
• Estimation Algorithm
• Conclusion

12
Link and Tip Image Alignment

• Residual Model
• Measure difference between actual and predicted
  positions
• Applied for both links and tips
• Tip Residual
• Euclidean dist. between predicted (ci) and
  measured (ti) positions
• 2D Vector in the image plane
• vi(q) ci(q) ti
• Link Residual
• Scalar deviation of projected cylinder axis from
  measured feature line
• li(q) mtpi(q) - ?
• m a b 0t
• Note that we use entire line (hard to measure
  beginning and end)
• Concatenate to residual vector R(q)

13
Estimation Algorithm

• Minimize residual error
• H(q) 0.5 R(q) 2
• Inherently non-linear due to trigonometric
  functions in kinematic chain (rotations of links)
• Use Levenburg-Marquardt Nonlinear Least Squares
  Method
• Let R(qi) be the residual vector for image j
• Apply the LM state update equation
• qj1 qj JtjJj S-1JtjRj
• Compute Jacobian Jj for Residual Rj evaluated at
  qj
• Projection of kinematic Jacobian for points on
  link in direction of feature normal
• Computation for each i is one row of matrix

14
Estimation Algorithm (contd)

• Compute Jacobian for rotational joint
• This includes the base of fingers and 2 joints of
  thumb
• Compute Jacobian for plane DOF
• S is a constant diagonal conditioning matrix

15
Talk outline

• Introduction
• State Model
• Feature Measurement
• State Estimation
• Conclusion

16
Conclusion

• We have achieved accurate tracking of the hand