Real-Time Enveloping with Rotational Regression

1
Real-Time Enveloping with Rotational Regression
Robert WangKari PulliJovan Popovic
2
Enveloped (skinned) characters are pervasive.
skeleton
mesh

• Skeletons are often used to control meshes.

3
Physically based modeling provides realistic
deformations.

• Realistic deformations
• Finite-element based Teran et al. 2005
• Anatomy based Scheepers et al. 1997
• Elastically deformable Capell et al. 2002,
  2005
• Used in movie production
• Off-the-shelf commercial tools
• Slow evaluation

Teran et al. 2005
Absolute Character Tools 1.6
4
We learn a fast model from exported examples.
Exported Examples (skeleton-mesh pairs)
Black Box Simulation
Fast Model
Our method
5
Artists can still use existing modeling tools or
scanned data.
Exported Examples (skeleton-mesh pairs)
3-D Scan Data
Fast Model
6
This is analogous to mesh simplification.

• Faster to render
• Optimized for interactive applications

• Higher quality
• Used in movie production

7
How do we map a skeleton to a mesh?
What parameters should we learn? How to model
muscle deformations for fast evaluation?
8
Linear blend skinning linearly maps joint
rotations to vertex positions.

• Most popular enveloping technique for games
• Coarse modeling parameters (but very simple)
• Not very expressive (but very fast)

Figure from Wang and Phillips 2002
9
Linear blend skinning has many names.

• Also known as,
• Single-Weight Enveloping
• Skeletal Subspace Deformation (SSD)
• Or just, Skinning
• We will use Linear Blend Skinning or SSD.

10
The two steps of our work are deformation
gradients prediction and mesh reconstruction.
Deformation gradients prediction
(Rotational Regression)

• Mesh reconstruction

11
We present a replacement for linear blend
skinning.

• Coarse modeling parameters.
• Cant handle certain types of deformations.
• Fast

• Lets you use your existing modeling tool.
• Good for muscle bulges.
• Fast

Whenever you have an existing model, you should
use our technique instead of linear blend
skinning.
12
Our model is inspired by the behavior of a
flexing bicep.
Surface rotation
Bone rotation

• Rigid components move with the bone rotation
• Other surfaces rotate in the opposite direction

13
Angle is scaled by u. Axis is offset by rotation
W.
target rotation (surface)
source rotation (bone)
14
We map a sequence of bone rotations to a sequence
of surface rotations.


target rotation sequence (surface)
source rotation sequence (bone)
15
We fit parameters u and W by regression.
Surface rotations
u,W
Best-fit parameters
Skeleton rotations
16
Rotational regression is good at capturing muscle
bulges.
17
Mesh reconstruction stitches deformation
gradients together.

• Deformation gradients prediction

• Mesh reconstruction

18
Mesh reconstruction solved with least-squares.
deformation gradients
vertex positions
Least squares

• Least-squares problem equivalent to linear
  system.
• Computation is matrix-multiplication.

19
Near-rigid vertices help eliminate low-frequency
errors at extremities.

• Low-frequency errors can accumulate at
  extremities of mesh
• We fix a set of near-rigid vertices to their SSD
  predictions
• Still a least squares problem

20
We build upon existing mesh reconstruction work.

• Mesh IK Sumner et al. 2005, Der et al. 2006
• SCAPE Anguelov et al. 2005
• Similar formulation, faster evaluation.

Anguelov et al. 2005
21
Heres a review of what weve covered.
Rotational Regression
Deformation Gradients Prediction
Least-squares problem
Mesh Reconstruction
22
Model reduction lowers the dimensionality of
problem.
C
?
C
?
Dl(q)
Dk(q)
SSD

• Large multiplication on CPU

• Smaller multiplications on GPU

23
Model reduction uses greedy clustering.

• Vertices clustered into proxy-bones.
• Per-triangle deformation gradients clustered into
  key deformation gradients.

24
Mesh reconstruction reduced to the following
matrix-multiplications.
C
?

• All on GPU
• Computation in fragment program

Dl(q)
key deformation gradients
Map from key deformation gradients to
proxy-bones
SSD weights
25
Skinning Mesh Animations is a an alternative
approach to model reduction.

• The method from Skinning Mesh Animations uses
  mean-shift clustering and is more robust to
  errors. James and Twigg 2005
• Our method minimizes vertex error and is faster

26
Deformation gradients prediction is now on key
deformation sequences.

• Fewer deformation gradient sequences to predict
  rotational regression.

27
Mesh reconstruction step reduced to
matrix-multiplications on GPU.

• Smaller matrix-multiplications
• Supported on graphics hardware

C
?
Dl(q)
28
Our Technical Contributions
Rotational Regression
Accurate and GPU-Ready Poisson Reconstruction
Model Reduction
29
Results
30
Results
31
Results
32
Our work approximates the training examples
better than SSD and also generalizes well.
33
Our model is suitable for interactive techniques.

• Evaluation speed within a factor of two of SSD
• Off-line training preprocess is usually less than
  half an hour

34
How does our work fit with previous work?
35
Our work is complementary to displacement
correcting techniques.
Figure from Kry et al. 2001

• Previous work provide corrective displacements.
• Pose space deformation Lewis et al. 2000,
• Shape by example Sloan et al. 2001,
• Eigenskin Kry et al. 2002
• Our work provides better approximation of
  rotations.
• Our work complements approaches that build upon
  SSD.

36
Displacement correcting approaches fail when SSD
is very wrong.
37
Our work builds upon previous ideas on enriching
the SSD model.

• Multi-weight enveloping Wang and Phillips 2002
• Additional joints Mohr and Gleicher 2003
• Our technique has more parameters than SSD and
  generalizes the additional-bones model.

38
A more expressive model is useful here.
39
Our model doesnt do a perfect job.

• Not perfect reproduction
• Inspired by muscle bulging and twisting. Other
  behaviors empirically validated.
• Displacement correcting technique can be used for
  exact reproduction of examples.

40
Conclusion Fast and accurate enveloping.

• Fast evaluation of physical simulations through
  learning.
• Within a factor of two of SSD on most models
• Accurate reproduction of details
• Better approximation and generalization
• Complementary to previous work
• A replacement for linear blend skinning

41
Acknowledgements

• Funding
• Nokia Research Center
• National Science Foundation
• Pixar Animation Studios
• Hardware/Software
• NVIDIA Corporation
• Autodesk

• Data
• Drago Anguelov
• Joel Anderson
• Michael Comet, Comet Digital, LLC
• Mark Snoswell, CG Character
• Joseph Teran, Ron Fedkiw
• MIT Graphics Group
• Ilya Baran
• Jiawen Chen
• Sylvain Paris

42
Questions?

• Thank you for coming to our talk!

43
Learning tasks trade expressiveness and
simplicity.
More Expressive Captures more types of
deformation.
Simpler Easier to fit Fewer training examples
needed. Less likely to overfit.
Rotational Regression
44
Linear blend skinning (SSD) is a rough and ready
map from joint rotation matrices to vertex
positions.

• Most popular enveloping technique for games
• Coarse modeling parameters (but very simple)
• Not expressive enough (but very fast)

desired deformation
SSD deformation
45
Model Reduction

• True optimization not as tractable
• We approximate it with a greedy algorithm
  inspired by mesh simplification.

difficult to solve simultaneously
discrete optimization
46
Our work builds upon previous ideas on

• Additional joints Mohr and Gleicher 2003
• Multi-weight enveloping Wang and Phillips 2002
• Our technique generalize the additional-
• bones model
• We evaluate cross-validation error to
• test for overfitting

Wang and Phillips 2002
47
Rotational regression is good at capturing muscle
bulges.