The space of human body shapes: reconstruction and parameterization from range scans

1
The space of human body shapesreconstruction
and parameterization from range scans

• Brett Allen
• Brian Curless
• Zoran Popovic
• University of Washington

2
Motivation
Traditional high level characterization and
sparse anthropometric measurements do not capture
the detailed shape variations needed for
realism PS Sheldon, W.H. ,Stevens,
S.S.,Tucker, W.B. 1940 The varieties of Human
Physique ,New York
3
Motivation

• Traditional anthropometry has focused on sets of
• one-dimensional measurements.

4
Motivation

• Full body shape capture promises to advance the
• state of the art.

5
CAESAR

• Civilian American European Surface
  Anthropometry Resource

• thousands of subjects in the U.S. and Europe
• traditional anthropometry
• demographic survey
• laser range scans

Well use 250 of these scans (125 male, 125
female).
6
Related work

• Praun, E.,Sweldens, W,,Schroder, P. 2001
• Consistent Mesh Parameterization
• (In proceeding of ACM SIGGRAPH 2001)
• Lee, A.W.F.,Moreton, H.,Hoppe, H. 2000
• Displaced Subdivision Surfaces
• (In proceeding of ACM SIGGRAPH 2000)
• Blanz, V.,Vetter, T. 1999
• A Morphable Model For The Synthesis Of 3D
  Faces
• (In proceeding of ACM SIGGRAPH 1999)

7
Scan detail

• 250,000 triangles
• incomplete coverage

8
Scan detail
Caused by occlusions and grazing angle views
After template-based parameterization and hole
filling
9
The Correspondence Problem
10
Consistent Mesh Parameterization
Emil Praun Wim Sweldens Peter Schroder Princeton
Bell Lab
11
Purpose and Problem Specification

• Algorithm Input
• Set of meshes S
• Feature points F defined on each mesh M
• Algorithm Goal
• Determine common base domain B and connectivity
  L0
• Remesh each M with base domain B
• Create fair patch boundaries equivalent to L0

12
Algorithms
13
Topologically Equivalent Nets

• Definition A patch is a region of semi-regular
  connectivity in which triangles correspond to
  a single triangle
• Definition A net is the outline of patch
  boundaries
• We want a net that matches the connectivity L0
• Two patch boundaries may only intersect at a
  feature vertex
• Each feature vertex has a consistent cyclical
  ordering of edges
• Patch boundaries may not intersect
• Naïve algorithm (shortest path) does not achieve
  this

14
Restricted Brush Fire Algorithm

• Standard Brush Fire
• Starts fire at vertex
• Fire expands uniformly until it hits other vertex
• That is the shortest path
• Restricted Brush Fire
• Uses existing paths as firewalls
• Will only connect to vertex if approached from
  correct ordering
• Must avoid blocking off vertices
• Avoid completing any cycle until spanning tree
  of L0 traced

15
Fair Boundary Curves

• Rather than simply accept topologically
  equivalent net, we would like certain properties
• Equal distribution of surface area between
  patches
• Smooth Patch Boundaries
• Fair patch boundaries (should not swirl)
• First two can be handled using relaxation
• Iterative technique that involves progressively
  improving curve
• Third requires optimization of complicated
  expression
• Intractable, so try heuristics

16
Swirl Detection
If the shortest path from c to a,b
falls on the wrong side (left) ,the triangle
a,b,c is considered flipped and may lead to
swirls . On the right the trace reaches
a,bon the correct side and the path is
accepted.

17
Implementation
18
Example
19
Example
20
Matching algorithm

• Find the shape that
• 1. Matches the template markers to the scanned
  markers

2. Moves template vertices to scanned
vertices 3. Minimizes the deformation
scan
template
21
Match Algorithm

1. Applied a set of affine transformation Ti to the
   vertices of the template surface T and result in
   a new surface T
2. Minimizing three error term(data error,
   smoothness error, marker error)

22
Matching Algorithm
23
Objective Function

• Objective Function has three weighted terms
• Data error
• Smoothness error
• Marker error
• Will use different weights in each phase of
  process
• Multistep / Multi-resolution fitting process

24
Objective Function Data Error

• Data Error term requires current match to be
  close to target
• Uses distance from each transformed vertex to the
  target surface
• Weighted by confidence measure (from scanning)
• Hole regions have weight 0
• Sums total error
• Distance function
• Uses transformed template vertex
• Takes minimum distance to compatible vertices
  in target

25
Objective Function Smoothness Error

• Measures smoothness of deformation applied to
  template
• Problem is under-constrained using data error
• E_s measures change in T_I between adjacent
  vertices
• Encourages similarly-shaped features to be mapped
  to each other
• Uses Frobenius norm (vector L2 norm)

26
Objective Function Smoothness Error
?? Frobenius norm
A f
27
Smoothness Error
By smoothness, we are not referring to
smoothness of the deformed surface itself, but
rather smoothness of the actual deformation
applied to the template surface. In particular,
we require afine transformations applied within a
region of the surface to be as similar as
possible.
28
Objective Function Marker Error

• Data and Smoothness Error can hit local minima
• Example left arm transformed to right arm
• Solution Use pre-labeled markers on the test
  subjects
• Viewed as white dots in the range image
• Correspondences set up beforehand (as in
  Consistent Mesh Parameterization)
• 74 markers per subject (not all are used,
  however)
• Measure distance from template marker to target
  marker
• K_I are the indices of the markers in template,
  m_I are target markers

29
Algorithm Procedure

• Minimize error function using L-BFGS-B algorithm
• Quasi-Newton method with limited memory usage
• Make four passes over data (2 low res, 2 high
  res)
• Fit markers (low res, ? 0, ? 1, ? 10)
• Refit using data term (low res, ? 1, ? 1, ?
  10)
• Repeat in high resolution (hi res, ? 1, ? 1,
  ? 10)
• Refit using predominantly data term (hi res, ?
  10, ? 1, ? 1)

30
Hole Filling
31
PCA
(Principal component analysis)
The vectors with low variance can be discarded
,and then the full data set does not to be
retained in order to closely approximate the
original example
32
Feature analysis
Provide a direct way to explore the range of
bodies with intuitive controls,such as
height,weight,age and sex
33
Feature analysis
34
Markerless Matching

• Using PCA, we can remove the marker requirement
• A set of training data is fit using markers
• Other data can be registered using unmarked range
  scans
• Uses PCA weights to search PCA space, not
  transformation space

35
Markless-only

• We can use PCA to remove the range images
• Only use markers can be captured with much
  cheaper equipment
• Allows us to determine approximate shape of object

36
Application
1.Transfer of Texture
37
Application
1.Morphing between model
38
Application
1.Instrumentation transfer
39
Statistical analysis
mean PCA component 1
40
Statistical analysis
mean PCA component 2
41
Statistical analysis
mean PCA component 3
42
PCA reconstruction
43
Fitting to attributes
44
Fitting to points

• Using the distribution of the PCA weights as a
  prior, we can find the most likely person that
  fits a set of point constraints.

PCA variance
45
Summary

• Contributions
• - an algorithm for creating a consistent mesh
  representation from range scan data.
• - several ways to explore the variation in human
  body shape, and to synthesize and edit body
  models

46
Future work

• - analyze shape variation between poses

47
Future work

• - combine with anatomical models and physical
  simulation

Aubel 2003
48
(No Transcript)