HardwareAccelerated Silhouette Matching

1
Hardware-AcceleratedSilhouette Matching

• Hendrik Lensch, Wolfgang Heidrich,
• and Hans-Peter Seidel
• Max-Planck-Institut für Informatik,
• Saarbrücken (Germany)

2
Overview

• Motivation
• Comparing Silhouettes
• Stitching and Combining Textures
• Results and Conclusions

3
Acquiring Real World Models

• Geometry
• 3D scanner

• Texture data
• digital camera

single sensor vs. multiple sensors
4
3D 2D Registration

• Find the camera setting for each 2D image.

5
Camera Model

• Transformations
• to camera coordinates (extrinsic)
• to 2D image space (intrinsic)
• ? determine R, t and f (61 dimensions)

6
Similarity Measure

• Which features to investigate?
• no color information on the model
• correspondence of geometric features hard to find

7
Similarity Measure

• Compare silhouettes Etienne de Silhouette
  1709-1767
• model render monochrome
• photo automatic histogram-based segmentation

8
Similarity Measure

• Compare silhouettes Etienne de Silhouette
  1709-1767
• model render monochrome
• photo automatic histogram-based segmentation

9
Distance Measure for Silhouettes

• Point-to-outline distances
• slow because points on the outline must be
  determined
• speedup by distance maps

10
Pixel-based Distance Measure

• Count the number of pixels covered by just one
  silhouette.
• XOR the images
• compute histogram (hardware)
• gives linear response to the displacement

1
11
Pixel-based Distance Measure

• Count the number of pixels covered by just one
  silhouette.
• XOR the images
• compute histogram (hardware)
• gives linear response to the displacement

displacement
12
Approximation ofSquared Distances

• Use smooth transitions
• blur images
• integrate squared differences
• faster convergence
• reduced variance
• higher evaluation cost

13
Approximation ofSquared Distances

• Use smooth transitions
• blur images
• integrate squared differences
• faster convergence
• reduced variance
• higher evaluation cost

1
difference
x
14
Non-linear Optimization

• Downhill Simplex Method Press 1992
• works for N dimensions
• no derivatives
• easy to control

15
Simplex Method in 3D
original simplex
reflection and/or expansion
shrinking
random perturbation
16
Hierarchical Optimization

• optimize on low resolution first
• restart optimization to avoid local minima
• switch to higher resolution
• mesh resolution can be adapted

17
Starting Point Generation

• set camera distance tz depending on object size
• set tx and ty to zero
• select 48 sample rotations
• run optimization for each of the samples
• (40 evaluations)
• select top 5 results
• restart optimization (200 evaluations)
• take best result as starting point

18
Texture Stitching

• projective texture mapping
• assign one image to each triangle
• triangle visible in image? (test every vertex)
• select best viewing angle
• discard data near depth discontinuities

19
Blending Across Assignment Borders

• find border vertices
• release all triangles around them
• assign boundary vertices to best region
• assign alpha-values for each region
• 1 to vertices included in the region
• 0 to all others.

20
Entire Texture
21
Results and Conclusions

• Problems solved
• automatic texture registration (R, t, f)
• view-independent texture stitching
• blending across assignment boundaries
• rough manual alignment helps (speedup, failures)
• Further problems
• extract purely diffuse part of texture
• generate texture where data is missing

22
Questions?

• visit us at
• www.mpi-sb.mpg.de

23
Erroneous Pixels

• Reasons
• imprecise 3D model
• parts visible in image but not modeled
• occluded parts
• segmentation errors

• Outline affected?

no ? dont care
yes ? mask out
24
Texture Acquisition

• Imaging all visible surfaces
• Stuerzlinger 1999, Matsushita Kaneko 1999
• 3D 2D registration
• Lowe 1991, Brunie et al. 1992, Guenter 1998
• Neugebauer Klein 1999, Matsushita Kaneko 1999
• Texture preparation / rendering
• Sato et al. 1997, Rocchini et al. 1999,
  Marschner 1998, Wood et al. 2000

25
Optimizing the Focal Length

• 1D search
• start with f determined from the applied lens
• optimize R,t for this f
• increment f by d
• update tz with respect to new f
• optimize R,t for new f
• proceed while better result is achieved
• otherwise step back to previous f, halve d