From Photohulls to Photoflux Optimization

1
BMVC 2006
From Photohulls to Photoflux Optimization
Yuri Boykov, Victor Lempitsky
Moscow StateUniversity
University of Western Ontario
Contribution novel class of geometrically
motivated functionals for volumetric multiview
reconstruction photoflux
Benefits

• captures properties of photohull
  Kutolakos and Seits, 2002
• can be combined with regularization (e.g. based
  on photoconsistency)
• unifies two major approaches to multiview
  reconstruction
• space carving and
  deformable models
• e.g. KutolakosSeits, 02
  e.g. FaugerasKeriven,
  98
• addresses shrinking and over-smoothing of
  regularization methods
• data-driven ballooning recovers thin
  protrusions or fine shape details
• can be optimized with graph cuts or variational
  methods, e.g. level-sets

Example with 4 cameras Kutolakos and Seitz, 02
Photoflux
Photohull
Flux-based methods
Combining Photoflux and Regularization for
Multiview Reconstruction
Regularization
Greedy methods
this work
Kutolakos Seits, 2000
Level-sets Vasilevsky and Sidiqqi, 2002 Kimmel et
al., 2003 Graph cuts Kolmogorov and Boykov, 2005
Snakes Kass et al., 1988 Level-sets Malladi et
al., 1994 Graph cuts Boykov and Jolly, 2001
Thresholding Region growing
The largest photoconsistent surface S computed by
carving out photo-inconsistent voxels from a
volume.
image segmentation
Mash-based Esteban and Schmitt,
2004 Level-sets Faugeras and Keriven, 1998 Graph
cuts Vogiatzis et al., 2005
Voxel coloring Seitz and Dyer, 1997 Space
carving Kutulakos and Seitz, 2002
multi-view reconstruction (volumetric approach)
This work
Local non-binary photoconsistency P(XS)
(non-deterministic decision at each voxel X)
Local binary photoconsistency P(XS)
(deterministic decision at each voxel X)
regularization
photoflux
Regularization uniform Ballooning (e.g.
FaugerasKeriven, Vogiatis et.al. 2005)
predefined threshold
Sum of color-discrepancies between cameras
observing voxel X given its visibility defined by
S
Photohull (Space Carving) KutolakosSeits, 2002
This work
Regularization intelligent ballooning (photo
flux)
1 out of 17 input photos
Low noise, but some details smoothed out
Details are fine, but noisy
Low noise and no shrinking (details are preserved)
More results are in the paper and in the tech.
report