VideoTrace: Rapid interactive scene modelling from video

1
VideoTrace Rapid interactive scene modelling
from video

• Anton van den Hengel
• Anthony Dick
• Thorsten Thormahlen
• Ben Ward1 Philip H. S. Torr
• Siggraph 2007

2
Outline

• Introduction
• Related work
• Implementation
• Conclusion

3
Introduction

• The recovery of 3D models from video
• Automatic reconstruction techniques 2D
  interactions
• Automatically recover a cloud of 3D scene points
• Modeling package such as Blender3D
• Surface-based reconstruction of general object

4
Related Work

• Photomodeler Eos System 2005
• Image-based plant modelingQuan Sig06?hair
  modellingWei et al Sig05
• Building models of regular scenes from structure
  and motion van den et al BMVC06

5
Video Pre-processing

• Structure and motion analysis by use of Voodoo
  camera Tracker
• 3D scene points (point cloud)
• camera parameters
• Superpixel segmentation accelerate
• Region clustering
• Edge finding

6
Structure from motion

• Feature extraction
• By use of tracked or matched features between
  multiple views
• Iterative
• Update zi,j
• 3D point clouds camera parameters

7
Polygon Modeling and Tracing(1/3)

• Fitting multiple planes to subsets of the 3D
  points
• These sets of planes are then combined to form a
  set of hypothesized object models

8
Polygon Modeling and Tracing(2/3)

• for each hypothesized model
• distance from each face to the corresponding set
  of reconstructed 3D points
• For hypothesized models which score well
• Tracing the face in other frame (end point
  clustering)
• Form a color histogram for each face
• Difference between the histograms for
  corresponding faces in different images,
• Sum over all faces

9
Curves Modeling(1/2)
NURBS Curves parametric form
10
Curves Modeling(2/2)
Max-flow concept
11
NURBS Surface

• 1-rail sweep surface 2-rail sweep surface
• Minimizes the distance between the object points
  and the surface
• (Levenberg-Marquardt optimiser )

12
Mirror Planes

• Estimate each surface separately
• Lose regularity and symmetry
• choose sets of 3 midpoints to form a mirror plane
• Optimize the mirror plane

13
Conclusion

• Well known limitations by use of structure and
  motion
• degenerate camera motion
• completely planar scenes
• Using robust surface fitting methods
• for instance reflective surfaces can still be
  modeled
• Modeling parts of the scene that are not visible
  in the video
• Combining the benefits of automatic and manual
  modeling systems