Finite Element Surface-Based Stereo 3D Reconstruction

1
Finite Element Surface-Based Stereo 3D
Reconstruction
Edgar J. Lobaton Lopez, UC BerkeleyTracy
Xiaoxiao Wang, UC Berkeley

• The Project
• Abstract
• The goal is the design of an algorithm for 3D
  reconstruction of scenes based on stereo pair
  images. Here we aim for a surface based approach
  using finite elements and a triangular wavelets
  for real-time computations and compactness of the
  representation. Compactness is important for
  future transmission of the data over a network.
• Why A Surface Representation?
• A surface representation is compact compared to a
  dense cloud of points in 3D.
• A surface defining connectivity between vertices
  in a triangulation gives structure for the design
  of operators and data manipulation.
• Triangular Wavelets

• Finite Element Process
• Using Finite Elements, we can define operators
  (filters) in the same way as it is done for
  regular grids.
• Operators in our triangulated domain can include
  scale information for faster convergence without
  losing much detail.
• In order to compute 3D structure, we look for the
  disparity between two images, which can be
  interpreted as the horizontal shift that maps one
  image to another.
• Steps
• We start with a stereo pair of images (left and
  right)
• A set of nodes from the triangular representation
  is selected for correlation computation (NCC).
• Wholes are filled-in using the information that
  is available.
• Use the previous result as an initial condition
  for an iterative process in which we minimize an
  energy functional. This energy has an
  image-driven smoothing term and an energy term
  from the correlation values.

• Data Size
• The size of the data shown below is before any
  compression is applied to it
• Original Image (352 x 288 pixels)
• 100 KB
• Representation and Disparity Map
• 80 KB
• The later includes all the information required
  for 3D reconstruction.
• Benefits
• No gradient computations are required, only
  integrations, which make it robust to noise.
• Most of the steps in the algorithm are designed
  for parallel implementation.
• Multiscale treatment comes naturally.
• Compact representation which gives connectivity
  information and allows the design of operators
  for the data.

• Hierarchical Structure
• Selecting Representation
• Selection based on a threshold of the normalized
  error per triangle
• We obtain a Multiscale Representation
• Mean and Variance values propagate recursively in
  parallel through the tree

• Sample Reconstructed Views
• The original view and a reconstructed view are
  shown below.
• The result is a surface represented using linear
  finite elements. Further smoothing and
  interpolation is still possible for better
  rendering of images.

April 27, 2006