Imagebased Water Surface Reconstruction with Refractive Stereo

1
Image-based Water Surface Reconstruction with
Refractive Stereo

• Nigel Morris
• University of Toronto

2
Motivation

• Computational Fluid Dynamics are extremely
  complex and difficult to simulate
• Why not capture fluid effects from reality?
• We present the first step to capturing fluids
  from reality reconstructing water surfaces
• May eventually be useful for determining fluid
  flow

3
Previous Work

• Shape from shading Schultz94
• Requires large area light source or multiple
  views
• Shape from refractive distortion Murase90
• Limited wave amplitude, orthographic camera model
• Laser range finders Wu90
• Specialized equipment

4
Previous work

• Shape from refractive irradiance Jähne92,
  Zhang94 Daida95
• Requires underwater lens, orthographic camera
  model

5
Goals of our system

• Physically-consistent water surface
  reconstruction
• Reconstruction of rapid sequences of flowing,
  shallow water
• High reconstruction resolution
• Use of a minimal number of viewpoints and props

6
Technical Contributions

• We present a design for a stereo system for
  capturing sequences of dynamic water
• System implementation and results
• Refractive stereo matching metrics and analysis
• Effective localization of surface points of
  shallow water

7
Refraction

• Snells Law
• r1sin Ti r2sin Tr
• For air ? water
• sin Ti rwsin Tr

8
Imaging water

• Image point f at q without water
• Image f at q with water
• qq is the refractive disparity

9
Deriving the surface normal

• Suppose we know the location of the surface point
  p and its depth from the camera z
• We know the angle ?d between the refracted rays u
  and v
• Can compute the incident angle ?i, then the
  normal n

10
Solution space

• For given refractive disparity, set of solution
  pairs
• nmzm
• For every depth z, there is at most one normal n

11
Reconstruction with Stereo

• Same setup as with one camera, but with
  additional calibrated camera
• We search through the ltnmzmgt solution space for a
  particular refractive disparity
• We use the second camera to determine the error
  for each instance of nmzm
• Return best surface point p

12
Refractive stereo matching
Camera 2
Camera 1
n2
n1
n
Tank Bottom
13
Matching metric

• Normal collinearity metric
• Measure the angle between the two normals n1 and
  n2 to give an error.
• Disparity difference metric
• Swap n1 and n2 and reproject to tank plane,
  measure disparity from the projection before
  swapping.
• Seeks to minimize error due to inaccurate normal
  measurements as water depth approaches
  localization error range.

14
Disparity Difference Metric
Camera 2
Camera 1
Tank Bottom
e1
e2
15
Metric Comparison

• Disparity difference metric in red
• Normal collinearity metric in blue

16
Implementation details

• Pattern choice
• Checkered pattern used
• Tracking pattern and localization
• Lucas-Kanade matching
• Interpolation of the discrete pattern

17
System Inputs

• Calibrated stereo camera system
• Images of pattern without water from both cameras
  to give refractive disparities
• Distorted pattern image sequences

18
Corner tracking

• In order to reconstruct a sequence of frames, the
  corners must be localized at every frame
• We employ a Lucas-Kanade matching technique,
  matching templates of the corners to the next
  frame

19
Corner Interpolation

• We cannot assume that our verification ray will
  land on one of the corners
• We thus find the four nearest non-collinear
  corners
• The surface may be distorted so we cannot assume
  a grid formation
• We interpolate between these corners to find the
  distortion of the verification ray

20
Results

• Ripple Drop
• Waves
• Pouring water

21
Future Work

• Global surface minimization vs local
• Planar tank constraint removal
• More complex water scenario capturing

22
References

• Jähne92 B. Jähne, J. Klinke, P. Geissler, and
  F. Hering. Image sequence analysis of ocean wind
  waves. In Proc. International Seminar on Imaging
  in Transport Processes, 1992.
• Murase90 H. Murase. Shape reconstruction of an
  undulating transparent object. In Proc. IEEE
  Intl. Conf. Computer Vision, pages 313317, 1990.
• Schultz94 H. Schultz. Retrieving shape
  information from multiple images of a specular
  surface. IEEE Transactions on Pattern Analysis
  and Machine Intelligence, 16(2)195201, 1994.
• Wu90 Z. Wu and G. A. Meadows. 2-D surface
  reconstruction of water waves. In Engineering in
  the Ocean Environment. Conference Proceedings,
  pages 416421, 1990.
• Zhang94 X. Zhang and C. Cox. Measuring the
  two-dimensional structure of a wavy water surface
  optically A surface gradient detector.
  Experiments in Fluids, Springer Verlag,
  17225237, 1994.