Teleimmersion

1
Teleimmersion
2
Old method

• Pixel-based correlation
• Compute correlation scores for every pixel using
  left and right camera pairs
• Determine disparity if both pairs form peaks on
  agreed disparity
• Exhaustive but slow
• Typically takes 300ms over 1sec

3
Issues

• Not all points produce good correlation scores
• Reliable disparity from
• points on textured surface, i.e. edge points
• Low correlation scores on
• textureless surface
• occluded region

4
Points on textureless regions

• Low correlation scores on
• (1) textureless surface low S/N ratio
• (2)occluded region missing correspondences
• Thus typical normalized correlation fails on both
  types of regions.

5
New approach

• Selectively apply different correlation measures
  on textured and textureless points.
• For points on textured surface
• For points on textureless surface

6
New approach

• Region-based correlation
• Tessellate the scene into homogeneous regions
• MNCC (modified normalized corr.) for boundary
  points
• Maximum gray value diff. for interior points

7
Advantage

• Fast and efficient
• apply heavy MNCC on only sparse set of points
• Perform better on textureless regions and
  occluded regions
• Can provide high-level depth data which can be
  grouped together
• Ex) same surface if (i) homogeneous (ii) smooth
  depth change

8
Detail

• Region-based correlation
• (1) edge extraction
• (2) tessellate scene into triangular surfaces
• (3) measure correlation
• apply MNCC -gt edge pts
• apply MAX(G) -gt homogeneous pts

9
Detail
10
Detail

• Region-based correlation
• (4) measure disparity
• (5) do median filtering

11
Performance Comparison
Old method
New method

• Resize_img 1620ms
• Rectify 15 ms 30ms
• BK substract 30 ms
• Edge extraction 1520ms
• Delaunay TR 1530ms
• Compute CORR 3001200ms 5080ms
• Recon 50100ms 35ms

Total
4501300ms
130180 ms