Stereo Correspondence using Differential Geometry

1
Stereo Correspondence using Differential Geometry

• Ricardo Fabbri
• EN 161, 11/2004

2
The Problem

• 3D scene
• 2D views
• Determine 3D information
• Depth

3
Key Step Correspondence

• Basic setup

4
Key Step Correspondence

• Find matching point
• Given only the two 2D views

5
Key Step Correspondence

• Need some criteria to guide match

6
Key Step Correspondence

• Epipolar line

7
Key Step Correspondence

• Further criteria needed
• To restrict/speedup match
• To match under nontrivial conditions

8
Li Zucker 2003

• Uses differential Geometry measures to guide
  correspondence
• What is New
• More robust
• Should handle Occlusions, branching,
  discontinuities, etc.
• Differential geometry never used before in this
  problem

9
Main ideas

• Compute intrinsic geometry measures at the 2D
  views
• Orientation
• Curvature
• Understand how they relate
• Corresponding measures in the 2D images come from
  the spatial measures on the same point of a
  spatial curve

10
Main ideas

• Use that knowledge to guide search for the right
  match
• Relaxation labeling
• Further criterion on binocular selective neurons

11
Geometry of Corresponding Points

• Given
• Two points, each at a 2D view
• Their orientation
• Their curvature
• Result the measures at the 3D point are
  determined
• Frenet frame and curvature
• EXCEPT FOR TORSION

12
Geometry of Corresponding Points
13
Geometry of Corresponding Points

• Central idea
• Resulting correspondence between all points
  should be compatible
• The 3D structure defining it should be a regular
  space curve
• Its measures are determined, except for torsion
• Minimize torsion

14
Results
15
Results
16
Results
17
Results
18
Results
19
Plan
20
Next 3 weeks

• Understand the math and other details
• Try to get torsion
• From curvature variation on the views
• Learn more about stereo and projective geometry
• Start synthetic experiments
• Generate curves and projections

21
Last 3 weeks

• Finish synthetic experiments
• Improvements
• Dream
• Practical results
• Edge map computation from real images
• Depth reconstructions