Stereo part 2

1
Stereo (part 2)

• Jim Rehg
• CS 4495/7495 Computer Vision
• Lecture 4
• Wed Aug 28, 2002

2
Stereo Vision
Z(x, y) is depth at pixel (x, y) d(x, y) is
disparity
Left
Right
Matching correlation windows across scan lines
3
Basic Stereo Geometry
(X, Y, Z)
L
R
W
f
B
xL
xR
How do matching errors affectthe depth error?
4
Stereo Correspondence

• Search over disparity to find correspondences
• Range of disparities to search over can change
  dramatically within a single image pair.

5
Correspondence Using Correlation
Left
Right
scanline
SSD error
disparity
Left
Right
6
Sum of Squared (Pixel) Differences
Left
Right
7
Image Normalization

• Even when the cameras are identical models, there
  can be differences in gain and sensitivity.
• The cameras do not see exactly the same surfaces,
  so their overall light levels can differ.
• For these reasons and more, it is a good idea to
  normalize the pixels in each window

8
Images as Vectors
Unwrap image to form vector, using raster scan
order
Left
Right
row 1
row 2
Each window is a vectorin an m2
dimensionalvector space.Normalization
makesthem unit length.
row 3
9
Image Metrics
(Normalized) Sum of Squared Differences
Normalized Correlation
10
Correspondence Using Correlation
Left
Disparity Map
Images courtesy of Point Grey Research
Left
Right
11
Stereo Results

• Trinocular stereo system available from Point
  Gray Research for 5K (circa 97)

12
Stereo Requirements

• Matching or scoring function
• Sum of squared (pixel) differences (SSD)
• Equivalent to normalized correlation
• Constraints
• Rectified images
• Match order constraint
• Search algorithm
• Dynamic programming

13
Epipolar Geometry

• The epipolar geometry is the fundamental
  constraint in stereo.
• Rectification aligns epipolar lines with
  scanlines

Epipolar plane
Epipolar line for p
Epipolar line for p
14
Correspondence

• It is fundamentally ambiguous, even with stereo
  constraints

Ordering constraint
and its failure