Multiple View Geometry

1
Multiple View Geometry

• Jim Rehg
• CS 4495/7495
• Many slides by Frank Dellaert
• (reconstruction by Antonio Criminsi)

2
Book Sequence
3
Book Structure Motion
4
2 Problems !

• Correspondence
• Optimization

5
A Correspondence Problem
6
An Optimization Problem

• Find the most likely structure and motion ?

7
Bundle Adjustment
Image i
Image i
uik
8
Geometric Intuition !!

• From first lecture on cameras !

(0.6,2.5,1) ?
Z
Y
X
9
Geometric Intuition

• Projection matrix form

Z
Y
X
10
Affine Cameras

• Easy projection equation

11
Translation of image i ?

• Take derivative wrpt to ti

12
World origin trick

• Move scene so centroid origin

13
New Problem !

• Just subtract centroid from all measurements

14
Matrix Form

• Projection in one image (i)

15
Matrix Form

• Projection in two images

16
Matrix Form

• Projection all points in all images

n
16
13
14
M1
1
15
2
3
11
12
4
5
6

24
M2
25
26
21
22
23
2m
34
M3
35
36
31
32
33
44
M4
45
46
41
42
43
17
Geometric Intuition !!

• Joint Image of point 8-dimensional

Z
Y
X
18
Subspace Constraints

• Does it really span 8-dim space ??

Z
Y
X
19
Joint Camera Mapping
83
R3
R8
Mapping from 3D to 8D space (for 4 cameras)
20
But Range is only 3D !!!
83
R3
R3
R5
5-dimensional null-space !
21
What does SVD look like ?
AU S VT
3x3
83
3x3
R3
R3P2
R5
Rank 3, economy SVD
22
And measurement matrix ?
Also Rank 3 !!!
23
SVD Approach to SFM

• Form measurement matrix
• Do rank 3 economy size SVD
• Voila

Motion
Structure
24
Problem

• Where to stick the extra 3x3 ?

Motion
Structure
3x3
3x3
83

25
Euclidean Upgrade

• Uncalibrated not so easy
• Calibrated cameras
• We know they are rotation matrices
• Impose constraints
• unit length 2m
• orthogonal m
• Solve for Q

3x3
3x3
26
Additional Details

• How to obtain measurements?
• Tracking of point features across video
• (We will discuss this later)
• Two-step solution process in general
• Initial solution
• Factorization, 2-view or 3-view solutions
• Bundle adjustment
• Nonlinear minimization from good starting point

27
UrbanScape

• Large scale 3D modeling from video
• Integrated system combining recent techniques
• Large scale 500K frames
• Uses GPU acceleration
• DARPA funded project, PIs
• Marc Pollefeys (UNC Chapel Hill, now ETH)
• David Nister (Univ. Kentucky, now MSR)