Calibration

1
Calibration

• Marc Pollefeys
• COMP 256

Many slides and illustrations from J. Ponce
2
Tentative class schedule
Aug 26/28 - Introduction
Sep 2/4 Cameras Radiometry
Sep 9/11 Sources Shadows Color
Sep 16/18 Linear filters edges (hurricane Isabel)
Sep 23/25 Pyramids Texture Multi-View Geometry
Sep30/Oct2 Stereo Project proposals
Oct 7/9 Tracking (Welch) Optical flow
Oct 14/16 - -
Oct 21/23 Silhouettes/carving (Fall break)
Oct 28/30 - Structure from motion
Nov 4/6 Project update Proj. SfM
Nov 11/13 Camera calibration Segmentation
Nov 18/20 Fitting Prob. segm.fit.
Nov 25/27 Matching templates (Thanksgiving)
Dec 2/4 Matching relations Range data
Dec ? Final project
3
GEOMETRIC CAMERA MODELS

• Elements of Euclidean Geometry
• The Intrinsic Parameters of a Camera
• The Extrinsic Parameters of a Camera
• The General Form of the Perspective Projection
  Equation
• Line Geometry

Reading Chapter 2.
4
Quantitative Measurements and Calibration
Euclidean Geometry
5
Euclidean Coordinate Systems
6
Planes
7
Coordinate Changes Pure Translations
OBP OBOA OAP , BP AP BOA
8
Coordinate Changes Pure Rotations
9
Coordinate Changes Rotations about the z Axis
10
A rotation matrix is characterized by the
following properties

• Its inverse is equal to its transpose, and
• its determinant is equal to 1.

Or equivalently

• Its rows (or columns) form a right-handed
• orthonormal coordinate system.

11
Coordinate Changes Pure Rotations
12
Coordinate Changes Rigid Transformations
13
Block Matrix Multiplication
What is AB ?
Homogeneous Representation of Rigid
Transformations
14
Rigid Transformations as Mappings
15
Rigid Transformations as Mappings Rotation
about the k Axis
16
Pinhole Perspective Equation
17
The Intrinsic Parameters of a Camera
Units
k,l pixel/m
f m
a,b
pixel
Physical Image Coordinates
Normalized Image Coordinates
18
The Intrinsic Parameters of a Camera
Calibration Matrix
The Perspective Projection Equation
19
Extrinsic Parameters
20
Explicit Form of the Projection Matrix
Note
M is only defined up to scale in this setting!!
21
Theorem (Faugeras, 1993)
22
GEOMETRIC CAMERA CALIBRATION

• The Calibration Problem
• Least-Squares Techniques
• Linear Calibration from Points
• Linear Calibration from Lines
• Analytical Photogrammetry

Reading Chapter 3
23
Calibration Problem
24
Linear Systems
Square system

• unique solution
• Gaussian elimination

A
x
b

Rectangular system ??

• underconstrained
• infinity of solutions

A
x
b

• overconstrained
• no solution

2
Minimize Ax-b
25
How do you solve overconstrained linear equations
??
26
Homogeneous Linear Systems
Square system

• unique solution 0
• unless Det(A)0

A
x
0

Rectangular system ??

• 0 is always a solution

A
x
0

2
Minimize Ax under the constraint x 1
2
27
How do you solve overconstrained homogeneous
linear equations ??
The solution is e .
1
remember EIG(ATA)SVD(A), i.e. solution is Vn
28
Linear Camera Calibration
29
Once M is known, you still got to recover the
intrinsic and extrinsic parameters !!!
This is a decomposition problem, not an
estimation problem.
r

• Intrinsic parameters
• Extrinsic parameters

30
Degenerate Point Configurations
Are there other solutions besides M ??

• Coplanar points (l,m,n )(P,0,0) or (0,P,0) or
  (0,0,P )

• Points lying on the intersection curve of two
  quadric
• surfaces

straight line twisted cubic
Does not happen for 6 or more random points!
31
Analytical Photogrammetry
Non-Linear Least-Squares Methods

• Newton
• Gauss-Newton
• Levenberg-Marquardt

Iterative, quadratically convergent in favorable
situations
32
Mobile Robot Localization (Devy et al., 1997)
33
From Projective to Euclidean Images
If z , P , R and t are solutions, so are l z
, l P , R and l t .
Absolute scale cannot be recovered! The Euclidean
shape (defined up to an arbitrary similitude) is
the best that can be recovered.
34
From uncalibrated to calibrated cameras
Perspective camera
Calibrated camera
Problem what is Q ?
35
From uncalibrated to calibrated cameras II
Perspective camera
Calibrated camera
Problem what is Q ?
Example known image center
36
Sequential SfM

• Initialize motion from two images
• Initialize structure
• For each additional view
• Determine pose of camera
• Refine and extend structure
• Refine structure and motion

37
Initial projective camera motion

• Choose P and Pcompatible with F
• Reconstruction up to projective ambiguity

Same for more views?

• Initialize motion
• Initialize structure
• For each additional view
• Determine pose of camera
• Refine and extend structure
• Refine structure and motion

different projective basis
38
Initializing projective structure

• Reconstruct matches in projective frame
• by minimizing the reprojection error

Non-iterative optimal solution

• Initialize motion
• Initialize structure
• For each additional view
• Determine pose of camera
• Refine and extend structure
• Refine structure and motion

39
Projective pose estimation

• Infere 2D-3D matches from 2D-2D matches
• Compute pose from (RANSAC,6pts)

X
F
x

• Initialize motion
• Initialize structure
• For each additional view
• Determine pose of camera
• Refine and extend structure
• Refine structure and motion

Inliers
40
Refining and extending structure

• Refining structure
• Extending structure
• 2-view triangulation

(Iterative linear)

• Initialize motion
• Initialize structure
• For each additional view
• Determine pose of camera
• Refine and extend structure
• Refine structure and motion

41
Refining structure and motion

• use bundle adjustment

                       
Also model radial distortion to avoid bias!
42
Metric structure and motion

• use self-calibration (see next class)

Note that a fundamental problem of the
uncalibrated approach is that it fails if a
purely planar scene is observed (in one or more
views) (solution possible based on model
selection)
43
Dealing with dominant planes
44
PPPgric
HHgric
45
Farmhouse 3D models
(note reconstruction much larger than camera
field-of-view)
46
Application video augmentation
47
Next class Segmentation
Reading Chapter 14