Camera calibration

1
Camera calibration

• Digital Visual Effects, Spring 2007
• Yung-Yu Chuang
• 2007/4/17

with slides by Richard Szeliski, Steve Seitz, and
Marc Pollefyes
2
Announcements

• Project 2 is due next Tuesday before the class

3
Outline

• Camera projection models
• Camera calibration (tools)
• Nonlinear least square methods
• Bundle adjustment

4
Camera projection models
5
Pinhole camera
6
Pinhole camera model
(X,Y,Z)
P
origin
p
(x,y)
principal point
(optical center)
7
Pinhole camera model
principal point
8
Pinhole camera model
principal point
9
Principal point offset
principal point
intrinsic matrix
only related to camera projection
10
Intrinsic matrix
Is this form of K good enough?

• non-square pixels (digital video)
• skew
• radial distortion

11
Distortion
No distortion
Pin cushion
Barrel

• Radial distortion of the image
• Caused by imperfect lenses
• Deviations are most noticeable for rays that pass
  through the edge of the lens

12
Camera rotation and translation
extrinsic matrix
13
Two kinds of parameters

• internal or intrinsic parameters such as focal
  length, optical center, aspect ratiowhat kind
  of camera?
• external or extrinsic (pose) parameters including
  rotation and translationwhere is the camera?

14
Other projection models
15
Orthographic projection

• Special case of perspective projection
• Distance from the COP to the PP is infinite
• Also called parallel projection (x, y, z) ?
  (x, y)

Image
World
16
Other types of projections

• Scaled orthographic
• Also called weak perspective
• Affine projection
• Also called paraperspective

17
Fun with perspective
18
Perspective cues
19
Perspective cues
20
Fun with perspective
Ames room
21
Forced perspective in LOTR
22
Camera calibration
23
Camera calibration

• Estimate both intrinsic and extrinsic parameters
• Mainly, two categories
• Photometric calibration uses reference objects
  with known geometry
• Self calibration only assumes static scene, e.g.
  structure from motion

24
Camera calibration approaches

• linear regression (least squares)
• nonlinear optimization
• multiple planar patterns

25
Chromaglyphs (HP research)
26
Linear regression
27
Linear regression

• Directly estimate 11 unknowns in the M matrix
  using known 3D points (Xi,Yi,Zi) and measured
  feature positions (ui,vi)

28
Linear regression
29
Linear regression
30
Linear regression
Solve for Projection Matrix M using least-square
techniques
31
Normal equation

• Given an overdetermined system

the normal equation is that which minimizes the
sum of the square differences between left and
right sides
Why?
32
Normal equation
nxm, n equations, m variables
33
Normal equation
34
Normal equation
35
Normal equation
36
Linear regression

• Advantages
• All specifics of the camera summarized in one
  matrix
• Can predict where any world point will map to in
  the image
• Disadvantages
• Doesnt tell us about particular parameters
• Mixes up internal and external parameters
• pose specific move the camera and everything
  breaks

37
Nonlinear optimization

• A probabilistic view of least square
• Feature measurement equations
• Likelihood of M given (ui,vi)

38
Optimal estimation

• Log likelihood of M given (ui,vi)
• It is a least square problem (but not necessarily
  linear least square)
• How do we minimize C?

39
Optimal estimation

• Non-linear regression (least squares), because
  the relations between ûi and ui are non-linear
  functions M
• We can use Levenberg-Marquardt method to minimize
  it

unknown parameters
known constant
40
A popular calibration tool
41
Multi-plane calibration

Images courtesy Jean-Yves Bouguet, Intel Corp.

• Advantage
• Only requires a plane
• Dont have to know positions/orientations
• Good code available online!
• Intels OpenCV library http//www.intel.com/rese
  arch/mrl/research/opencv/
• Matlab version by Jean-Yves Bouget
  http//www.vision.caltech.edu/bouguetj/calib_doc/i
  ndex.html
• Zhengyou Zhangs web site http//research.micros
  oft.com/zhang/Calib/

42
Step 1 data acquisition
43
Step 2 specify corner order
44
Step 3 corner extraction
45
Step 3 corner extraction
46
Step 4 minimize projection error
47
Step 4 camera calibration
48
Step 4 camera calibration
49
Step 5 refinement
50
Nonlinear least square methods
51
Least square fitting
number of data points
number of parameters
52
Linear least square fitting
y
model
parameters
t
53
Nonlinear least square fitting
54
Function minimization
Least square is related to function minimization.

• It is very hard to solve in general. Here, we
  only consider a simpler problem of finding local
  minimum.

55
Function minimization
56
Quadratic functions
Approximate the function with a quadratic
function within a small neighborhood
57
Quadratic functions
A is positive definite. All eigenvalues are
positive. Fall all x, xTAxgt0.
negative definite
A is singular
A is indefinite
58
Function minimization
59
Descent methods
60
Descent direction
61
Steepest descent method
the decrease of F(x) per unit along h direction
?
hsd is a descent direction because hTsd F(x)-
F(x)2lt0
62
Line search
63
Line search
64
Steepest descent method
isocontour
gradient
65
Steepest descent method

• It has good performance in the initial stage of
  the iterative process. Converge very slow with a
  linear rate.

66
Newtons method
?
?
?
?

• It has good performance in the final stage of the
  iterative process, where x is close to x.

67
Hybrid method

• This needs to calculate second-order derivative
  which might not be available.