Title: 3-D Computer Vision CSc 83020
13-D Computer VisionCSc 83020
2Camera Calibration
- Problem Estimate cameras extrinsic intrinsic
parameters. - Method Use image(s) of known scene.
- Tools
- Geometric camera models.
- SVD and constrained least-squares.
- Line extraction methods.
3Camera Calibration
Perspective Equations
Feature Extraction
From Sebastian Thrun and Jana Kosecka
4Perspective Projection, Remember?
O
X
x
f
Z
From Sebastian Thrun and Jana Kosecka
5Intrinsic Camera Parameters
- Determine the intrinsic parameters of a camera
(with lens) - What are Intrinsic Parameters?
- (can you name 7?)
From Sebastian Thrun and Jana Kosecka
6Intrinsic Parameters
O
Image center(ox, oy)
X
f
Z
From Sebastian Thrun and Jana Kosecka
7Intrinsic Camera Parameters
- Intrinsic Parameters
- Focal Length f
- Pixel size sx , sy
- Image center ox , oy
- (Nonlinear radial distortion coefficients k1 ,
k2) - Calibration Determine the intrinsic parameters
of a camera
From Sebastian Thrun and Jana Kosecka
8Coordinate Frames
Camera Coordinate Frame
Zc
Pixel Coordinates
Yc
Intrinsic Parameters
Xc
Extrinsic Parameters
Image Coordinate Frame
Zw
Yw
World Coordinate Frame
Xw
9Why Calibrate?
Image point
Image
Calibration relates points in the image to rays
in the scene
10Why Calibrate?
Image point
x
y
z
Image
x
y
Calibration relates points in the image to rays
in the scene
11Example Calibration Pattern
From Sebastian Thrun and Jana Kosecka
Calibration Pattern Object with features of
known size/geometry
12Harris Corner Detector
From Sebastian Thrun and Jana Kosecka
13Perspective Camera
r
r
(x,y,z)
(X,Y,Z)
Center of Projection
r (x,y,z) r(X,Y,Z)
xf X/Z yf Y/Z zf
r/fr/Z
f effective focal length distance of image
plane from O.
14Extrinsic Parameters
T
PcR(Pw-T) Translation followed by rotation
15Extrinsic Parameters (2nd formulation)
R same as before T different
PcR Pw T Rotation followed by translation
16The Rotation Matrix
T
T
R R R R I gt R R Orthonormal
Matrix Degrees of freedom? 1 0 0 I 0 1 0 0 0 1
T
-1
17Perspective Camera Model
- Step 1 Transform into camera coordinates
- Step 2 Transform into image coordinates
From Sebastian Thrun and Jana Kosecka
18Intrinsic Parameters
19Image and Camera Frames
Ycamera
Zcamera
Ximage
(xim, yim)
Xcamera
(ox,oy)
Yimage
20Geometric Model
3D Point in Camera Coordinate Frame
XcT
x
ZcT
YcT
y
ZcT
- Transformation from Image
- to Camera Frame.
- (ox,oy,sx,sy)
- No distortion!
- Transformation from
- World to Camera Frame.
- Perspective projection
- (f, R, T)
Point in Camera Frame
21Camera Calibration Issues
- Which parameters need to be estimated.
- Focal length, image center, aspect ratio
- Radial distortions
- What kind of accuracy is needed.
- Application dependent
- What kind of calibration object is used.
- One plane, many planes
- Complicated three dimensional object
22Camera Calibration
Calibration object
Extracted features
23Camera Calibration
Extract centers of circles
24Basic Equations
25Basic Equations
In the remaining slides x means xim and y means
yim
26Basic Equations
27Basic Equations
Extrinsic Parameters 1) Rotation matrix
R (3x3) 2) Translation vector T (3x1)
- Intrinsic Parameters
- fxf/sx, length in effective horizontal pixel
size units. - asy/sx, aspect ratio.
- (ox,oy), image center coordinates.
- Radial distortion coefficients.
Total number of parameters (excluding
distortion) ?
28Basic Equations
- Assume that image center is known.
- Solve for the remaining parameters.
- Use N image points and their
- corresponding
- N world points
29Basic Equations
(1)
- Assume that image center is known.
- Solve for the remaining parameters.
- Use N image points and their
- corresponding
- N world points
Here x means xim - ox and y means yim - oy
30Basic Equations
(2)
- Assume that image center is known.
- Solve for the remaining parameters.
- Use N image points and their
- corresponding
- N world points
31Basic Equations
(3)
32Basic Equations
(3)
How would we solve this system?
33Basic Equations
(3)
How would we solve this system? Rank of matrix
A? Solution up to a scale factor.
34Singular Value Decomposition
Appendix A.6 (Trucco)
A m x n U m x m, columns orthogonal unit
vectors. V n x n , -//- D m x n ,
diagonal. The diagonal elements si are
the singular values s1gt s2gt gt sn gt
0
35Singular Value Decomposition
Appendix A.6 (Trucco)
- Square A non-singular iff si ! 0
- For square A Cs1/sN is the condition number
- For rectangular A of non-zero si is the rank
- For square non-singular A
- For square A, pseudoinverse
- Singular values of A square roots of
- eigenvalues of and
- 7. Columns of U, V
- Eigenvectors of
- 8. Frobenius norm of a matrix
36Singular Value Decomposition
Appendix A.6 (Trucco)
If rank(A)n-1 (7 in our case) then the solution
is the eigenvector which corresponds to the ONLY
zero eigenvalue. Solution up to a scale factor.
37Solving for v
(3)
How would we solve this system SVD. Solution
Uknown scale factor ?? Aspect ratio a?
38Solving for Tz and fx?
39Solving for Tz and fx?
How would we solve this system?
40Solving for Tz and fx?
How would we solve this system?
Solution in the least squares sense.
41Camera Center
42Camera Models (linear versions)
3D Point in World Coordinate Frame
x
y
- Transformation from
- World to Camera Frame.
- Perspective projection
- (f, R, T)
- Transformation from Image
- to Camera Frame.
- (ox,oy,sx,sy)
- No distortion!
Point in Camera Frame
43Camera Models (linear versions)
Elegant decomposition. No distortion!
Homogeneous Coordinates
World Point (Xw, Yw,Zw)
Measured Pixel (xim, yim)
?
44Camera Calibration Other method
Extracted features
Step 1 Estimate P Step 2 Decompose P into
internal and external parameters R,T,C
45Camera Calibration Step 1
Extracted features
Each point (x,y) gives us two equations
46Camera Calibration Step 1
Extracted features
Each corner (x,y) gives us two equations
47Camera Calibration Step 1
2n
Extracted features
n points gives us 2n equations
48Camera Calibration Step 1
2n
Extracted features
We need to solve
In the presence of noise we need to solve
The solution is given by the eigenvector with the
smallest eigenvalue of
49Camera Calibration Step 1
The result can be improved through non-linear
minimization.
Extracted features
50Camera Calibration Step 1
The result can be improved through non-linear
minimization.
Extracted features
Minimize the distance between the predicted and
detected features.