Camera Parameters

1
Camera Parameters

• CS485/685 Computer Vision
• Prof. George Bebis

2
CCD array and frame buffer

• The physical image plane is the CCD array of n x
  m rectangular grid of photo-sensors.
• The pixel image plane (frame buffer) is an array
  of N x M integer values (pixels).

3
CCD array and frame buffer (contd)

• The position of the same point on the image plane
  will be different if measured in CCD elements (x,
  y) or image pixels (xim, yim).

(assuming that the origin in both cases is the
upper-left corner)
(xim, yim) measured in pixels (x, y) measured in
millimeters.
4
Reference Frames

• Five reference frames are needed in general for
  3D scene analysis.
• Object
• World
• Camera
• Image
• Pixel

5
(1) Object Coordinate Frame

• 3D coordinate system (xb, yb, zb)
• Useful for modeling objects (i.e., check if a
  particular hole is in proper position relative to
  other holes)
• Object coordinates do not change regardless how
  the object is placed in the scene.
• Our notation (Xo, Yo, Zo)T

6
(2) World Coordinate Frame

• 3D coordinate system (xw, yw, zw)
• Useful for interrelating objects in 3D
• Our notation (Xw, Yw, Zw)T

7
(3) Camera Coordinate Frame

• 3D coordinate system (xc, yc, zc)
• Useful for representing objects with respect to
  the location of the camera.
• Our notation (Xc, Yc, Zc)T

8
(4) Image Plane Coordinate Frame (i.e., CCD plane)

• 2D coordinate system (x f , y f )
• Describes the coordinates of 3D points projected
  on the image plane.
• Our notation (x, y)T

9
(5) Pixel Coordinate Frame

• 2D coordinate system (c, r)
• Each pixel in this frame has integer pixel
  coordinates.

y
Our notation (xim, yim)T
x
10
Transformations between frames
11
World and Camera coordinate systems

• In general, the world and camera coordinate
  systems are not aligned.

center of projection
optical axis
12
World and Camera coordinate systems (contd)

• To simplify mathematics, lets assume
• (1) The center of projection coincides with the
  origin of the world coordinate system.
• (2) The optical axis is aligned with the worlds
  z-axis and
• x,y are parallel with X, Y

13
World and Camera coordinate systems (contd)

• (3) Avoid image inversion by assuming that the
  image plane is in front of the center of
  projection.
• (4) The origin of the image plane is the
  principal point.

14
Terminology - Summary

• The model consists of a plane (image plane) and a
  3D point O (center of projection).
• The distance f between the image plane and the
  center of projection O is the focal length (e.g.,
  the distance between the lens and the CCD array).

15
Terminology - Summary (contd)

• The line through O and perpendicular to the image
  plane is the optical axis.
• The intersection of the optical axis with the
  image plane is called principal point.

Note the principal point is not necessarily the
image center.
16
The equations of perspective projection
17
The equations of perspective projection (contd)

• Using matrix notation
• Verify the correctness of the above matrix
• homogenize using w Z

or
18
Properties of perspective projection

• Many-to-one mapping
• The projection of a point is not unique
• Any point on the line OP has the same projection

19
Properties of perspective projection (contd)

• Scaling/Foreshortening
• Objects image size is inversely proportional to
  the distance of the object from the camera.

20
Properties of perspective projection (contd)

• When a line (or surface) is parallel to the image
  plane, the effect of perspective projection is
  scaling.
• When an line (or surface) is not parallel to the
  image plane, the effect is foreshortening (i.e.,
  perspective distortion).

21
Properties of perspective projection (contd)

• Effect of focal length
• As f gets smaller, more points project onto the
  image plane (wide-angle camera).
• As f gets larger, the field of view becomes
  smaller (more telescopic).

22
Properties of perspective projection (contd)

• What happens to lines, distances, angles and
  parallelism?
• Lines in 3D project to lines in 2D (with an
  exception )
• Distances and angles are not preserved.
• Parallel lines do not in general project to
  parallel lines due
• to foreshortening (unless they are parallel to
  the image plane).

23
Properties of perspective projection (contd)

• Vanishing point
• Parallel lines in space project perspectively
  onto lines that on extension intersect at a
  single point in the image plane called vanishing
  point (or point at infinity).
• The vanishing point of a line depends on the
  orientation of the line and not on the position
  of the line.

Note vanishing points might lie outside of the
image plane!
24
Properties of perspective projection (contd)

• Alternative definition for vanishing point
• The vanishing point of any given line in space is
  located at the point in the image where a
  parallel line through the center of projection
  intersects the image plane.

25
Properties of perspective projection (contd)

• Vanishing line
• The vanishing points of all the lines that lie on
  the same plane form the vanishing line.
• Also defined by the intersection of a parallel
  plane through the center of projection with the
  image plane.

vanishing line
26
Orthographic Projection

• The projection of a 3D object onto a plane by a
  set of parallel rays orthogonal to the image
  plane.
• It is the limit of perspective projection as

27
Orthographic Projection (contd)

• Using matrix notation
• Verify the correctness of the above matrix
  (homogenize using w1)

28
Properties of orthographic projection

• Parallel lines project to parallel lines.
• Size does not change with distance from the
  camera.

29
Weak-perspective projection

• Approximate perspective projection by scaled
  orthographic projection (i.e., linear
  transformation).
• Good approximation if
• (1) the object lies close to the optical axis.
• (2) the objects dimensions are small compared
  to its average distance from the camera

30
Weak perspective projection (contd)

• The term is a scale factor now (e.g.,
  every point is scaled by the same factor).
• Using matrix notation
• Verify - homogenize using

31
What assumptions have we made so far?

• Camera and world coordinate systems have been
  aligned (i.e., all distances are measured in the
  cameras reference frame).
• The origin of the image plane is the principal
  point.

32
World Pixel Coordinates

• In general, world and pixel coordinates are
  related by additional parameters such as
• the position and orientation of the camera
• the focal length of the lens
• the position of the principal point
• the size of the pixels

33
Types of parameters

• Extrinsic the parameters that define the
  location and orientation of the camera reference
  frame with respect to a known world reference
  frame.
• Intrinsic the parameters necessary to link the
  pixel coordinates of an image point with the
  corresponding coordinates in the camera reference
  frame.

34
Types of parameters (contd)
35
Extrinsic camera parameters

• Describe the transformation between the unknown
  camera reference frame and the known world
  reference frame.
• Typically, determining these parameters means
• (1) find the translation
• vector that maps the cameras
• origin to the worlds origin.
• (2) find the rotation matrix
• that aligns the cameras axes
• with the worlds axes.

RT, T
R, -T
36
Extrinsic camera parameters (contd)

• Using the extrinsic camera parameters, we can
  find the relation between the coordinates of a
  point P in world (Pw) and camera (Pc)
  coordinates

37
Extrinsic camera parameters (contd)
or
where RiT corresponds to the i-th row of the
rotation matrix
38
Intrinsic camera parameters

• Characterize the geometric, digital, and optical
  characteristics of the camera
• (1) the perspective projection (focal length f
  ).
• (2) the transformation between image plane
  coordinates and pixel coordinates.
• (3) the geometric distortion introduced by the
  optics.

39
Intrinsic camera parameters
(1) From Camera Coordinates to Image Plane
Coordinates
perspective projection
40
Intrinsic camera parameters (contd)

• (2) From Image Plane Coordinates to Pixel
  coordinates

(ox , oy) are the coordinates of the principal
point e.g., ox N/2, oy M/2 if the principal
point is the center of the image sx , sy
correspond to the effective size of the pixels in
the horizontal and vertical directions (in
millimeters)
41
Intrinsic camera parameters (contd)

• Using matrix notation

42
Intrinsic camera parameters (contd)
(3) Relating pixel coordinates to world
coordinates
or
43
Intrinsic camera parameters (contd)

• Image distortions due to optics

(1) Radial distortion
k1, k2, and k3 are intrinsic parameters
44
Correcting radial distortion
45
Intrinsic camera parameters (contd)

• Image distortions due to optics

(2) Tangential distortion
p1 and p2 are intrinsic parameters
46
Combine extrinsic with intrinsic camera parameters

• The matrix containing the intrinsic camera
  parameters
• (not including distortion parameters for
  simplicity)
• The matrix containing the extrinsic camera
  parameters

47
Combine extrinsic with intrinsic camera
parameters (contd)

• Using homogeneous coordinates
• M is called the projection matrix (i.e., 3 x 4
  matrix).

48
Combine extrinsic with intrinsic camera
parameters (contd)

• Warning homogenization is required to obtain the
  pixel coordinates

49
Perspective projection - revisited

• Assuming ox oy 0 and sx sy 1
• Verify

M ? Mp
Homogenize
v
50
Weak-perspective projection - revisited
M ? Mwp
where is the centroid of the object
(i.e., average distance from the camera)

• Verify

v
Homogenize