Digital Camera and Computer Vision Laboratory

1
Chapter 14Analytic Photogrammetry

• Presented by ??? and Dr. Fuh
• R94922103_at_ntu.edu.tw
• 0937384214

2
Analytic Photogrammetry

• Make inferences about
• 3D position
• Orientation
• Length of the observed 3D object parts
• in a world reference frame from
• measurements of one or more 2D-
• perspective projections of a 3D object

3
Analytic Photogrammetry (cont.)

• These inference problems can be construed as
  nonlinear least-square problems
• Iteratively linearize the nonlinear functions
  from an initially given approximate solution

4
Photogrammetry

• Provide a collection of methods for determining
  the position and orientation of cameras and range
  sensors in the scene and relating camera
  positions and range measurements to scene
  coordinates
• GIS Geographic Information System
• GPS Global Positioning System

5
(No Transcript)
6
Exterior Orientation

• Determine position and orientation of camera in
  absolute coordinate system from projections of
  calibration points in scene
• The exterior orientation of the camera is
  specified by all parameters of camera pose, such
  as perspectivity center position, optical axis
  direction.

7
Exterior Orientation (cont.)

• Exterior orientation specification requires 3
  rotation angles, 3 translations

8
(No Transcript)
9
Interior Orientation

• Determine internal geometry of camera
• The interior orientation of camera is specified
  by all the parameters that determines the
  geometry of 3D rays from measured image
  coordinates

10
Interior Orientation (cont.)

• The parameters of interior orientation relate the
  geometry of ideal perspective projection to the
  physics of a camera.
• Parameters camera constant, principal point,
  lens distortion,

11
Interior Orientation (cont.)

• With interior and external orientation, we can
  complete specify the camera orientation.

12
Relative Orientation

• Determine relative position and orientation
  between 2 cameras from projections of calibration
  points in scene
• Calibrate relation between two cameras for stereo
• Relates coordinate systems of two cameras to each
  other, not knowing 3D points themselves, only
  their projections in image

13
Relative Orientation (cont.)

• Assume interior orientation of each camera known
• Specified by 5 parameters 3 rotation angles, 2
  translations

14
(No Transcript)
15
Absolute Orientation

• Determine transformation between 2 coordinate
  systems or position and orientation of range
  sensor in absolute coordinate system from
  coordinates of calibration points
• Convert depth measurements in viewer-centered
  coordinates to absolute coordinate system for the
  scene

16
Absolute Orientation (cont.)

• Orientation of stereo model in world reference
  frame
• Determine scale, 3 translations, 3 rotations
• Recovery of relation between two coordinate system

17
(No Transcript)
18
Symbol Definition
19
Rotation Matrix
20
Rotation Matrix (cont.)
21
Rotation Matrix (cont.)
22
World Frame to Camera Frame

• (x, y, z) in world frame represented by
• (p, q, s) in camera frame

23
Pinhole Camera Projection

• Pinhole camera with image at distance f from
  camera lens, projection
• where f is a camera constant, related to focal
  length of lens

24
Principal Point

• Origin of measurement image plane coordinate
• Represented by (u0, v0)

25
Perspective Projection Equations

• Collinearity equation

26
Perspective Projection Equations (cont.)

• Show that the relationship between the measured
  2D-perspective projection coordinates and the 3D
  coordinates is a nonlinear function of u0, v0,
  x0, y0, z0, ?, ?, and ?

27
Take a Break
28
Nonlinear Least-Square Solutions

• Noise model

29
Nonlinear Least-Square Solutions (cont.)

• Maximum likelihood solution ß1, , ßM maximize
  Prob(a1, , ak ß1, , ßM )
• In other words, this solution minimizes
  least-squares criterion
• where

30
First-Order Taylor Series Expansion

• First-order Taylor series expansion of gk taken
  around ßt

31
First-Order Taylor Series Expansion (cont.)
32
Exterior Orientation Problem

• Determine the unknown rotation and translation
  that put the camera reference frame in the world
  reference frame.

33
Exterior Orientation Problem (cont.)
34
One Camera Exterior Orientation Problem

• Known (xn, yn, zn) and (un, vn)
• (un, vn) is the corresponding set of
  2D-perspective projections, n 1, , N
• Unknown (?,?,?) and (x0, y0, z0)

35
Other Exterior Orientation Problem

• Camera calibration problem unknown position of
  camera in object frame
• Object pose estimation problem unknown object
  position in camera frame
• Spatial resection problem in photogrammetries 3D
  positions from 2D orientation

36
Nonlinear Transformation For Exterior Orientation
37
Standard Solution

• By chain rule,

38

• In matrix form,

39
(No Transcript)
40
Standard Solution (cont.)
41
Auxiliary Solution

• Not iteratively adjust the angles directly
• Reorganize the calculation such that we
  iteratively adjust the three auxiliary parameters
  of a skew symmetric matrix associated with the
  rotation matrix
• Then, we determine the adjustment of the angles

42
(No Transcript)
43
Quaternion Representation

• From any skew symmetric matrix,
• we can construct a rotation matrix R by
  choosing scalar d R (dI S)(dI - S)-1
• which guarantees that RR I

44
Quaternion Representation (cont.)

• Expanding the equation for R
• parameters a, b, c, d can be constrained to
  satisfy a2 b2 c2 d2 1

45
Quaternion Representation (cont.)
46
Take a Break
47
Relative Orientation

• The transformation from one camera station to
  another can be represented by a rotation and a
  translation
• The relation between the coordinates, rl and rr
  of a point P can be given by means of a rotation
  matrix and an offset vector

48
(No Transcript)
49
Relative Orientation (cont.)

• Relative orientation is typically with the
  determination of the position and orientation of
  one photograph with respect to another, given a
  set of corresponding image points

50
Relative Orientation (cont.)

• Relative orientation specified by five
  parameters (yR - yL), (zR - zL), (?R - ?L),
• (?R - ?L), (?R - ?L)
• Assumption
• Camera interior orientation known
• Image positions expressed to identical scale and
  with respect to principal point

51
Standard Solution

• Let QL and QR be the rotation matrices with the
  exterior orientation of the left and the right
  image

52
Standard Solution (cont.)

• fR distance between right image plane and right
  lens
• fL distance between left image plane and left
  lens
• From perspective collinearity equation

53
Standard Solution (cont.)

• Hence,
• where

54
Quaternion Solution

• Instead of determining the relative orientation
  of the right image with respect to the left
  image, we aligns a reference frame having its
  x-axis along the line from the left image lens to
  the right image lens

55
Quaternion Solution (cont.)

• The relative orientation is then determined by
  the angles (?R, ?R, ?R), which rotate the right
  image into this reference frame, and the angles
  (?L, ?L, ?L), which rotate the left image into
  this reference frame

56
Interior Orientation

• A camera is specified by
• Camera constant f distance between image plane
  and camera lens
• Principal point (up, vp) intersection of optic
  axis with image plane in measurement reference
  frame located on image plane
• Geometric distortion characteristics of the lens
  assuming isotropic around the principal point

57
(No Transcript)
58
Stereo

• Optical axes parallel to one another and
  perpendicular to baseline simple camera geometry
  for stereo photography

59
(No Transcript)
60
(No Transcript)
61
Stereo (cont.)

• Parallax deplacement in perspective projection
  by position translation
• (x, y, z) 3D point position
• (uL, vL) perspective projection on left image
  of stereo pair
• (uR, vR) perspective projection on right image
  of stereo pair
• bx baseline length in x-axis

62
Stereo (cont.)
63
Stereo (cont.)
64
(No Transcript)
65
Stereo (cont.)

• Relation is close to being
  useless in real-world, because
• Observed perspective projections are subject to
  measurement errors so that vL ? vR for
  corresponding points
• Left and right camera frames may have slightly
  different orientations
• When two cameras used, almost always fR ? fL

66
Take a Break
67
Relationship Between Coordinate System

• The relationship between two coordinate systems
  is easy to find if we can measure the coordinates
  of a number of points in both systems

68
Relationship Between Coordinate System(cont.)

• It takes three measurements to tie two coordinate
  systems together uniquely
• A single measurement leaves three degrees of
  freedom motion
• A second measurement removes all but one degree
  of freedom
• Third measurement rigidly attaches two coordinate
  systems to each other

69
(No Transcript)
70
2D-2D Pose Detection Problem

• Determine from matched points more precise
  estimate of rotation matrix R and translation t
  such that yn Rxn t, n 1, , N
• Determine R and t that minimize weighted sum of
  residual errors

71
3D-3D Absolute Orientation

• We must determine rotation matrix R and
  translation vector t satisfying
• Constrained least-squares problem to minimize

72
3D-3D Absolute Orientation (cont.)

• The least-square problem can be modeled by a
  mechanical system in which corresponding points
  in the two coordinate systems are attached to
  each other by means of springs
• The solution to the least-squares problem
  corresponds to the equilibrium position of the
  system, which minimizes the energy stored in the
  springs

73
(No Transcript)
74
Robust M-Estimation

• Least-squares techniques are ideal when random
  data perturbations or measurement errors are
  Gaussian distribution
• We need some robust techniques for nonlinear
  regression

75
Robust M-Estimation (cont.)

• M-Estimator

76
Robust M-Estimation (cont.)
or
77
Robust M-Estimation (cont.)

• ?
• Symmetric
• Positive-defined function
• Has unique minimum at zero
• Chosen to be less increasing than square

78
Robust M-Estimation (cont.)
79
Error Propagation

• If we have the input parameter x1, , xN , and
  random errors ?x1, , ?xN , the quantity y
  depends on input parameters through known
  function f y f(x1, , xN ) will become
• y ?y f(x1 ?x1, , xN ?xN )

80
Error Propagation Analysis

• Determines expected value and variance of
• y ?y
• Known information about ?x1, , ?xN mean and
  variance

81
Implicit Form

• A known function f has the form
• f(x1, , xN, y) 0
• The quantities (x1 ?x1, , xN ?xN ) are
  observed, and the quantity y ?y is determined
  to satisfy
• f(x1 ?x1, , xN ?xN , y ?y ) 0

82
Implicit Form General Case

• General case y is not a scalar but a L 1
  vector ß
• x1, , xN are K N 1 vectors representing true
  values
• x1 ?x1, , xK ?xK are K N 1 vectors
  representing noisy observed values
• ?x1, , ?xK random perturbations
• ß a L 1 vector representing unknown true
  parameters

83
Implicit Form General Case

• Noiseless model
• With noisy observations, the idealized model

84
Summary

• We have shown how to
• Take a nonlinear least-squares problem
• Linearize it
• Solve by iteratively solving successive
  linearized least-squares problems