Analytical Photogrammetry

1
Analytical Photogrammetry
Hande Demirel, PhD, Assistant Prof. Istanbul
Technical University (ITU), Faculty of Civil
Engineering, Geodesy and Photogrammetry
Department Division of Photogrammetry
2
Analytic Photogrammetry

• Make inferences about
• 3D position
• Orientation
• Length of the observed 3D object parts
• in a world reference frame from
• measurements of one or more 2D-
• perspective projections of a 3D object

3
Analytic Photogrammetry (cont.)

• These inference problems can be construed as
  nonlinear least-square problems
• Iteratively linearize the nonlinear functions
  from an initially given approximate solution

4
Photogrammetry

• Provide a collection of methods for determining
  the position and orientation of cameras and range
  sensors in the scene and relating camera
  positions and range measurements to scene
  coordinates
• GIS Geographic Information System
• GPS Global Positioning System

5
(No Transcript)
6
Basic Geometric Elements (vertical photo)
7
Photogrammetric Coverage
8
Parallax
Parallax the apparent change in relative
positions of stationary objects caused by a
change in viewing position
9
Analytic Photogrammetry
L
L
B (Base)
f
f
b
a
xa
xa
Image plane
o
a
o
b
H
paxa xa
OA
OA
hAH-Bf/pa
A
OB
OB
Similary, hBH-Bf/pb
B
hA
hB
hA-hBBf(1/pb-1/pa)
10
Interior Orientation

• Determine internal geometry of camera
• The interior orientation of camera is specified
  by all the parameters that determines the
  geometry of 3D rays from measured image
  coordinates

11
Interior Orientation (cont.)

• The parameters of interior orientation relate the
  geometry of ideal perspective projection to the
  physics of a camera.
• Parameters camera constant, principal point,
  lens distortion,

12
Interior Orientation (cont.)
13
Photo Coordinate System
14
Interior Orientation (cont.)

• With interior and external orientation, we can
  complete specify the camera orientation.

15
Exterior Orientation
16
Mathematical relation between image and ground
coordinates

17
Exterior Orientation

• Determine position and orientation of camera in
  absolute coordinate system from projections of
  calibration points in scene
• The exterior orientation of the camera is
  specified by all parameters of camera pose, such
  as perspectivity center position, optical axis
  direction.

18
Exterior Orientation (cont.)

• Exterior orientation specification requires 3
  rotation angles, 3 translations

19
(No Transcript)
20
Relative Orientation

• Determine relative position and orientation
  between 2 cameras from projections of calibration
  points in scene
• Calibrate relation between two cameras for stereo
• Relates coordinate systems of two cameras to each
  other, not knowing 3D points themselves, only
  their projections in image

21
Relative Orientation (cont.)

• Assume interior orientation of each camera known
• Specified by 5 parameters 3 rotation angles, 2
  translations

22
(No Transcript)
23
Absolute Orientation

• Determine transformation between 2 coordinate
  systems or position and orientation of range
  sensor in absolute coordinate system from
  coordinates of calibration points
• Convert depth measurements in viewer-centered
  coordinates to absolute coordinate system for the
  scene

24
Absolute Orientation (cont.)

• Orientation of stereo model in world reference
  frame
• Determine scale, 3 translations, 3 rotations
• Recovery of relation between two coordinate system

25
(No Transcript)
26
Symbol Definition
27
Rotation Matrix
28
Rotation Matrix (cont.)
29
Rotation Matrix (cont.)
30
World Frame to Camera Frame

• (x, y, z) in world frame represented by
• (p, q, s) in camera frame

31
Pinhole Camera Projection

• Pinhole camera with image at distance f from
  camera lens, projection
• where f is a camera constant, related to focal
  length of lens

32
Principal Point

• Origin of measurement image plane coordinate
• Represented by (u0, v0)

33
Perspective Projection Equations

• Collinearity equation

34
Perspective Projection Equations (cont.)

• Show that the relationship between the measured
  2D-perspective projection coordinates and the 3D
  coordinates is a nonlinear function of u0, v0,
  x0, y0, z0, ?, ?, and ?

35
Nonlinear Least-Square Solutions

• Noise model

36
Nonlinear Least-Square Solutions (cont.)

• Maximum likelihood solution ß1, , ßM maximize
  Prob(a1, , ak ß1, , ßM )
• In other words, this solution minimizes
  least-squares criterion
• where

37
First-Order Taylor Series Expansion

• First-order Taylor series expansion of gk taken
  around ßt

38
First-Order Taylor Series Expansion (cont.)
39
Exterior Orientation Problem

• Determine the unknown rotation and translation
  that put the camera reference frame in the world
  reference frame.

40
Exterior Orientation Problem (cont.)
41
One Camera Exterior Orientation Problem

• Known (xn, yn, zn) and (un, vn)
• (un, vn) is the corresponding set of
  2D-perspective projections, n 1, , N
• Unknown (?,?,?) and (x0, y0, z0)

42
Other Exterior Orientation Problem

• Camera calibration problem unknown position of
  camera in object frame
• Object pose estimation problem unknown object
  position in camera frame
• Spatial resection problem in photogrammetries 3D
  positions from 2D orientation

43
Nonlinear Transformation For Exterior Orientation
44
Standard Solution

• By chain rule,

45

• In matrix form,

46
(No Transcript)
47
Standard Solution (cont.)
48
Auxiliary Solution

• Not iteratively adjust the angles directly
• Reorganize the calculation such that we
  iteratively adjust the three auxiliary parameters
  of a skew symmetric matrix associated with the
  rotation matrix
• Then, we determine the adjustment of the angles

49
(No Transcript)
50
Quaternion Representation

• From any skew symmetric matrix,
• we can construct a rotation matrix R by
  choosing scalar d R (dI S)(dI - S)-1
• which guarantees that RR I

51
Quaternion Representation (cont.)

• Expanding the equation for R
• parameters a, b, c, d can be constrained to
  satisfy a2 b2 c2 d2 1

52
Quaternion Representation (cont.)
53
Relative Orientation

• The transformation from one camera station to
  another can be represented by a rotation and a
  translation
• The relation between the coordinates, rl and rr
  of a point P can be given by means of a rotation
  matrix and an offset vector

54
(No Transcript)
55
Relative Orientation (cont.)

• Relative orientation is typically with the
  determination of the position and orientation of
  one photograph with respect to another, given a
  set of corresponding image points

56
Relative Orientation (cont.)

• Relative orientation specified by five
  parameters (yR - yL), (zR - zL), (?R - ?L),
• (?R - ?L), (?R - ?L)
• Assumption
• Camera interior orientation known
• Image positions expressed to identical scale and
  with respect to principal point

57
Standard Solution

• Let QL and QR be the rotation matrices with the
  exterior orientation of the left and the right
  image

58
Standard Solution (cont.)

• fR distance between right image plane and right
  lens
• fL distance between left image plane and left
  lens
• From perspective collinearity equation

59
Standard Solution (cont.)

• Hence,
• where

60
Quaternion Solution

• Instead of determining the relative orientation
  of the right image with respect to the left
  image, we aligns a reference frame having its
  x-axis along the line from the left image lens to
  the right image lens

61
Quaternion Solution (cont.)

• The relative orientation is then determined by
  the angles (?R, ?R, ?R), which rotate the right
  image into this reference frame, and the angles
  (?L, ?L, ?L), which rotate the left image into
  this reference frame

62
Interior Orientation

• A camera is specified by
• Camera constant f distance between image plane
  and camera lens
• Principal point (up, vp) intersection of optic
  axis with image plane in measurement reference
  frame located on image plane
• Geometric distortion characteristics of the lens
  assuming isotropic around the principal point

63
(No Transcript)
64
Stereo

• Optical axes parallel to one another and
  perpendicular to baseline simple camera geometry
  for stereo photography

65
Stereo (cont.)

• Parallax deplacement in perspective projection
  by position translation
• (x, y, z) 3D point position
• (uL, vL) perspective projection on left image
  of stereo pair
• (uR, vR) perspective projection on right image
  of stereo pair
• bx baseline length in x-axis

66
Stereo (cont.)
67
Stereo (cont.)
68
(No Transcript)
69
Stereo (cont.)

• Relation is close to being
  useless in real-world, because
• Observed perspective projections are subject to
  measurement errors so that vL ? vR for
  corresponding points
• Left and right camera frames may have slightly
  different orientations
• When two cameras used, almost always fR ? fL

70
Relationship Between Coordinate System

• The relationship between two coordinate systems
  is easy to find if we can measure the coordinates
  of a number of points in both systems

71
Relationship Between Coordinate System(cont.)

• It takes three measurements to tie two coordinate
  systems together uniquely
• A single measurement leaves three degrees of
  freedom motion
• A second measurement removes all but one degree
  of freedom
• Third measurement rigidly attaches two coordinate
  systems to each other

72
(No Transcript)
73
2D-2D Pose Detection Problem

• Determine from matched points more precise
  estimate of rotation matrix R and translation t
  such that yn Rxn t, n 1, , N
• Determine R and t that minimize weighted sum of
  residual errors

74
3D-3D Absolute Orientation

• We must determine rotation matrix R and
  translation vector t satisfying
• Constrained least-squares problem to minimize

75
3D-3D Absolute Orientation (cont.)

• The least-square problem can be modeled by a
  mechanical system in which corresponding points
  in the two coordinate systems are attached to
  each other by means of springs
• The solution to the least-squares problem
  corresponds to the equilibrium position of the
  system, which minimizes the energy stored in the
  springs

76
(No Transcript)
77
Robust M-Estimation

• Least-squares techniques are ideal when random
  data perturbations or measurement errors are
  Gaussian distribution
• We need some robust techniques for nonlinear
  regression

78
Robust M-Estimation (cont.)

• M-Estimator

79
Robust M-Estimation (cont.)
or
80
Robust M-Estimation (cont.)

• ?
• Symmetric
• Positive-defined function
• Has unique minimum at zero
• Chosen to be less increasing than square

81
Robust M-Estimation (cont.)
82
Error Propagation

• If we have the input parameter x1, , xN , and
  random errors ?x1, , ?xN , the quantity y
  depends on input parameters through known
  function f y f(x1, , xN ) will become
• y ?y f(x1 ?x1, , xN ?xN )

83
Error Propagation Analysis

• Determines expected value and variance of
• y ?y
• Known information about ?x1, , ?xN mean and
  variance

84
Implicit Form

• A known function f has the form
• f(x1, , xN, y) 0
• The quantities (x1 ?x1, , xN ?xN ) are
  observed, and the quantity y ?y is determined
  to satisfy
• f(x1 ?x1, , xN ?xN , y ?y ) 0

85
Implicit Form General Case

• General case y is not a scalar but a L 1
  vector ß
• x1, , xN are K N 1 vectors representing true
  values
• x1 ?x1, , xK ?xK are K N 1 vectors
  representing noisy observed values
• ?x1, , ?xK random perturbations
• ß a L 1 vector representing unknown true
  parameters

86
Implicit Form General Case

• Noiseless model
• With noisy observations, the idealized model

87
Summary

• We have shown how to
• Take a nonlinear least-squares problem
• Linearize it
• Solve by iteratively solving successive
  linearized least-squares problems