Perspective Structure from Motion

1
PerspectiveStructure from Motion

2
Orthographic ? Perspective

• Last time, considered weak perspective
• With full perspective, can recover more
  information (translation along optical axis)
• Result can recover geometry and full motion up
  to global scale factor

3
Perspective SFM Methods

• Bundle adjustment (full nonlinear minimization)
• Methods based on factorization
• Methods based on fundamental matrices
• Methods based on vanishing points

4
Motion Field for Camera Motion

• Translation
• Motion field lines converge (possibly at ?)

5
Motion Field for Camera Motion

• Rotation
• Motion field lines do not converge

6
Motion Field for Camera Motion

• Combined rotation and translationmotion field
  lines have component that converges, and
  component that does not
• Algorithms can look for vanishing point,then
  determine component of motion around this point
• Focus of expansion / contraction
• Instantaneous epipole

7
SFM Algorithm

• Compute optical flow
• Find vanishing point (least squares solution)
• Find direction of translation from epipole
• Find perpendicular component of motion
• Find velocity, axis of rotation
• Find depths of points (up to global scale)

8
Shape from Shading and Texture

9
Lambertian Reflectance Model

• Diffuse surfaces appear equally bright from all
  directions
• For point illumination, brightness proportional
  to cos q

10
Lambertian Reflectance Model

• Therefore, for a constant-colored object with
  distant illumination, can write E L r l?nE
  observed brightnessL brightness of light
  sourcer reflectance (albedo) of surfacel
  direction to light sourcen surface normal

11
Shape from Shading

• The above equation contains some information
  about shape, and in some cases is enough to
  recover shape completely(in theory) if Lr and l
  are known
• Similar to integration (surface normal is like a
  derivative), but only know a part of derivative
• Have to assume surface continuity
• No solution in the presence of noise

12
Variational Shape from Shading

• Approach energy minimization
• Given observed E(x,y), find shape z(x,y)that
  minimizes energywhere

13
Variational Shape from Shading

• Solve by techniques from calculus of variations
• Use Euler-Lagrange equations to get a PDE, solve
  numerically
• Unlike with snakes, greedy methods tend not to
  work

14
Difficulties with Shape from Shading

• How to find L, r, l?
• Estimate based on scene statistics
• Shadows
• Non-Lambertian (e.g., specular) surfaces
• More than 1 light, or diffuse illumination
• Interreflections

15
Shape from Shading Results
Trucco Verri
16
Shape from Shading Results
17
Active Shape from Shading

• Idea several (user-controlled) light sources
• More data
• Allows determining surface normal directly
• Allows spatially-varying reflectance
• Redundant measurements discard shadows and
  specular highlights
• Often called photometric stereo

18
Photometric Stereo Setup
Rushmeier et al., 1997
19
Photometric Stereo Math

• For each point p, can write
• Constant a incorporates light source brightness,
  camera sensitivity, etc.

20
Photometric Stereo Math

• Solving above equation gives (r /a) n
• n must be unit-length ? uniquely determined
• Determine r up to global constant
• With more than 3 light sources
• Discard highest and lowest measurements
• If still more, solve by least squares

21
Photometric Stereo Results
Recovered normals (re-lit)
Inputimages
Recovered color
Rushmeier et al., 1997
22
Texture

• Texture repeated pattern on a surface
• Elements (textons) either identical or come
  from some statistical distribution
• Shape from texture comes from looking at
  deformation of individual textons or from
  distribution of textons on a surface

23
Shape from Texture

• Much the same as shape from shading, but have
  more information
• Foreshortening gives surface normal (not just
  one component, as in shape from shading)
• Perspective distortion gives information about
  depth directly
• Sparse depth information (only at textons)
• About the same as shape from shading, because of
  smoothness term in energy eqn.

24
Shape from Texture Results
Forsyth