Computer Vision 2 mm2

1
Computer Vision 2 mm2

• Agenda
• Model-based Computer Vision
• What is it
• How does it work (brick example)
• What to remember
• Advanced tracking
• Multiple hypotheses tracking
• What to remember

2
Model-based CV

• What is it?
• What can it be used for?

3
Model-based CV According to TBM

• What can it be used for?
• Pose estimation
• Object recognition
• Tracking
• What is it?
• Everything is based on a model
• Contains a geometrical model
• Analysis-by-synthesis approach

4
Model-based CV Characteristics

• Contains a (3D) geometrical model
• Cylinder, ellipsoid, box, truncated cones, etc.
• Brick represented by a box
• Analysis-by-synthesis approach (AbS)
• Project different configurations of the brick
  into the image
• Assume camera calibration
• Compare with image data via similarity measure
• Highest similarity gt pose of brick

5
Analysis-by-Synthesis

• Model representation
• Image representation
• Matching

6
AbS Model Representation

• Model representation state-space representation
• Degrees of freedom (DoF)
• External and internal DoF
• DoF for your brick?
• How would you represent these DoF?

7
AbS Model Representation

• Internal (geometric shape)
• Length, width, height (3 DoF)
• 1, relative width, relative height, scale (3 DoF)
• Relative width and height known, scale (1 DoF)
• Known (0 DoF)
• External (pose)
• CoG, corner Cartesian (3 DoF)
• Angles Around fixed axes, Eulers, ,
  Quaternions, Rodriguezs parameters, Eulers
  parameters, (3 DoF)
• Screw axis representation (helical axis rep.) (6
  DoF )

8
AbS Image Representation

• Image representation
• Edges Contours Silhouettes

9
AbS - Matching

• Compare every possible model configuration (pose
  geometry) with the image data
• Image representation
• Easy
• Matching
• Difficult
• Why is matching difficult?

10
Why is matching so difficult?

• Huge state-space gt too many configurations
• Brick 9 DoF
• Resolution 1mm and 1deg
• Limits
• Internal 0100mm and 0200mm and 0300mm
• External 01000mm and 0360deg
• Size of state-space ( of different
  configurations)
• 100200300100033603 2.81023 !!!!!!!!
• Human skeleton 20 DoF 36020 1051
  infinity
• Brute force whatever!!!
• Search space solution space state-space 1
  DoF

11
What can we do about it? (1)

• Reduce
• Resolution, DoF (measured beforehand)
• Constraints on state-space parameters
• Based on setup and physics
• Based on image pre-processing

12
What can we do about it? (2)

• Assume a smooth and uni-modal
• solution space
• Apply an iterative approach
• Coarser-to-finer search
• Gradient search in solution space
• Other methods exist
• Be aware of local minima!
• After the break.

13
What to remember

• Model-based Computer Vision
• Usage pose estimation, object rec. and tracking
• Geometrical model Cylinders, boxes, ..
• Analysis-by-synthesis approach
• Project model into the image and compare
• Model representation
• State-space representation, degrees-of-freedom
  (DoF)
• Image representation
• Edges, contours, silhouettes
• Matching
• Brute force is not possible!
• Apply constraints and some kind of search strategy

14
Multiple-Hypotheses Tracking

• Why care about this
• Theory
• The principle of factored sampling
• The Condensation algorithm
• Example videos
• What to remember

15
The Condensation algorithm
Posterior at time K-1
Predicted state at time K
Posterior at time K
16
Illustration of Condensation
17
Condensation demos
18
What to remember

• Many DoF gt multi-modal PDF
• We need Multi hypotheses tracking
• Solution Bayes rule p(xz) p(zx) p(x)
• Estimate posterior via Condensation
• Factored sampling over time
• 3 steps sampling, predicting, weighting
• Condensation Particle filter Sequential
• Monte Carlo Multiple hypotheses tracker

19
Implementational issues

• Init The algorithm requires P(x0z0)
• P(x0z0) P(z0x0)
• P(x0z0) uniform density
• P(x0z0) constant density (train off-line)
• Motion Model The more correct the better
• Number of samples N
• Depends on solution space (dim(x) and
  resolution),
• quality of the predictions (motion model and
  process noise), and quality of the measurements
  (measurement noise)
• Fx N100, N1000, N 500-1500
• N can be changed from frame to frame, e.g.
  N(unc.)

20
Tracking multiple objects

• Track one object gt track multiple objects for
  free

21
The Condensation algorithm

• Visual tracking in complex scenes
• Based on Particle Filtering gt Estimation of
  Bayes rule
• General Non-Gaussian densities and/or
• high dimensional problems
• Condensation Conditional Density Propagation

22
The Kalman Filter
Deterministic drift
Stoc. diffusion
Effect of measurements
Model
23
Propagation of densities in the KF

• Gaussian densities. Only 2 parameters mean and
  covar

24
Multi modal densities
25
Chamfer Matching

• Generates a more smooth search space