Computer Vision lecture 4

1
Computer Vision lecture 4

• Agenda
• Active Shape Model (ASM)
• Advanced tracking
• KLT-tracker

2
Active Shape Model (ASM)

• 2nd lecture Geometric model
• Pose estimation
• Tracking (pose estimation over time)
• Object recognition
• What if the shape of the model can change in
  complex ways?
• Biological objects have random shapes within
  certain parameters
• Example Cactus

3
ASM

• Much hand labeling is done in e.g., medical
  applications because its hard to do automatically
• In the beginning of the 1990s Tim Cootes et. al.
  from Manchester University developed ASM for
  medical applications
• Segment the object (e.g., bones) automatically
  using an Active Shape Model (ASM)

4
ASM

• The overall idea
• Off-line Training
• Learn the statistics regarding the shape of the
  object
• On-line Segmentation
• Iterative Analysis-by-synthesis approach
• Iteration
• Project current model (shape and pose par.) into
  the image
• Compare projected shape with the image data
  (Error measure)
• Is error small? (Converged?)
• Otherwise Use the error to choose different
  model parameters within the allowed statistics
• Requires an initial model guess (shape and pose
  parameters)

5
ASM

• Ill use a concrete example
• Pose Estimation of Cactus Leaves
• Student project CVG9 2003
• The problem posed by a company
• Last process to automate
• Can ASM solve the problem?
• Solution
• Init. guess given
• Assumes the leaves are 2D objects (flat)
• ASM finds the four top spikes
• TM to find the four spikes in a second (cali.)
  camera
• Triangulation gt four 3D points gt pose estimation

6
ASM

• First Off-line
• Then On-line

7
ASM Off-line
Model training
Data alignment
Data extraction
Calculating data statistic
8
ASM Offline - Data extraction

• Trainings image
• Contains images of all
• the likely shapes of leaves
• Light box with
• white paper
• 50 images
• 200 leaves

9
ASM Offline - Data extraction

• Leaf segmentation
• Threshold
• Minimize within-group variance

10
ASM Offline - Data extraction

• Leaf segmentation
• Threshold
• Minimize within-group variance
• Segment 4 biggest blobs

11
ASM Offline - Data extraction

• Leaf segmentation
• Threshold
• Minimize within-group variance
• Segment 4 biggest blobs
• Find edges

12
ASM Offline - Data extraction

• Leaf segmentation
• Threshold
• Minimize within-group variance
• Segment 4 biggest blobs
• Find edges
• Combine edge and blob image
• Show to the operator

13
ASM Offline - Data extraction

• Placing landmarks
• Manually placing 13 landmarks
• Ensure similar representation

14
ASM Offline - Data extraction

• Placing landmarks
• Manually placing 13 landmarks
• Ensure similar representation
• Placing intermediate points
• Automatic edge detection
• of edges between landmarks
• Equal distance
• Number of points found
• experimental 78
• Points in total 91

15
ASM Offline
Model training
Data alignment
Data extraction
Calculating data statistic
16
ASM Offline - Data alignment

• What we want
• A model that can represent all different shapes
• How do we get that
• Method Generalized Procrustes Analysis
• Data representation
• One leaf 91 points
• Represented as a vector with 182 elements
• x x1,x2,,x91,y1,y2,,y91

17
ASM Offline - Data alignment

• Generalized Procrustes Analysis
• Align all leaves wrt. position, size, and
  rotation
• Then only the variations in shape remain

1. Step Position normalisation
Position, size and rotation normalisation
18
ASM Offline - Data alignment

• Generalized Procrustes Analysis iteration
• Calculate mean shape
• Align (by rotating and scaling) all training
  shapes to the mean shape
• Converged? (The mean shape has not changed
  significantly)

Position, size and rotation normalisation
Final mean
First mean first leaf (scaled to 1)
19
ASM Offline
Model training
Data alignment
Data extraction
Calculating data statistics
20
ASM Offline - Data statistics

• The 182 elements are highly correlated,
• hence we can represent them much more
• compactly
• Principal Component Analysis (PCA)
• Reduces the number of parameters
• from 182 to 14
• Models 98.16 of the variation in the training
  data
• The shape of the model is given by

21
ASM Offline - Data statistics

• Point Distribution Model (PDM)
• A model that represents the different shape
  variations
• So by changing we can generate all leaf shapes

22
ASM Offline - Data statistics
(Video - divx)
23
ASM On-line
ASM On-line
24
ASM - Online
Image preprocessing
Locating Object
Calculate initial pose parameters
Low level preprocessing
Project model into image and compare with image
data
Correct model according to image data

• Preprocessing Saturate interior
• to avoid interior edges
• Pose parameters 2D translation,
• scale, image rotation

25
ASM Online - Project

• Given the current model parameters state
• Shape parameters and pose parameters
• Init
• Pose init guess
• Shape mean shape
• Project the state into
• the image
• Change green shape
• until it fits images data

26
ASM Online - Compare

• Search along the normals
• Find best edge with correct direction
• New points image shape

27
ASM - Online
Image preprocessing
Locating Object
Calculate initial pose parameters
Low level preprocessing
Project model into image and compare with image
data
Correct model according to image data
28
ASM Offline Correct model

• Find the pose parameters which best align the
  image shape with the mean shape (in model space)
• Least square
• We know the correspondences between points

29
ASM Offline Correct model

• Transform image shape to model space using the
  inverse pose parameters just found
• Translation
• Scale
• Rotation
• Mean shape

30
ASM Offline Correct model

• Calculate the shape transformation parameters
• which best explain the shape difference
• Shape variation from mean
• Map into 14D

31
ASM - Online
Image preprocessing
Locating Object
Calculate initial pose parameters
Low level preprocessing
Project model into image and compare with image
data
Correct model according to image data
Iterate until the shape parameters ( ) have
converged
32
ASM - Online
Init.
33
ASM - Online
Init.
34
ASM - Online
Init.
35
ASM - Online
Init.
36
ASM - Online
Init.

• Least square method
• - Correspondences between points known

37
ASM - Online
Init.
38
ASM - Online
Init.
39
ASM - Online
Init.
40
ASM - Online
Init.

• Converged?
• If error above Threshold go on
• - Set bold b
• Else stop

41
ASM Online - Align model shape
Init. Pose and shape
42
ASM Online - Align model shape
Init. Pose and shape
43
ASM Online - Align model shape
Init. Pose and shape
44
ASM Online - Align model shape
Init. Pose and shape
45
ASM Online - Align model shape
Init. Pose and shape
46
ASM Online - Align model shape
Init. Pose and shape
(Video - divx)
47
What to remember

• ASM Active Shape Model
• Good for biological objects, but
• Requires initial pose
• Requires manual labeling
• Off-line Align training data (translation,
  scale, rotation)
• Learn the statistics regarding the shape of the
  object
• Point distribution model
• On-line Segmentation
• Iteration
• Project current model (shape and pose par.) into
  the image
• Compare projected shape with the image data
  (Error measure)
• Is error small? (Converged?)
• Otherwise Use the error to choose a different
  model

48
ASM - Forbedringer

• Affine transformation
• Simulerer 3D pose af blad
• Seks frihedsgrader frem for fire
• Bedre alignment

49
ASM - Forbedringer

• Variabel opløsning
• Gaussisk pyramide
• Tre niveauer
• Aligner på groveste niveau først
• Færre Iterationer

50
ASM Offline - Data statistics
51
ASM Offline - Data statistics
52
ASM Offline - Data statistics