Computer Vision: Vision and Modeling

1
Computer Vision Vision and Modeling
2
Computer Vision Vision and Modeling

• Lucas-Kanade Extensions
• Support Maps / Layers
• Robust Norm, Layered Motion, Background
  Subtraction, Color Layers
• Statistical Models (ForsythPonce Chap. 6,
  DudaHartStork Chap. 1-5)
• - Bayesian Decision Theory
• - Density Estimation

3
A Different View of Lucas-Kanade
2
E S ( )
I (i) - I(i) v
D
i
t
i
2
I (1) - I(1) v
D
1
t
High Gradient has Higher weight
I (2) - I(2) v
D
2
t
...
D
I (n) - I(n) v
n
t
? White board
4
Constrained Optimization
2
I (1) - I(1) v
D
1
t
I (2) - I(2) v
D
2
t
...
D
I (n) - I(n) v
n
t
5
Constraints Subspaces
Constrain
-
V
V
E(V)
Analytically derived Affine / Twist/Exponential
Map
Learned Linear/non-linear Sub-Spaces
6
Motion Constraints

• Optical Flow local constraints

• Region Layers rigid/affine constraints

• Articulated kinematic chain constraints

• Nonrigid implicit / learned constraints

7
Constrained Function Minimization
Constrain
-
V
V
2
I (1) - I(1) v
D
1
t
V M( q )
E(V)
I (2) - I(2) v
D
2
t
...
D
I (n) - I(n) v
n
t
8
2D Translation Lucas-Kanade
2D
Constrain
-
V
V
2
I (1) - I(1) v
D
dx, dy
1
t
V
E(V)
dx, dy
I (2) - I(2) v
D
2
t
...
...
dx, dy
D
I (n) - I(n) v
n
t
9
2D Affine Bergen et al, Shi-Tomasi
6D
Constrain
-
V
V
2
I (1) - I(1) v
D
1
t
dx
x
v
E(V)
i

I (2) - I(2) v
D
dy
y
i
2
t
i
...
D
I (n) - I(n) v
n
t
10
Affine Extension

• Affine Motion Model
• 2D Translation
• 2D Rotation
• Scale in X / Y
• Shear

Matlab demo -gt
11
Affine Extension
Affine Motion Model -gt Lucas-Kanade
Matlab demo -gt
12
2D Affine Bergen et al, Shi-Tomasi
6D
Constrain
-
V
V
13
K-DOF Models
K-DOF
Constrain
-
V
V
2
I (1) - I(1) v
D
1
t
V M( q )
E(V)
I (2) - I(2) v
D
2
t
...
D
I (n) - I(n) v
n
t
14
Quadratic Error Norm (SSD) ???
Constrain
-
V
V
2
I (1) - I(1) v
D
1
t
V M( q )
E(V)
I (2) - I(2) v
D
2
t
...
D
I (n) - I(n) v
n
t
? White board (outliers?)
15
Support Maps / Layers

• L2 Norm vs Robust Norm
• Dangers of least square fitting

L2
D
16
Support Maps / Layers

• L2 Norm vs Robust Norm
• Dangers of least square fitting

L2
robust
D
D
17
Support Maps / Layers

• Robust Norm -- good for outliers
• nonlinear optimization

robust
D
18
Support Maps / Layers

• Iterative Technique

Add weights to each pixel eq (white board)
19
Support Maps / Layers

• how to compute weights ?
• -gt previous iteration how good does G-warp
  matches F ?
• -gt probabilistic distance Gaussian

20
Error Norms / Optimization Techniques
SSD Lucas-Kanade (1981) Newton-Raphson SSD
Bergen-et al. (1992) Coarse-to-Fine SSD
Shi-Tomasi (1994) Good Features Robust Norm
Jepson-Black (1993) EM Robust Norm
Ayer-Sawhney (1995) EM MRF MAP
Weiss-Adelson (1996) EM MRF ML/MAP
Bregler-Malik (1998) Twists / EM ML/MAP
Irani (Ananadan) (2000) SVD
21
Computer Vision Vision and Modeling

• Lucas-Kanade Extensions
• Support Maps / Layers
• Robust Norm, Layered Motion, Background
  Subtraction, Color Layers
• Statistical Models (ForsythPonce Chap. 6,
  DudaHartStork Chap. 1-5)
• - Bayesian Decision Theory
• - Density Estimation

22
Support Maps / Layers

• Black-Jepson-95

23
Support Maps / Layers

• More General Layered Motion (Jepson/Black,
  Weiss/Adelson, )

24
Support Maps / Layers

• Special Cases of Layered Motion
• - Background substraction
• - Outlier rejection ( robust norm)
• - Simplest Case Each Layer has uniform color

25
Support Maps / Layers

• Color Layers

P(skin F(x,y))
26
Computer Vision Vision and Modeling

• Lucas-Kanade Extensions
• Support Maps / Layers
• Robust Norm, Layered Motion, Background
  Subtraction, Color Layers
• Statistical Models (DudaHartStork Chap. 1-5)
• - Bayesian Decision Theory
• - Density Estimation

27
Statistical Models / Probability Theory

• Statistical Models Represent Uncertainty and
  Variability
• Probability Theory Proper mechanism for
  Uncertainty
• Basic Facts ? White Board

28
General Performance Criteria

• Optimal Bayes
• With Applications to Classification

29
Bayes Decision Theory

• Example Character Recognition
• Goal Classify new character in a way as to
• minimize probability of misclassification

30
Bayes Decision Theory

• 1st Concept Priors

?
P(a)0.75 P(b)0.25
a a b a b a a b a b a a a a b a a b a a b a a a a
b b a b a b a a b a a
31
Bayes Decision Theory

• 2nd Concept Conditional Probability

black pixel
black pixel
32
Bayes Decision Theory

• Example

X7
33
Bayes Decision Theory

• Example

X8
34
Bayes Decision Theory

• Example

Well P(a)0.75 P(b)0.25
X8
35
Bayes Decision Theory

• Example

P(a)0.75 P(b)0.25
X9
36
Bayes Decision Theory

• Bayes Theorem

37
Bayes Decision Theory

• Bayes Theorem

38
Bayes Decision Theory

• Bayes Theorem

Likelihood x prior
Posterior
Normalization factor
39
Bayes Decision Theory

• Example

40
Bayes Decision Theory

• Example

41
Bayes Decision Theory

• Example

Xgt8 class b
42
Bayes Decision Theory

• Goal Classify new character in a way as to
• minimize probability of misclassification
• Decision boundaries

43
Bayes Decision Theory

• Goal Classify new character in a way as to
• minimize probability of misclassification
• Decision boundaries

44
Bayes Decision Theory

• Decision Regions

R3
R1
R2
45
Bayes Decision Theory

• Goal minimize probability of misclassification

46
Bayes Decision Theory

• Goal minimize probability of misclassification

47
Bayes Decision Theory

• Goal minimize probability of misclassification

48
Bayes Decision Theory

• Goal minimize probability of misclassification

49
Bayes Decision Theory

• Discriminant functions
• class membership solely based on relative sizes
• Reformulate classification process in terms of
• discriminant functions
• x is assigned to Ck if

50
Bayes Decision Theory

• Discriminant function examples

51
Bayes Decision Theory

• Discriminant function examples 2-class problem

52
Bayes Decision Theory

• Why is such a big
  deal ?

53
Bayes Decision Theory

• Why is such a big
  deal ?
• Example 1 Speech Recognition

FFT melscale bank
x
y e /ah/, /eh/, .. /uh/
apple, ...,zebra
54
Bayes Decision Theory

• Why is such a big
  deal ?
• Example 1 Speech Recognition

/t/
/t/
/t/
/t/
FFT melscale bank
/aal/
/aol/
/owl/
55
Bayes Decision Theory

• Why is such a big
  deal ?
• Example 1 Speech Recognition

How do Humans do it?
56
Bayes Decision Theory

• Why is such a big
  deal ?
• Example 1 Speech Recognition

This machine can recognize speech ??
57
Bayes Decision Theory

• Why is such a big
  deal ?
• Example 1 Speech Recognition

This machine can wreck a nice beach !!
58
Bayes Decision Theory

• Why is such a big
  deal ?
• Example 1 Speech Recognition

FFT melscale bank
x
y
59
Bayes Decision Theory

• Why is such a big
  deal ?
• Example 1 Speech Recognition

Language Model
FFT melscale bank
x
y
P(wreck a nice beach) 0.001 P(recognize
speech) 0.02
60
Bayes Decision Theory

• Why is such a big
  deal ?
• Example 2 Computer Vision

Low-Level Image Measurements
High-Level Model Knowledge
61
Bayes

• Why is such a big
  deal ?
• Example 3 Curve Fitting

E bW ln p(xc) ln p(c)
62
Bayes

• Why is such a big
  deal ?
• Example 4 Snake Tracking

E bW ln p(xc) ln p(c)
63
Computer Vision Vision and Modeling

• Lucas-Kanade Extensions
• Support Maps / Layers
• Robust Norm, Layered Motion, Background
  Subtraction, Color Layers
• Statistical Models (ForsythPonce Chap. 6,
  DudaHartStork Chap. 1-5)
• - Bayesian Decision Theory
• - Density Estimation

64
Probability Density Estimation
?
Collect Data x1,x2,x3,x4,x5,...
x
Estimate
x
65
Probability Density Estimation

• Parametric Representations
• Non-Parametric Representations
• Mixture Models

66
Probability Density Estimation

• Parametric Representations
• - Normal Distribution (Gaussian)
• - Maximum Likelihood
• - Bayesian Learning

67
Normal Distribution
68
Multivariate Normal Distribution
69
Multivariate Normal Distribution

• Why Gaussian ?
• Simple analytical properties
• - linear transformations of Gaussians are
  Gaussian
• - marginal and conditional densities of
  Gaussians are Gaussian
• - any moment of Gaussian densities is an
  explicit function of m,s
• Good Model of Nature
• - Central Limit Theorem Mean of M random
  variables is distributed
• normally in the limit.

70
Multivariate Normal Distribution
Discriminant functions
71
Multivariate Normal Distribution
Discriminant functions
equal priors cov Mahalanobis dist.
72
Multivariate Normal Distribution
How to learn it from examples

• Maximum Likelihood
• Bayesian Learning

73
Maximum Likelihood
How to learn density from examples
?
x
?
x
74
Maximum Likelihood
Likelihood that density model q generated data X
75
Maximum Likelihood
Likelihood that density model q generated data X
76
Maximum Likelihood
Learning optimizing (maximizing likelihood /
minimizing E)
77
Maximum Likelihood
Maximum Likelihood for Gaussian density
Close-form solution