Flexible templates, density estimation, mean shift

1
Flexible templates, density estimation, mean shift

• CS664 Lecture 23
• Tuesday 11/16/04
• Some slides care of Dan Huttenlocher, Dorin
  Comaniciu

2
Administrivia

• Quiz 4 on Thursday
• Coverage through last week
• Quiz 5 on last Tuesday of classes (11/30)
• 1-page paper writeup due on 11/30
• Hand in via CMS
• List the papers assumptions, possible
  applications.
• What is good and bad about it?

3
DT as lower envelope
4
Robust Hausdorff distance

• Standard distance not robust

Can use fraction or distance
5
DT based recognition

• Better than explicit search for correspondence
• Features can merge or split, leading to a
  computational nightmare
• Outlier tolerance is important
• Should take advantage of the spatial coherence of
  unmatched points

6
Flexible models

• Classical recognition doesnt handle non-rigid
  motions at all well
• Pictorial structures
• Parts springs
• Appearance models

7
Model definition

• Parts are Vv1,,vn, graph vertices
• Configuration L(l1, , ln)
• Specifying locations of the parts
• Appearance parameters A(a1, , an)
• Model for each part
• Edge eij, (vi,vj) ? E for connected parts
• Explicit dependency between part locations li, lj
• Connection parameters Ccij eij ? E
• Spring parameters for each pair of connected
  parts

8
Flexible Template Algorithms

• Difficulty depends on structure of graph
• General case exponential time
• Consider special case in which parts translate
  with respect to common origin
• E.g., useful for faces

• Parts V v1, vn
• Distinguished central part v1
• Spring ci1 connecting vi to v1
• Quadratic cost for spring

9
Central Part Energy Function

• Location L(l1, , ln) specifies where each part
  positioned in image
• Best location minL (?i mi(li) di(li,l1))
• Part cost mi(li)
• Measures degree of mismatch of appearance ai when
  part vi placed at location li
• Deformation cost di(li,l1)
• Spring cost ci1 of part vi measured with respect
  to central part v1
• Note deformation cost zero for part v1 (wrt self)

10
Consider Case of 2 Parts

• minl1,l2 (m1(l1) m2(l2)??l2T2(l1)??2)
• Here, T2(l1) transforms l1 to ideal location wrt
  l2 (offset)
• minl1 (m1(l1) minl2 (m2(l2)??l2T2(l1)??2))
• But minx (f(x) ??xy??2) is a distance
  transform
• minl1 (m1(l1) Dm2(T2(l1))

11
Intuition
Never chosen!
Rest position of the part is 2 to the right of
the root
12
Lower envelope tells you for any root loc. the
best part loc!
cost
location
root
part
13
Application to Face Detection

• Five parts eyes, tip of nose, sides of mouth
• Each part a local image patch
• Represented as response to oriented filters
• 27 filters at 3 scales and 9 orientations
• Learn coefficients from labeled examples
• Parts translate with respect to central part, tip
  of nose

14
Face Detection Results

• Runs at several frames per second
• Compute oriented filters at 27 orientations and
  scales for part cost mi
• Distance transform mi for each part other than
  central one (nose tip)
• Find maximum of sumfor detected location

15
General trees via DP

• Want to minimize ?V mj(lj)?E dij(li,lj) over
  (V,E)
• Can express this as a function Bj(li)
• Cost of best location of vj given location li of
  vi
• Recursion in terms of children Cj of vj
• Bj(li) minlj( mj(lj) dij(li,lj) ?Cj BC(lj)
  )
• For leaf node no children, so last term empty
• For root node no parent, so second term empty

16
Further optimization via DT

• This recurrence can be solved in time O(ns2) for
  s locations, n parts
• Still not practical since s is in the millions
• Couple with distance transform method for finding
  best pair-wise locations in linear time
• Resulting method is O(ns)!

17
Example Finding People

• Ten part 2D model
• Rectangular regions for each part
• Translation, rotation, scaling of parts
• Configurations may allow parts to overlap

Image Data
Likely Configurations
Best Match
18
More examples
19
Probability outcomes, events
Outcome finest-grained description of
situation Event set of outcomes Probability
measure of (relative) event size Conditional
probability relative to containing event
written Pr(EE)
H,T,H
T,T,H
20
Random variables
Function from events to a domain (Z or R) Defined
by a Cumulative Distribution Function
(CDF) Derivative of distribution is the density
(PDF)
H,T,H
T,H,H
Event X2
21
Discrete vs continuous RVs

• Discrete case
• Easier to think about density (called PMF)
• For each value of the RV, a non-negative number
• Summing to 1

• Continuous case
• Often need to think about distribution
• PDF (density) can be larger than 1
• Integral over a range is most often useful

22
Density estimation

• Given some data drawn from an unknown
  distribution, how can you compute the density?
• If you know it is Gaussian, there are only 2
  parameters mean, variance
• You can compute the mean and variance of the
  data!
• These are provable good estimates

23
Density estimation

• Parametric methods
• ML estimators (Gaussians)
• Robust estimators
• Non-parametric methods
• Kernel estimators

24
Mean shift

• Non-parametric method to compute the nearest mode
  of a distribution
• Hill-climbing in terms of density

25
Applications of mean shift

• Density modes are useful!
• Segmentation
• Color, motion, etc.
• Tracking
• Edge-preserving smoothness
• See papers for more examples

26
Image and histogram
27
Example
28
Segmentation example