Discussion of Pictorial Structures

1
Discussion ofPictorial Structures

• Pedro FelzenszwalbDaniel HuttenlocherSicily
  WorkshopSeptember, 2006

2
What are Pictorial Structures?

• Local appearance
• Part models
• Parts ? feature detection
• Global geometry
• Not necessarily fully connected graph
• Joint optimization
• Combine appearanceand geometry withouthard
  constraints
• Stretch and fit
• Qualitative

3
Pictorial Structure Models

• Parts have match quality at each location
• Location in a configuration space
• No feature detection
• Maps for parts combined together into overall
  quality map
• According to underlying graph structure

4
A History of Pictorial Structures

• Fischler and Elschlager original 1973 paper
• Burl, Weber and Perona ECCV 1998
• Probabilistic formulation
• Full joint Gaussian spatial model
• Computational challenges led to feature-based
• Felzenszwalb and Huttenlocher CVPR 2000
• Explicit revisiting of FE73 for trees,
  probabilistic
• Efficient algorithms using distance transforms
• Crandall et al CVPR 2005, ECCV 2006
• Low tree-width graph structures, unsupervised

5
Matching Pictorial Structures

• Cost map for each part
• Distance transform (soft max) using spatial model
• Shift and combine
• Localize root then recursively other parts

6
Learning Models

• Automatically determine which spatial
  relationships to represent FH03
• Weakly supervised learning CH06
• Learn part appearance and geometric relations
  simultaneously
• No labeling of part locations
• Use large number of patches, similar to Ullman
• Better detection accuracy than strongly
  supervised

Car (rear) star topology
7
Parts as Context

• No part detected without using context provided
  by other parts
• Detect overall configuration composed of parts in
  a spatial arrangement
• Allows for weak evidence for a part
• Unlike feature detection
• Combination of matches can constrain pose
• In contrast to scene-level context
• More spatial regularity

8
Factored Models

• For n parts in fixed arrangement with k templates
  per part
• Exponential number of possibilities, O(kn)
• For variable arrangement, another exponential
  factor
• Important both for representation and algorithmic
  efficiency
• Pictorial structures takes particular advantage
  of this factoring

9
Closely Related Work

• Ioffe and Forsyth, Ramanan and Forsyth human body
  pose
• Part detection but very dense part locations
• Constellation models
• Fergus, Perona, Zisserman and others
• Hard feature detection in contrast with BWP98
  soft feature matching
• Amits patch models
• No assumption of independent part appearance
• Fergus and Zisserman star models

10
Whats Important

• No decisions until the end
• No feature detection
• Quality maps or likelihoods
• No hard geometric constraints
• Deformation costs or priors
• Efficient algorithms
• Dynamic programming critical or cant get away
  without making intermediate decisions
• Not applicable to all problems, need good
  factorizations of geometry and appearance

11
Some Pros

• Good for categorical object recognition
• Qualitative descriptions of appearance
• Factoring variability in appearance and geometry
• Deals well with occlusion
• In contrast to hard feature detection
• Weakly supervised learning algorithms
• Sampling as way of dealing with models that dont
  factor more Saturday

12
Some Cons/Limitations

• Most applicable to 2D objects defined by
  relatively small number of parts
• Unclear how to extend to large number of
  transformation parameters per part
• Explicit representation grows exponentially
• No known way of using to index into model
  databases

13
Role of Spatial Constraints

• For k-fans, spatial information substantially
  improves detection accuracy
• However, limited by relatively small number of
  parts compared to features in a bag
• General question