Describing Visual Scenes using Transformed Dirichlet Processes

1
Describing Visual Scenes using Transformed
Dirichlet Processes
Paper by E. B. Sudderth, A. Torralba, W. T.
Freeman, and A. S. Willsky, NIPS 2005
Duke University Machine Learning Group Presented
by Kai Ni May 19, 2006
2
Outline

• Motivation
• Hierarchical Dirichlet Processes (HDP)
• Transformed Dirichlet Processes (TDP)
• Application on Visual Scene

3
Motivation

• Problem Analyzing the features composing a
  visual scene, thereby localizing and categorizing
  the objects in an image.
• Goal Exploit relationships among multiple,
  partially labeled object categories with little
  manual supervision and labeling.
• Application
• Image object detection

4
Generative Models

• Constellation models Use fixed, small set of
  spatially constrained parts for single objects.
• Latent Dirichlet allocation (LDA) Use a
  spatially unstructured bag of features extracted
  from local image patches.
• Transformed DP Extension of nonparametric
  version LDA (which is HDP), making use of the
  spatial structure information.

5
Generative Models
6
Dirichlet Processes

• A single clustering problem can be analyzed as a
  Dirichlet processes (DP).

7
Hierarchical Dirichlet Process

• Mathematical form

8
HDP Chinese Restaurant Franchise

• First level within each group, DP mixture
• Fj1,,Fj(i-1), i.i.d., r.v., distributed
  according to Gj ?j1,, ?jTj to be the values
  taken on by Fj1,,Fj(i-1), njk be of Fji ?jt,
  0ltilti.
• Second level across group, sharing clusters
• Base measure of each group is a draw from DP
• ?1,, ?K to be the values taken on by ?j1,, ?jTj
  , mk be of ?jt?k, all j, t.

9
HDP CRF graph

• The values of ? are shared between groups, as
  well as within groups. This is a key property of
  HDP.

Integrating out G0
10
Transformed Dirichlet Process

• An extension of the HDP in which global mixture
  components undergo a set of random
  transformations before being reused in each
  group.
• Conditioning on
• The discreteness of Gj ensures that
  transformations are shared between observations
  within group j.

11
CRF view of TDP

• Customers prefer tables t at which many customers
  njt are already seated.
• Sometimes a new table t is chosen. Each new table
  is assigned with a dish kjt. Popular dishes are
  more likely to be ordered, but a new dish
  may also be selected.
• Each time the dish is ordered, the recipe is
  seasoned differently according to

12
Graphical Models
13
Gibbs Sampling

• Gibbs sampling variants including table
  assignment t, cluster assignment k,
  transformations , and parameters
• Sampling scheme is very similar to HDP
• Conjugacy assumption of F Q, H Q and R F
  to make sampling practical.

14
TDP for Visual Scenes Modeling

• Observed data xji (oji, yji), cluster
  parameters
• Assume the location of same object is different
  from image to image, transformation are defined
  on the cluster mean
• Different translation allow the same object
  cluster to be reused at multiple locations within
  a single image.

15
Synthetic Data Results

• HDP uses a large set of global clusters to
  discretize the transformations underlying the
  data, and may have poor generalization for
  modeling visual scenes.

16
Analyzing Street Scenes
17
Conclusion

• HDP is a hierarchical, nonparametric model for
  sharing information between multiple groups of
  data.
• TDP is an extension of HDP with allows the
  cluster parameters transform differently before
  reusing the sharing component.
• Visual scenes modeling and detecting is an good
  application of TDP, which gives a better
  generalized model than HDP.