Hierarchical Kernel StickBreaking Process for MultiTask Image Analysis

1
Hierarchical Kernel Stick-Breaking Process for
Multi-Task Image Analysis

• Qi An1, Chunping Wang1, Ivo Sheterev1, Eric
  Wang1, David B. Dunson2, Lawrence Carin1

1 Dept. of Electrical and Computer Engineering 2
Dept. of Statistical Science Duke University,
Durham, NC 27705
2
Image Segmentation

• How to segment images?
• Manual segmentation (very expensive)
• Algorithm segmentation
• K-means
• Statistical mixture models
• Spectral clustering

3
Image Segmentation

• Problems with most existing algorithms
• Ignore the spatial information
• Perform the segmentation one image at a time
• Need to specify the number of segments a priori
• Our algorithm is designed to address these
  difficulties

4
Dirichlet process (DP)

• A measure over measures
• Written as

5
Kernel stick-breaking process (KSBP)

• Kernel stick-breaking process is an extension of
  the Dirichlet process, introduced by Dunson and
  Park.
• Written as
• KSBP augmented the stick-breaking representation
  of DP to employ a kernel function to quantify
  some additional prior.
• For image analysis, we want to impose the belief
  that spatially proximate patches are more
  probable to be associated with the same cluster.

6
KSBP for image analysis
Consider a image composed of patches, the
features vectors and the associated
locations can be modeled as
follows
7
DP and KSBP

• Connections
• Both of them take the general form of a
  stick-breaking representation
• The draws from both processes are guaranteed to
  be discrete
• The extensions to the multi-task setting are both
  straightforward
• Differences yielded by KSBP
• The construction of weights and therefore
  are sample-dependent (remove exchangeability
  of samples)
• Basis functions serve to localize in the
  space of

8
DP and KSBP

• Graphical representation of both DP and KSBP
  models

9
Spatial correlation property

• Two drawn samples, and , are encouraged
  to share the same atoms if they are close, and
  the correlation coefficient between two
  probability measures is

10
Multi-task image segmentation

• Why segment multiple images simultaneously?
• Transfer expertise between images to achieve
  better performance
• Study the similarities between images via the
  sharing of clusters

11
Multi-task image segmentation

• We model each image with a
  , and link multiple KSBP models with an
  overarching DP, i.e. .
• The discrete form of ensures that the
  different 1 will share the same set of
  discrete atoms 1 with varying
  weights

12
Multi-task image segmentation

• A hierarchical KSBP (H-KSBP) model for image
  analysis is represented as

13
Multi-task Image Segmentation
14
Posterior inference

• For the large-scale problems of interest, here we
  employ variational Bayesian (VB) inference.
• However, we cannot obtain a closed form for the
  posterior of the node , because of the
  kernel function.
• Motivated by the Monte Carlo Expectation
  Maximization algorithm, we develop a Monte Carlo
  Variational Bayesian inference algorithm.

15
Experiments

• Single task learning

Original image
DP result
KSBP result
16
Experiments

• Multi-task learning
• We test our algorithms on a subset of images from
  Microsoft Research Cambridge.
• There are sever types of images used in this
  database buildings, clouds, countryside, faces,
  fireworks, offices and urban, with twenty images
  randomly selected from the database for each
  type.

17
Experiments
Matrix on the usage of atoms across images
18
Demonstration of different atoms as inferred by
an example run
19
A representative set of segmentation results
20
Experiments
A confusion matrix over image types based on
top-ten most similar images
21
Conclusions

• The kernel stick-breaking process has been
  extended for use in image segmentation.
• The algorithm explicitly impose the belief that
  feature vectors from proximate locations are more
  likely to be associated with the same segment.
• We have extended the KSBP algorithm to the MTL
  setting, exploring the inter-relationship of
  images by sharing the same mixing components

22
Thank you !