A Trainable Graph Combination Scheme for Belief Propagation

1
A Trainable Graph Combination Scheme for Belief
Propagation

• Kai Ju Liu
• New York University

2
Images
3
Pairwise Markov Random Field
4
1
2
3
5

• Basic structure vertices, edges

4
Pairwise Markov Random Field

• Basic structure vertices, edges
• Vertex i has set of possible states Xi

5
Pairwise MRF Probabilities

• Joint probability

• Advantage allows average over ambiguous states
• Disadvantage complexity exponential in number of
  vertices

6
Belief Propagation
4
1
2
3
5
7
Belief Propagation

• Beliefs replace probabilities

8
Belief Propagation Example
4
1
3
5
9
BP Questions

• When can we calculate beliefs exactly?
• When do beliefs equal probabilities?
• When is belief propagation efficient?

10
BP on Loopy Graphs

• Messages do not terminate

• Energy approximation schemes Freeman et al.
• Standard belief propagation
• Generalized belief propagation
• Standard belief propagation
• Approximates Gibbs free energy of system by Bethe
  free energy
• Iterates, requiring convergence criteria

11
BP on Loopy Graphs

• Tree-based reparameterization Wainwright
• Reparameterizes distributions on singly-connected
  graphs
• Convergence improved compared to standard belief
  propagation
• Permits calculation of bounds on approximation
  errors

12
BP-TwoGraphs

• Eliminates iteration
• Utilizes advantages of SCGs

13
BP-TwoGraphs

• Consider loopy graph with n vertices
• Select two sets of SCGs that approximate the
  graph

14
BP-TwoGraphs on Images

• Rectangular grid of pixel vertices
• Hi horizontal graphs
• Gi vertical graphs

vertical graph
original graph
horizontal graph
15
Image Segmentation
add noise
segment
16
Image Segmentation Results
17
Image Segmentation Revisited
add noise
ground truth
max-flow ground truth
18
Image SegmentationHorizontal Graph Analysis
19
Image SegmentationVertical Graph Analysis
20
BP-TwoLines

• Rectangular grid of pixel vertices
• Hi horizontal lines
• Gi vertical lines

vertical line
original graph
horizontal line
21
Image Segmentation Results II
22
Image Segmentation Results III
23
Natural Image Segmentation
24
Boundary-Based Image Segmentation Window Vertices

• Square 2-by-2 window of pixels
• Each pixel has two states
• foreground
• background

25
Boundary-Based Image Segmentation Overlap
26
Boundary-Based Image Segmentation Graph
27
Real Image Segmentation Training
28
Real Image Segmentation Results
29
Real Image Segmentation Gorilla Results
30
Conclusion

• BP-TwoGraphs
• Accurate and efficient
• Extensive use of beliefs
• Trainable parameters
• Future work
• Multiple states
• Stereo
• Image fusion