Belief Propagation

1
Belief Propagation

• Kai Ju Liu
• March 9, 2006

2
Statistical Problems

• Medicine
• Finance
• Internet
• Computer vision

3
Inference Problems

• Given data B, infer A p(AB)
• Computer vision
• Given image, find objects
• Given two images, resolve 3D object
• Given multiple images, track object

4
Conditional Probability

• Given event B, what is probability of A?
• Independence p(AB)p(A)

5
Bayes Rule
6
e.g.
7
e.g. (cont.)
8
Approach

• Define variables and connections
• Calculate marginal probabilities efficiently
• Find most likely configuration

9
Pairwise Markov Random Field
4
1
2
3
5

• Basic structure vertices, edges

10
Pairwise Markov Random Field

• Basic structure vertices, edges
• Vertex i has set of possible states Xi

11
Pairwise MRF Probabilities

• Joint probability

• Advantage allows average over ambiguous states
• Disadvantage complexity exponential in number of
  vertices

12
Belief Propagation
4
1
2
3
5
13
Belief Propagation

• Beliefs replace probabilities

14
Belief Propagation Example
4
1
3
5
15
BP Questions

• When can we calculate beliefs exactly?
• When do beliefs equal probabilities?
• When is belief propagation efficient?

16
BP on Loopy Graphs

• Messages do not terminate
• Possible approximate solutions
• Standard belief propagation
• Generalized belief propagation

17
BP-TwoGraphs SCGs

• Consider loopy graph with n vertices
• Select two sets of SCGs that approximate the
  graph

18
BP-TwoGraphs Vision SCGs

• Rectangular grid of pixel vertices
• Hi horizontal graphs
• Gi vertical graphs

19
Image Segmentation
add noise
segment
20
Image Segmentation Results
21
Real Image Segmentation
22
Real Image Segmentation Training
23
Real Image Segmentation Results
24
Stereo Vision