236372 - Bayesian Networks

1
236372 - Bayesian Networks

• Clique tree algorithm
• Presented by Sergey Vichik

2
Algorithm sequence

1. Translate a BN to Markov graph (moralization)
2. Add edges to create chordal graph
3. Find cliques
4. Construct clique tree
5. Enter evidences
6. Calc a posteriori probability (inference)

3
BN to Markov
A
A
B
B
G
G
D
D
C
C
E
E
F
F

• Add an edge between parents.

4
Create Chordal graph
A
A
B
B
G
G
D
D
C
C
E
E
F
F

• Add edges to form a chordal graph.

5
Cliques graph
1
AGC
ABD
2
2
ADC
1
1
2
CDE
1
EF

• Find all cliques, connect them to form a clique
  graph
• Cliques are connected if they are sharing a
  variable

6
Running Intersection property
1
AGC
ABD

• Many trees may be embedded in the cliques graph.
  What tree to choose?
• Lets try ADC-AGC-ABD-CDE-EF
• Now lets follow this order of cliques and perform
  an elimination
• Eliminate F, Eliminate E
• Now, in order to continue we need to eliminate
  either C or D. Eliminating any of them will
  result in creating an extra edge CB or GD,
  thus enlarging the probability tables.
• The required property If variable x is
  contained in cliques Y and Z, it must be
  contained in every. clique on path from Y to Z.

2
ADC
1
CDE
1
EF
A
B
G
D
C
E
F
7
Clique tree construction

• A maximal spanning tree of the clique graph, is
  the required clique tree.
• Proof At the end of the lecture.

AGC
ABD
2
2
ADC
2
CDE
1
EF
8
Enter evidence

• Entering evidence is eqivalent to removing the
  evidence variables and recalculating the effected
  probability functions.
• General approach build the clique tree without
  the evidence, and then recalculate the effected
  cliques. Actually it means reducing the tables to
  specific values
• f(E,F) -gt f(E,Ff).

9
Efficient Calculation of probabilities
AGC
ABD

• mij a message from i to j
• Wait for all messages excluding j (immediately
  for leaves).
• Store messages on the edge for efficient update.

AC
AD
ADC
CD
CDE
E
EF
10
Calculation of a marginal probability

• Select clique with a variable.
• Multiply all incoming messages.
• Marginalize to the required variable.
• Pf(ADC)f(AGC)f(ABD)f(CDE)f(EF)

AGC
ABD
AC
AD
ADC
CD
CDE
E
EF
11
Maximum Spanning tree is a Clique Tree a Proof
../1

• 1) From graph theory Cycle property
• For any cycle C in the graph, if the weight of an
  edge e of C is smaller than the weights of other
  edges of C, then this edge cannot belong to an
  MST.
• 2) For any chordal graph exists a clique tree.
• Every chordal graph have a perfect elimination
  order.

12
Proof ../2
Cut set - set of all edges connecting 2 graph
partitions

• Lets take a clique tree that has as much common
  edges with MST but different.
• For the purpose of contradiction, assume that
  edge (K1,K2) in MST is not an edge in CT (Clique
  Tree).
• take the cut set associated with (K1,K2) in MST
  from the full graph.
• There must be another edge (K3,K4) not equal to
  (K1,K2) in this cut set, and (K3,K4) is in CT but
  not in MST.

K1
K2
K3
K4
(K1,K2)?MST (K3,K4)?CT (K3,K4)??MST
13
Proof ../3

• But from properties of clique tree K1nK2
  ?K3nK4. if K1nK2 ? K3nK4, it contradicts the fact
  that MST is maximal.
• Therefore K1nK2 K3nK4.
• We can replace the (K3,K4) with (K1,K2) in CT,
  and still remain with a clique tree.
• Lets take K5 and K6 belong to different cut set
  sides. From properties of CT, K5nK6 ? K3nK4 and
  thus K5nK6 ? K1nK2.
• Therefore the clique intersection property holds
• We have contradicted the fact that CT is
  different from MST and proposed an algorithm to
  make them equal.

K1
K2
K3
K4
(K1,K2)?MST (K3,K4)?CT (K3,K4)??MST CTgt K1nK2
?K3nK4 MSTgt K1nK2 ? K3nK4 gt K1nK2 K3nK4 //