Summary of the Bayes Net Formalism

1
Summary of the Bayes Net Formalism

• David Danks
• Institute for Human Machine Cognition

2
Bayesian Networks

• Two components
• Directed Acyclic Graph (DAG)
• G There is a node for every variable
• D Some nodes have arrows btw. them (X ? Y)
• A Cant get from one node back to itself by
  following arrows
• Joint Probability Distribution
• For all X,Y,,Z, P(Xx,Yy,,Zz) is defined

3
Example of a Bayes Net

• Directed Acyclic Graph
• Joint Probability Distribution
• P(APHigh, BHigh, STYes) 0.2
• P(APHigh, BLow, STYes) 0.005

4
Connecting the Graph JPD

• Markov assumptionX is (probabilistically)
  independent of its (graphical) non-descendants
  conditional on its (graphical) parents.
• No edge ? Conditional independence

5
Connecting the Graph JPD

• Markov assumptionX is (probabilistically)
  independent of its (graphical) non-descendants
  conditional on its (graphical) parents.
• No edge ? Conditional independence
• Example X ? Y ? Z ? X Z Y

6
Connecting the Graph JPD

• The Markov assumption implies a factorization of
  the JPD based on the graphP(Xx,Yy,,Zz) ?
  P(v parents(v))
• The Markov assumption allows us to move from
  graph to probability distribution

7
Connecting the Graph JPD

• Faithfulness assumptionThe (probabilistic)
  effects of (graphical) paths never exactly
  offset.
• Conditional independence ? No edge
• The Faithfulness assumption allows us to move
  from probability distribution to graph

8
Bayesian Network Example

• Running causes you to eat more
• Eating more causes you to gain weight
• Running increases your metabolism
• Increased metabolism leads to weight loss
• Note Faithfulness rules out Running Weight

9
Learning Bayes Nets

• Given some data from the world, why would we want
  to learn a Bayes net?
• Compact representation of the data
• There are fast algorithms for prediction/inference
  given observations of the environment
• Causal knowledge
• There are fast algorithms for prediction/inference
  given interventions in the environment

10
Bayesian Updating

• Start with a probability distribution over
  possible graphs

11
Bayesian Updating

• Start with a probability distribution over
  possible graphs, then figure out which graph is
  most likely given the observed data

12
Features of Bayesian Updating

• Advantages
• Output is fine-grained probabilistic information
• Rational basis for the learning algorithm
• Robust to data errors
• Disadvantages
• Number of possible graphs is super-exponential,
  likelihood functions not always solvable
  analytically ? almost always use heuristics
• Cannot easily incorporate unobserved variables

13
Constraint-Based Learning

1. Determine the (un)conditional associations and
   independencies in the data
2. Determine the set of Bayes nets that could have
   produced that data

14
Constraint-Based Learning

• Determine the (un)conditional associations and
  independencies in the data
• Determine the set of Bayes nets that could have
  produced that data
• X W (all)
• Y Z (all)
• X Y,Z W
• Z X,W Y

15
Constraint-Based Learning

• Determine the (un)conditional associations and
  independencies in the data
• Determine the set of Bayes nets that could have
  produced that data
• X W (all)
• Y Z (all)
• X Y,Z W
• Z X,W Y

U
X
Z
W
Y
16
Features of C-B Learning

• Advantages
• Feasible, asymptotically correct algorithms
  (though worst case requires exponential comp.)
• Easily incorporates unobserved variables
• Gives exactly the information in the data
• Disadvantages
• Susceptible to mistaken independence judgments
  (big problem for small datasets)
• Cannot use fine-grained prior knowledge

17
Layers of Bayes Nets

• Start with simple association graph

18
Layers of Bayes Nets

• Then we can add a causal interpretation

19
Layers of Bayes Nets

• Or we can provide parameter information

20
Layers of Bayes Nets

• Or we can combine causation parameters

21
Useful Websites

• Tutorials
• http//www.cs.berkeley.edu/murphyk/Bayes/bayes.ht
  ml
• Constraint-based learning software
• http//www.phil.cmu.edu/tetrad/index.html (free)
• Bayesian learning software
• http//www.hugin.com (commercial)
• http//research.microsoft.com/dmax/winmine/tooldo
  c.htm (free)