CIS732-Lecture-13-20011004

1
GECCO-2002
A Permutation Genetic Algorithm for Variable
Ordering in Bayesian Networks from Data
William H. Hsu, Haipeng Guo, Benjamin B. Perry,
Julie A. Stilson Laboratory for Knowledge
Discovery in Databases Kansas State
University http//www.kddresearch.org http//www.
cis.ksu.edu/bbp9857
2
Overview

• Bayesian Network
• Definitions and examples
• Inference and learning
• Challenge of validating structures
• Learning Network Structure
• Problem
• K2 algorithm
• Improving K2 Permutation Genetic Algorithm
• Shortcoming greedy, sensitive to ordering
• Permutation GA
• Evaluation with Known Bayesian Networks
• Summary and Future Work

3
Bayesian Belief Networks (BBNS)Definition

• Bayesian Network
• Directed graph model of conditional dependence
  assertions (or CI assumptions)
• Vertices (nodes) denote events (each a random
  variable)
• Edges (arcs, links) denote conditional
  dependencies
• General Product (Chain) Rule for BBNs
• Example (Sprinkler BBN)

P(Summer, Off, Drizzle, Wet, Not-Slippery) P(S)
P(O S) P(D S) P(W O, D) P(N W)
4
Graphical Modelsof Probability Distributions

• Idea
• Want model that can be used to perform inference
• Desired properties
• Correlations among variables
• Ability to represent functional, logical,
  stochastic relationships
• Probability of certain events
• Inference Decision Support Problems
• Diagnosis (medical, equipment)
• Pattern recognition (image, speech)
• Prediction
• Want to Learn Most Likely Model that Generates
  Observed Data
• Under certain assumptions (Causal Markovity), it
  has been shown that we can do it
• Given data D (tuples or vectors containing
  observed values of variables)
• Return directed graph (V, E) expressing target
  CPTs (or commitment to acquire)

5
Learning StructureK2 Algorithm

• Algorithm Learn-BBN-Structure-K2 (D, Max-Parents)
• FOR i ? 1 to n DO // arbitrary ordering of
  variables x1, x2, , xn
• WHILE (Parentsxi.Size lt Max-Parents) DO // find
  best candidate parent
• Best ? argmaxjgti (P(D xj ? Parentsxi) // max
  Dirichlet score
• IF (Parentsxi Best).Score gt
  Parentsxi.Score) THEN Parentsxi Best
• RETURN (Parentsxi i ? 1, 2, , n)
• A Logical Alarm Reduction Mechanism Beinlich et
  al, 1989
• BBN model for patient monitoring in surgical
  anesthesia
• Vertices (37) findings (e.g., esophageal
  intubation), intermediates, observables
• K2 found BBN different in only 1 edge from gold
  standard (elicited from expert)

6
Learning StructureDirichlet (Bayesian) Score
and K2 Algorithm

• Bayesian Score (aka Dirichlet Score) for Marginal
  Likelihood P(D h)
• K2 Algorithm for General Case Structure Learning
• Greedy, Bayesian score-based add arcs based upon
  incremental gain each single arc induces in a
  global score
• See http//wilma.cs.brown.edu/research/ai/dynamic
  s/tutorial/
• Learning for Decision Support in Policy-Making
• Does smoking cause cancer?
• Do school vouchers improve education?
• Does reduction of visa duration decrease risk of
  terrorist activity?

7
GASLEAKA Permutation GA for Variable Ordering
8
Validation by Inference
9
Scores for Learning StructureThe Role of
Inference
Root Mean Square Error (RMSE)
10
Properties of the Genetic Algorithm

• Elitist
• Chromosome representation
• Integer permutation ordering
• Sample chromosome in a BBN of 5 nodes might look
  like 3 1 2 0 4
• Seeding
• Random shuffle
• Operators
• Order crossover
• Swap mutation
• Fitness
• RMSE
• Job farm
• Java-based Utilize many machines regardless of
  OS

11
Exhaustive Results
12
Results on Asia
13
Learning StructureCausal Discovery
14
Learning StructureCausal Discovery
15
Learning StructureCausal Discovery
16
Summary

• Bayesian Network
• Learning Structure
• K2 Problems
• GASLEAK

17
Future Work

• Fast Inference
• Larger networks
• Different Learning Algorithms
• Sparse candidate
• Tabu
• Different GA specs
• Mutation / Crossover rates
• Crossover type
• Partially matched
• Cycle

18

• Thanks for your attendance
• www.kddresearch.org
• www.ksu.edu
• www.cis.ksu.edu/bbp9857