Advances in Bayesian Learning Learning and Inference in Bayesian Networks

1
Advances in Bayesian Learning Learning
and Inference in Bayesian Networks

• Irina Rish
• IBM T.J.Watson Research Center
• rish_at_us.ibm.com

2
Road map

• Introduction and motivation
• What are Bayesian networks and why use them?
• How to use them
• Probabilistic inference
• How to learn them
• Learning parameters
• Learning graph structure
• Summary

3
Bayesian Networks
P (lung canceryes smokingno, dyspnoeayes )
?
4
What are they good for?

• Diagnosis P(causesymptom)?

• Prediction P(symptomcause)?

• Decision-making (given a cost function)

5
Bayesian Networks Representation
Smoking
lung Cancer
Bronchitis
X-ray
Dyspnoea
P(S) P(CS) P(BS) P(XC,S) P(DC,B)
P(S, C, B, X, D)
6
Example Printer Troubleshooting
7
Bayesian networks inference
P(Xevidence)?
P(sd1)
C
B
P(s)
D
X
Efficient inference variable orderings,
conditioning, approximations
8
Road map

• Introduction and motivation
• What are Bayesian networks and why use them?
• How to use them
• Probabilistic inference
• Why and how to learn them
• Learning parameters
• Learning graph structure
• Summary

9
Why learn Bayesian networks?

• Efficient representation and inference

• Handling missing data lt1.3 2.8 ?? 0 1 gt

10
Learning Bayesian Networks
11
Learning Parameterscomplete data

• ML-estimate

12
Learning Parametersincomplete data
EM-algorithm iterate until convergence
13
Learning graph structure

• Heuristic search
• Greedy local search

• Best-first search
• Simulated annealing
• Complete data local computations
• Incomplete data (score
• non-decomposable)
• Structural EM

• Constrained-based methods
• Data impose independence
• relations (constrains)

14
Scoring functionsMinimum Description Length
(MDL)

• Learning ? data compression
• Other MDL -BIC (Bayesian Information
  Criterion)
• Bayesian score (BDe) - asymptotically equivalent
  to MDL

DL(Model)
DL(Datamodel)
15
Summary

• Bayesian Networks graphical probabilistic
  models
• Efficient representation and inference
• Expert knowledge learning from data
• Learning
• parameters (parameter estimation, EM)
• structure (optimization w/ score functions
  e.g., MDL)
• Applications/systems collaborative filtering
  (MSBN), fraud detection (ATT), classification
  (AutoClass (NASA), TAN-BLT(SRI))
• Future directions causality, time, model
  evaluation criteria, approximate
  inference/learning, on-line learning, etc.