Bayesian networks for Knowledge Engineering

1
Bayesian networks for Knowledge Engineering

• Ann Nicholson

Faculty of Information Technology Monash
University
2
Overview

• Representing uncertainty
• Introduction to Bayesian Networks
• The knowledge engineering process
• Applications
• Medical Risk Assessment
• Intelligent Tutoring
• Bayesian Poker
• Weather Forecasting
• Ecological Risk Assessment

3
Sources of Uncertainty

• Ignorance
• which side of this coin is ?
• Complexity
• E.g. meteorology
• Physical randomness
• Which side of this coin will land up?
• AI representations
• Probability theory
• Dempster-Shafer
• Fuzzy logic

4
Probability theory for representing uncertainty

• Assigns a numerical degree of belief between 0
  and 1 to facts
• e.g. it will rain today is T/F.
• P(it will rain today) 0.2 prior probability
  (unconditional)
• Posterior probability (conditional)
• P(it will rain today rain is forecast)
  0.8
• Bayes Rule
• P(HE) P(EH) x P(H)
• P(E)

5
Bayesian networks

• Represents a probability distribution graphically
  (directed acyclic graphs)
• Nodes random variables,
• R it is raining, discrete values T/F
• T temperature, cts or discrete variable
• C colour, discrete values red,blue,green
• Arcs indicate conditional dependencies between
  variables

6
Bayesian networks

• Conditional Probability Distribution (CPD)
• Associated with each variable
• probability of each state given parent states

P(FluT) 0.05
Models causal relationship
P(TeHighFluT) 0.4 P(TeHighFluF) 0.01
Models possible sensor error
P(ThHighTeH) 0.95 P(ThHighTeL) 0.1
A BN provides a compact representation of the
joint distribution P(Flu, Te, Th) P(Flu)
P(TeFlu) P(ThTe)
7
BN inference

• Evidence observation of specific state
• Task compute the posterior probabilities for
  query node(s) given evidence.

Flu
8
BNs for Epidemiology(Kevin Korb, John McNeil,
Charles Twardy, Bin Han, Rodney ODonnell ARC
Discovery)
Problem assessment of risk for coronary heart
disease (CHD)
1. Knowledge Engineering
9
The Busselton BN arcs
uninformative
All nodes have an associated conditional prob.
distribution
predictor variables
10-year risk of CHD event
10
The Busselton BN discretization
11
The Busselton BN reasoning
12
The Busselton BN reasoning
13
The Busselton BN reasoning
14
The Busselton BN reasoning
15
Decision networks

• Extension to basic BN for decision making
• Decision nodes
• Utility nodes
• EU(Action) ? p( O Action , E) U( O )
• O
• sum over all the possible outcomes, O
• choose action with highest expect utility
  

16
A risk assessment tool for clinicians

• Previous tool TAKEHEART
• Combine risk assessment (probability) with costs.

17
Risk Assessment Tool example
18
Knowledge Engineering for Bayesian Networks
(KEBN)

• 1. Building the BN
• variables, structure, parameters, preferences
• combination of expert elicitation and knowledge
  discovery
• 2. Validation/Evaluation
• case-based, sensitivity analysis, accuracy
  testing
• 3. Field Testing
• alpha/beta testing, acceptance testing
• 4. Industrial Use
• collection of statistics
• 5. Refinement
• Updating procedures, regression testing

19
Intelligent Tutoring (Liz Sonenberg, Kaye
Stacey, Vicki Steinle U. Melb, Tali Boneh)

• Tutoring domain primary and secondary school
  students misconceptions about decimals
• Based on Decimal Comparison Test (DCT)
• student asked to choose the larger of pairs of
  decimals
• different types of pairs reveal different
  misconceptions
• ITS System involves computer games involving
  decimals
• A combination of expert elicitation and automated
  methods
• Publications UAI2001, UM2003
• (Demo of DecSys tutoring system)

20
Expert classification of Decimal Comparison Test
(DCT) results
H high (all correct or only one wrong)
L low (all wrong or only one correct)
21
The ITS architecture
Adaptive Bayesian Network
Inputs
Student
Generic BN model of student
Decimal comparison test (optional)
Item
Answers
Answer

• Diagnose misconception
• Predict outcomes
• Identify most useful information

Information about student e.g. age (optional)
Computer Games
Hidden number
Answer
Classroom diagnostic test results (optional)
Feedback
Answer
Flying photographer

• Select next item type
• Decide to present help
• Decide change to new game
• Identify when expertise gained

Item type
System Controller Module
Item
Decimaliens
New game
Sequencing tactics
Number between
Help
Help
.
Report on student
Classroom Teaching Activities
Teacher
22
Expert Elicited BN
23
Monash Bayesian Poker History

• Started with Kevin Korb 1993. 5 card stud.
• Honours projects
• Nathalie Jitnah, 1993. The basic BN.
• Scott Thomson, 1994. Some improvements.
• Aidan Doyle, 1995. Web interface.
• Jason Carlton, 2000. Extended to decision
  networks.
• Darren Boulton, 2002-03. Extended to Texas
  Hold'em Poker. Improved bluffing and opponent
  modelling.
• Publications
• Korb, A.E. Nicholson and N. Jitnah. Bayesian
  Poker. UAI99.
• Section 5.3. Bayesian Artificial Intelligence.
  Korb and Nicholson 2004.
• 2006
• Inauguaral Bot Poker competition, AAAI. (3rd)
• Web interface (Steven Mascaro)
• CSE1370 Advanced Project (learning opponent model
  from AAAI tournament logs)

24
Decision network for Texas Holdem Poker
25
Monash BPP

• Hand type abstraction 25 hand types
• Busted (5) low, medium, Q-high, K-high, A-high
• Pairs (13) all
• The rest (7)
• Two pairs, 3-of-a-kind, Flush, Straight, Full
  house, 4 of a kind, Straight flush
• Estimates winnings
• Betting with randomisation

26
Bayesian Poker Player (BPP)

• www.csse.monash.edu.au/bai/poker

27
Weather forecasting

• Seabreeze prediction joint project with Bureau
  of Meteorology (Chris Ryan Bur. Of Met, Russell
  Kennett, Honours project, Kevin Korb, 2001)
• Comparison of existing simple rule, expert
  elicited BN, and BNs from Tetrad-II and CaMML
• BNs for supporting meteorological forecasting
  process (ARC Linkage 2002-2004, Tali Boneh, Kevin
  Korb, BoM)
• Building domain ontology (in Protege) from expert
  elicitation
• Automatically generating BN fragments
• Case Studies Fog, Thunderstorms

28
Case Study I Fog(Gary Weymouth, Bur. Of Met,
Tali Boneh)
Environment
Refinements
Meteorology
Predictors
Guidance
needs utility function
29
Finding utility fn by solving constraints

• Unable to directly elicit aviation industry
  preferences, so
• make a set of constraints
• feed it to constraint solver
• use the new constraint solver-produced utilities

30
Set of constraints for preferences

• order
• X1ltX2ltX12ltX11ltX10ltX9ltX8ltX7ltX3ltX4ltX5ltX6
• use cut-off probabilities utility function d
• Efinition to form constraints
• (1ltilt7 and 7ltjlt13)
• for 0ltP(fog)lt1 gt
• P(fog)X1 P(nofog)X7 gt P(fog)Xi
  P(fog)xj
• (for iltgt1 and jltgt7)
• for 1ltP(fog)lt5 gt
• P(fog)X2 P(nofog)X8 gt P(fog)Xi
  P(fog)xj
• (for iltgt2 and jltgt8)
• etc.
• more constraints
• X2 lt 9X1 (X2 is not that great)
• X12 gt 16X1 (Worst false alarm is 16 times better
  than a complete miss)
• X3 lt 80X1 . . .

31
Different situations and constraints preferences
Relative importance of missing fog larger then
forecaster estimate
32
Ecological Risk Assessment

• Goulburn Water, native fish abundance (Carmel
  Pollino, Barry Hart Monash Water Studies, Kevin
  Korb, Owen Woodberry)
• GBR seagrass plume (Colette Thomas, Monash Water
  Studies)
• Sydney Harbour Water Quality (NSW EPA, 2003,
  Charles Twardy, Kevin Korb, Shannon Watson)

33
Goulburn Water Expert Elicited Conceptual Model
34
Goulburn Fish Network

• 5 sub-networks
• Water Quality
• Flow
• Structural Habitat
• Biological Interactions
• 2 query nodes
• Fish Abundance
• Fish Diversity
• 23 sites
• 6 reaches
• 2 temporal scales
• 1 and 5 year changes

35
Quantitative KE process
Woodberry et al, 2004
36
Methodology and Tools for KEBN

• Matilda visualisation of d-separation
• Tali Boneh, Liz Sonenberg U. Melb
• Support for sensitivity analysis
• Owen Woodberry
• VerbalBN
• Lucas Hope, Kevin Korb
• CausalReckoner
• Lucas Hope, Kevin Korb, Karl Axnick

37
Current Research directions

• Methodology for combining expert elicitation and
  automated methods
• expert knowledge used to guide search
• automated methods provide alternatives to be
  presented to experts
• Evaluation measures and methods
• Improved tools to support elicitation
• Reduce reliance on BN expert
• Industry adoption of BN technology