Bayesian networks practice (Weka)

1
Bayesian networks practice(Weka)
2
Weather data
What is the Bayesian Network corresponding to
Naïve Bayes?
3
Effects and Causes vs. Evidence and Class

• Why Naïve Bayes has this graph?
• Because when we compute in Naïve Bayes
• P(playyes E)
• P(OutlookSunny playyes)
• P(TempCool playyes)
• P(HumidityHigh playyes)
• P(WindyTrue playyes)
• P(playyes) / P(E)
• we are interested in computing P(playyes),
  which are probabilities of our evidence
  observations given the class.

• Of course, play isnt a cause for outlook,
  temperature, humidity, and windy.
• However, play is the class and knowing that it
  has a certain value, will influence the
  observational evidence probability values.
• For example, if playyes, and we know that the
  playing happens indoors, then it is more probable
  (than without this class information) the outlook
  to be observed rainy.

4
Right or Wrong Topology?

• In general, there is no right or wrong graph
  topology.
• Of course the calculated probabilities (from the
  data) will be different for different graphs.
• Some graphs will induce better classifiers than
  some other.
• If you reverse the arrows in the previous figure,
  then you get a pure causal graph,
• whose induced classifier might have estimated
  error (through cross-validation) better or worse
  than the Naïve Bayes one (depending on the data).
  
• If the topology is constructed manually, we
  (humans) tend to prefer the causal direction.
• In domains such as medicine the graphs are
  usually less complex in the causal direction.

5
Weka suggestion
How Weka finds the shape of the graph? Fixes an
order of attributes (variables) and then adds and
removes arcs until it gets the smallest estimated
error (through cross-validation). By default it
starts with a Naïve Bayes network. Also, it
maintains a score of graph complexity, trying to
keep the complexity low.
6
(No Transcript)
7
Laplace correction. Better change it to 1, to be
compatible with the counter initialization in
Naïve Bayes.
It is going to start with a Naïve Bayes graph and
then try to add/remove arcs.
You can change to 2 for example. If you do, then
the max number of parents for a node will be 2.
8
Play probability table
Based on the data
P(playyes) 9/14 P(playno) 5/14
Lets correct with Laplace
P(playyes) (91)/(142) .625 P(playyes)
(51)/(142) .375
9
Outlook probability table
Based on the data
P(outlooksunnyplayyes) (21)/(93)
.25 P(outlookovercastplayyes) (41)/(93)
.417 P(outlookrainyplayyes) (31)/(93)
.333 P(outlooksunnyplayno) (31)/(53)
.5 P(outlookovercastplayno) (01)/(53)
.125 P(outlookrainyplayno) (21)/(53) .375
10
Windy probability table
Based on the datalets find the conditional
probabilities for windy
P(windytrueplayyes,outlooksunny)
(11)/(22) .5
11
Windy probability table
Based on the data
P(windytrueplayyes,outlooksunny)
(11)/(22) .5 P(windytrueplayyes,outlookov
ercast) 0.5 P(windytrueplayyes,outlookrai
ny) 0.2 P(windytrueplayno,outlooksunny)
0.4 P(windytrueplayno,outlookovercast)
0.5 P(windytrueplayno,outlookrainy) 0.75
12
Final figure
Classify it
Classify it
13
Classification I
Classify it
P(playyesoutlooksunny, tempcool,humidityhigh
, windytrue) ?P(playyes) P(outlooksunnypl
ayyes) P(tempcoolplayyes,
outlooksunny) P(humidityhighplayyes,
tempcool) P(windytru
eplayyes,
outlooksunny) ?0.6250.250.40.20.5
?0.00625
14
Classification II
Classify it
P(playnooutlooksunny, tempcool,humidityhigh,
windytrue) ?P(playno) P(outlooksunnyplay
no) P(tempcoolplayno,
outlooksunny) P(humidityhighplay no,
tempcool) P(windytruepl
ayno,
outlooksunny) ?0.3750.50.1670.3330.4
?0.00417
15
Classification III
Classify it
P(playyesoutlooksunny, tempcool,humidityhigh
, windytrue) ?0.00625 P(playnooutlooksunn
y, tempcool,humidityhigh, windytrue)
?.00417 ? 1/(0.006250.00417)
95.969 P(playyesoutlooksunny,
tempcool,humidityhigh, windytrue)
95.9690.00625 0.60
16
Classification IV (missing values or hidden
variables)
P(playyestempcool, humidityhigh, windytrue)
??outlookP(playyes) P(outlookplayyes) P(
tempcoolplayyes,outlook) P(humidityhighplay
yes,
tempcool) P(windytrueplayyes,outlook) (nex
t slide)
17
Classification V (missing values or hidden
variables)
P(playyestempcool, humidityhigh, windytrue)
??outlookP(playyes)P(outlookplayyes)P(te
mpcoolplayyes,outlook)
P(humidityhighplayyes,tempcool)P(windytrue
playyes,outlook) ? P(playyes)P(outlook
sunnyplayyes)P(tempcoolplayyes,outlooksunny
) P(humidityhighplayyes,tempcool)P(windytru
eplayyes,outlooksunny) P(playyes)P(outlook
overcastplayyes)P(tempcoolplayyes,outlooko
vercast) P(humidityhighplayyes,tempcool)P(wi
ndytrueplayyes,outlookovercast) P(playyes)
P(outlook rainyplayyes)P(tempcoolplayyes,ou
tlookrainy) P(humidityhighplayyes,tempcool)
P(windytrueplayyes,outlookrainy) ?
0.6250.250.40.20.5 0.6250.4170.2860.20.5
0.6250.330.3330.20.2 ?0.01645
18
Classification VI (missing values or hidden
variables)
P(playnotempcool, humidityhigh, windytrue)
??outlookP(playno)P(outlookplayno)P(temp
coolplayno,outlook) P(humidityhighpl
ayno,tempcool)P(windytrueplayno,outlook)
? P(playno)P(outlooksunnyplayno)P(tempcoo
lplayno,outlooksunny) P(humidityhighplayno,
tempcool)P(windytrueplayno,outlooksunny) P
(playno)P(outlook overcastplayno)P(tempcool
playno,outlookovercast) P(humidityhighplayn
o,tempcool)P(windytrueplayno,outlookovercast
) P(playno)P(outlook rainyplayno)P(tempco
olplayno,outlookrainy) P(humidityhighplayno
,tempcool)P(windytrueplayno,outlookrainy)
? 0.3750.50.1670.3330.4
0.3750.1250.3330.3330.5 0.3750.3750.40.33
30.75 ?0.0208
19
Classification VII (missing values or hidden
variables)
P(playyestempcool, humidityhigh, windytrue)
?0.01645 P(playnotempcool, humidityhigh,
windytrue) ?0.0208 ?1/(0.01645 0.0208)
26.846 P(playyestempcool, humidityhigh,
windytrue) 26.846 0.01645
0.44 P(playnotempcool, humidityhigh,
windytrue) 26.846 0.0208 0.56 I.e.
P(playyestempcool, humidityhigh, windytrue)
is 44 and P(playnotempcool,
humidityhigh, windytrue) is 56 So, we predict
playno.