Bayesian Metanetworks for Context-Sensitive Feature Relevance

1
Bayesian Metanetworksfor Context-Sensitive
Feature Relevance

• Vagan Terziyan
• vagan_at_it.jyu.fi
• Industrial Ontologies Group, University of
  Jyväskylä, Finland

SETN-2006, Heraclion, Crete, Greece 24 May 2006
2
Contents

• Bayesian Metanetworks
• Metanetworks for managing conditional
  dependencies
• Metanetworks for managing feature relevance
• Example
• Conclusions

Vagan Terziyan Industrial Ontologies
Group Department of Mathematical Information
Technologies University of Jyvaskyla
(Finland) http//www.cs.jyu.fi/ai/vagan
This presentation http//www.cs.jyu.fi/ai/SETN-20
06.ppt
3
Bayesian Metanetworks
4
Conditional dependence between variables X and Y
P(Y) ?X (P(X) P(YX))
5
Bayesian Metanetwork

• Definition. The Bayesian Metanetwork is a set of
  Bayesian networks, which are put on each other in
  such a way that the elements (nodes or
  conditional dependencies) of every previous
  probabilistic network depend on the local
  probability distributions associated with the
  nodes of the next level network.

6
Two-level Bayesian C-Metanetwork for Managing
Conditional Dependencies
7
Contextual and Predictive Attributes
air pressure
dust
humidity
temperature
Machine
emission
Environment
Sensors
X
x5
x6
x7
x2
x3
x4
x1
contextual attributes
predictive attributes
8
Contextual Effect on Conditional Probability (1)
X
x5
x6
x7
x2
x3
x4
x1
contextual attributes
predictive attributes
Assume conditional dependence between predictive
attributes (causal relation between physical
quantities)
xt
some contextual attribute may effect directly
the conditional dependence between predictive
attributes but not the attributes itself
xk
xr
9
Contextual Effect on Conditional Probability (2)

• X x1, x2, , xn predictive attribute with n
  values
• Z z1, z2, , zq contextual attribute with q
  values
• P(YX) p1(YX), p2(YX), , p r(YX)
  conditional dependence attribute (random
  variable) between X and Y with r possible values
• P(P(YX)Z) conditional dependence between
  attribute Z and attribute P(YX)

10
Contextual Effect on Conditional Probability (3)
Xt1 I am in Paris Xt2 I am in Moscow
xt
P1(Xr Xk ) Xk1 Xk2
Xr1 0.3 0.9
Xr2 0.4 0.5
Xr1 visit football match Xr2 visit
girlfriend
Xk1 order flowers Xk2 order wine
xr
xk
P2(Xr Xk ) Xk1 Xk2
Xr1 0.1 0.2
Xr2 0.8 0.7
Xr Make a visit
Xk Order present
11
Contextual Effect on Conditional Probability (4)
Xt1 I am in Paris Xt2 I am in Moscow
xt
P( P (Xr Xk ) Xt ) Xt1 Xt2
P1(Xr Xk ) 0.7 0.2
P2(Xr Xk ) 0.3 0.8
xr
xk
P1(Xr Xk ) Xk1 Xk2
Xr1 0.3 0.9
Xr2 0.4 0.5
P2(Xr Xk ) Xk1 Xk2
Xr1 0.1 0.2
Xr2 0.8 0.7
12
Contextual Effect on Unconditional Probability (1)
X
x5
x6
x7
x2
x3
x4
x1
contextual attributes
predictive attributes
Assume some predictive attribute is a random
variable with appropriate probability
distribution for its values
xt
P(X)
some contextual attribute may effect directly
the probability distribution of the predictive
attribute
X
x1
x4
x2
x3
xk
13
Contextual Effect on Unconditional Probability (2)

•   X x1, x2, , xn predictive attribute with
  n values
•   Z z1, z2, , zq contextual attribute
  with q values and P(Z) probability distribution
  for values of Z
• P(X) p1(X), p2(X), , pr(X) probability
  distribution attribute for X (random variable)
  with r possible values (different possible
  probability distributions for X) and P(P(X)) is
  probability distribution for values of attribute
  P(X)
•    P(YX) is a conditional probability
  distribution of Y given X
•    P(P(X)Z) is a conditional probability
  distribution for attribute P(X) given Z

14
Contextual Effect on Unconditional Probability (3)
P( P (Xk ) Xt ) Xt1 Xt2
P1(Xk ) 0.4 0.9
P2(Xk ) 0.6 0.1
Xt1 I am in Paris Xt2 I am in Moscow
xt
P1(Xk)
P2(Xk)
0.7
0.5
0.3
Xk
Xk
0.2
Xk1
Xk2
Xk1
Xk2
Xk1 order flowers Xk2 order wine
xk
Xk Order present
15
Causal Relation between Conditional Probabilities
xm
xn
P(P(Xn Xm))
P(Xn Xm)
P2(XnXm)
P3(XnXm)
P1(XnXm)
P(P(Xr Xk)P(Xn Xm))
P(P(Xr Xk))
There might be causal relationship between two
pairs of conditional probabilities
P(Xr Xk)
P2(XrXk)
P1(XrXk)
xk
xr
16
Two-level Bayesian C-Metanetwork for managing
conditional dependencies
17
Example of Bayesian C-Metanetwork
The nodes of the 2nd-level network correspond to
the conditional probabilities of the 1st-level
network P(BA) and P(YX). The arc in the
2nd-level network corresponds to the conditional
probability P(P(YX)P(BA))
18
Two-level Bayesian R-Metanetwork for Modelling
Relevant Features Selection
19
Feature relevance modelling (1)
We consider relevance as a probability of
importance of the variable to the inference of
target attribute in the given context. In such
definition relevance inherits all properties of a
probability.
20
Feature relevance modelling (2)
X x1, x2, , xnx
21
Example (1)

• Let attribute X will be state of weather and
  attribute Y, which is influenced by X, will be
  state of mood.
• X (state of weather) sunny, overcast,
  rain
• P(Xsunny) 0.4
• P(Xovercast) 0.5
• P(Xrain) 0.1
• Y (state of mood) good, bad
• P(YgoodXsunny)0.7
• P(YgoodXovercast)0.5
• P(YgoodXrain)0.2
• P(YbadXsunny)0.3
• P(YbadXovercast)0.5
• P(YbadXrain)0.8

P(X)
Let ?X0.6
P(YX)
22
Example (2)

• Now we have
• One can also notice that these values belong to
  the intervals created by the two extreme cases,
  when parameter X is not relevant at all or it is
  fully relevant

!
23
General Case of Managing Relevance (1)
Predictive attributes   X1 with values
x11,x12,,x1nx1 X2 with values
x21,x22,,x2nx2 XN with values
xn1,xn2,,xnnxn   Target attribute   Y with
values y1,y2,,yny.   Probabilities P(X1),
P(X2),, P(XN) P(YX1,X2,,XN).   Relevancies ?X
1 P(?(X1) yes) ?X2 P(?(X2)
yes) ?XN P(?(XN) yes) Goal to
estimate P(Y).
24
General Case of Managing Relevance (2)
Probability P(XN)
25
Example of Relevance Bayesian Metanetwork (1)
Conditional relevance !!!
26
Example of Relevance Bayesian Metanetwork (2)
27
Example of Relevance Bayesian Metanetwork (3)
28
When Bayesian Metanetworks ?

• Bayesian Metanetwork can be considered as very
  powerful tool in cases where structure (or
  strengths) of causal relationships between
  observed parameters of an object essentially
  depends on context (e.g. external environment
  parameters)
• Also it can be considered as a useful model for
  such an object, which diagnosis depends on
  different set of observed parameters depending on
  the context.

29
Conclusion

• We are considering a context as a set of
  contextual attributes, which are not directly
  effect probability distribution of the target
  attributes, but they effect on a relevance of
  the predictive attributes towards target
  attributes.
• In this paper we use the Bayesian Metanetwork
  vision to model such context-sensitive feature
  relevance. Such model assumes that the relevance
  of predictive attributes in a Bayesian network
  might be a random attribute itself and it
  provides a tool to reason based not only on
  probabilities of predictive attributes but also
  on their relevancies.

30
Read more about Bayesian Metanetworks in
Terziyan V., A Bayesian Metanetwork, In
International Journal on Artificial Intelligence
Tools, Vol. 14, No. 3, 2005, World Scientific,
pp. 371-384.
http//www.cs.jyu.fi/ai/papers/IJAIT-2005.pdf
Terziyan V., Vitko O., Bayesian Metanetwork for
Modelling User Preferences in Mobile Environment,
In German Conference on Artificial Intelligence
(KI-2003), LNAI, Vol. 2821, 2003, pp.370-384.
http//www.cs.jyu.fi/ai/papers/KI-2003.pdf
Terziyan V., Vitko O., Learning Bayesian
Metanetworks from Data with Multilevel
Uncertainty, In M. Bramer and V. Devedzic
(eds.), Proceedings of the First International
Conference on Artificial Intelligence and
Innovations, Toulouse, France, August 22-27,
2004, Kluwer Academic Publishers, pp. 187-196 .
http//www.cs.jyu.fi/ai/papers/AIAI-2004.ps