Learning Agents Laboratory

1
CS 782 Machine Learning
10 Multistrategy Learning
Prof. Gheorghe Tecuci
Learning Agents Laboratory Computer Science
Department George Mason University
2
Overview
Introduction
Combining EBL with Version Spaces
Induction over Unexplained
Guiding Induction by Domain Theory
Plausible Justification Trees
Research Issues
Basic references
3
Multistrategy learning
Multistrategy learning is concerned with
developing learning agents that synergistically
integrate two or more learning strategies in
order to solve learning tasks that are beyond the
capabilities of the individual learning
strategies that are integrated.
4
Complementariness of learning strategies
Case Study Inductive Learning vs
Explanation-based Learning
Explanation- based learning
Multistrategy
Learningfrom examples
learning
Examples
needed
Knowledge
needed
Type of
inference
Effect on
agent's
behavior
5
Multistrategy concept learning

The Learning Problem
Input Background Knowledge (Domain
Theory) Goal
One or more positive and/or negative examples of
a concept
Weak, incomplete, partially incorrect, or complete
Learn a concept description characterizing the
example(s) and consistent with the background
knowledge by combining several learning strategies
6
Multistrategy knowledge base refinement

The Learning Problem Improve the knowledge base
so that the Inference Engine solves (classifies)
correctly the training examples.
Training
Examples
Improved
Knowledge
Multistrategy
Base (DT)
Knowledge
Knowledge Base
Base (DT)
Refinement
Inference
Engine
Inference
Engine
Similar names background knowledge domain
theory knowledge base knowledge base
refinement - theory revision
7
Types of theory errors (in a rule based system)
How would you call a KB where some positive
examples are not explained (classified as
positive)?
How would you call a KB where some negative
examples are wrongly explained (classified as
positive)?
Incorrect
KB (theory)



_
Overly
Overly
Specific
General
Missing
Extra
Additional
Missing
Rule
Rule
Premise
Premise
width(x, small)
What is the effect of each error on the systems
ability to classify graspable objects, or other
objects that need to be graspable, such as cups?
Positive examples
Negative examples
Proofs for some positive examples cannot be built
Proofs for some negative examples can be built
Positive examples that are not round, or have a
handle
Negative examples that are round, or are
insulating but not small
8
Overview
Introduction
Combining EBL with Version Spaces
Induction over Unexplained
Guiding Induction by Domain Theory
Plausible Justification Trees
Research Issues
Basic references
9
EBL-VS Combining EBL with Version Spaces

Apply explanation-based learning to generalize
the positive and the negative examples. Replace
each example that has been generalized with its
generalization. Apply the version space method
to the new set of examples.
Produce an abstract illustration of this
algorithm.
10
EBL-VS features

Apply explanation-based learning to generalize
the positive and the negative examples. Replace
each example that has been generalized with its
generalization. Apply the version space method
to the new set of examples.
Justify the following feature, considering
several cases
Learns from positive and negative examples
11
EBL-VS features

Apply explanation-based learning to generalize
the positive and the negative examples. Replace
each example that has been generalized with its
generalization. Apply the version space method
to the new set of examples.
Justify the following feature
Can learn with an incomplete background
knowledge
12
EBL-VS features

Apply explanation-based learning to generalize
the positive and the negative examples. Replace
each example that has been generalized with its
generalization. Apply the version space method
to the new set of examples.
Justify the following feature
Can learn with different amounts of knowledge,
from knowledge-free to knowledge-rich
13
EBL-VS features summary and references
Learns from positive and negative examples
Can learn with an incomplete background
knowledge Can learn with different amounts of
knowledge, from knowledge-free to knowledge-rich
References Hirsh, H., "Combining Empirical and
Analytical Learning with Version Spaces," in
Proc. of the Sixth International Workshop on
Machine Learning, A. M. Segre (Ed.), Cornell
University, Ithaca, New York, June 26-27, 1989.
Hirsh, H., "Incremental Version-space Merging,"
in Proceedings of the 7th International Machine
Learning Conference, B.W. Porter and R.J. Mooney
(Eds.), Austin, TX, 1990.
14
Overview
Introduction
Combining EBL with Version Spaces
Induction over Unexplained
Guiding Induction by Domain Theory
Plausible Justification Trees
Research Issues
Basic references
15
IOU Induction Over Unexplained
Justify the following limitation of EBL-VS
Limitation of EBL-VS Assumes that at least one
generalization of an example is correct and
complete IOU Knowledge base could be
incomplete but correct - the explanation-based
generalization of an example may be
incomplete - the knowledge base may explain
negative examples. Learns concepts with both
explainable and conventional aspects
16
IOU method

Apply explanation-based learning to generalize
each positive example Disjunctively combine
these generalizations (this is the explanatory
component Ce) Disregard negative examples not
satisfying Ce and remove the features mentioned
in Ce from all the examples Apply empirical
inductive learning to determine a generalization
of the reduced set of simplified examples (this
is the non-explanatory component Cn) The learned
concept is Ce Cn
17
IOU illustration
Positive examples of cups Cup1, Cup2 Negative
examples Shot-Glass1, Mug1, Can1 Domain Theory
incomplete - contains a definition of a
generalization of the concept to be learned (e.g.
contains a definition of drinking vessels but no
definition of cups) Ce has-flat-bottom(x)
light(x) up-concave(x)
width(x,small) insulating(x) ?
has-handle(x) Ce covers Cup1, Cup2, Shot-Glass1,
Mug1 but not Can1 Cn volume(x,small) Cn covers
Cup1, Cup2 but not Shot-Glass1, Mug1 C Ce Cn
Mooney, R.J. and Ourston, D., "Induction Over
Unexplained Integrated Learning of Concepts with
Both Explainable and Conventional Aspects,", in
Proc. of the Sixth International Workshop on
Machine Learning, A.M. Segre (Ed.), Cornell
University, Ithaca, New York, June 26-27, 1989.
18
Overview
Introduction
Combining EBL with Version Spaces
Induction over Unexplained
Guiding Induction by Domain Theory
Plausible Justification Trees
Research Issues
Basic references
19
Enigma Guiding Induction by Domain Theory
Justify the following limitations of IOU
Limitations of IOU Knowledge base rules have to
be correct Examples have to be
noise-free ENIGMA Knowledge base rules could
be partially incorrect Examples may be noisy
20
Enigma method
Trades-off the use of knowledge base rules
against the coverage of examples Successively
specialize the abstract definition D of the
concept to be learned by applying KB rules
Whenever a specialization of the definition D
contains operational predicates, compare it with
the examples to identify the covered and the
uncovered ones Decide between performing - a
KB-based deductive specialization of D - an
example-based inductive modification of D The
learned concept is a disjunction of leaves of the
specialization tree built.

21
Enigma illustration
Examples (4 positive, 4 negative) Positive
example4 (p4) light(o4) support(o4,b)
body(o4,a) above(a,b) up-concave(o4) ?
Cup(o4) Background Knowledge Liftable(x)
Stable(x) Open-vessel(x) ? Cup(x) light(x)
has-handle(x) ? Liftable(x) has-flat-bottom(x) ?
Stable(x) body(x, y) support(x, z) above(y,
z) ? Stable(x) up-concave(x) ? Open-vessel(x)
KB - partly overly specific (explains only p1
and p2) - partly overly general (explains n3)
Operational predicates start with a lower-case
letter
22
Enigma illustration (cont.)
Classification is based only on operational
features
(to cover p3,p4)
(to uncover n2,n3)
23
Learned concept
light(x) has-flat-bottom(x) has-small-bottom(x)
? Cup(x) Covers p1, p3 light(x) body(x, y)
support(x, z) above(y, z) up-concave(x) ?
Cup(x) Covers p2, p4
24
Application
Diagnosis of faults in electro-mechanical
devices through an analysis of their vibrations
209 examples and 6 classes Typical example 20
to 60 noisy measurements taken in different
points and conditions of the device A learned
rule IF the shaft rotating frequency is w0 and
the harmonic at w0 has high intensity and the
harmonic at 2w0 has high intensity in at least
two measurements THEN the example is an instance
of C1 (problems in the joint), C4 (basement
distortion) or C5 (unbalance)
25
Application (cont.)
Comparison between the KB learned by ENIGMA
and the hand-coded KB of the expert system MEPS
Bergadano, F., Giordana, A. and Saitta, L.,
"Automated Concept Acquisition in Noisy
Environments," IEEE Transactions on Pattern
Analysis and Machine Intelligence, 10 (4), pp.
555-577, 1988. Bergadano, F., Giordana, A.,
Saitta, L., De Marchi D. and Brancadori, F.,
"Integrated Learning in a Real Domain," in B.W.
Porter and R.J. Mooney (Eds. ), Proceedings of
the 7th International Machine Learning
Conference, Austin, TX, 1990. Bergadano, F. and
Giordana, A., "Guiding Induction with Domain
Theories," in Machine Learning An Artificial
Intelligence Approach Vollume 3, Y. Kodratoff and
R.S. Michalski (Eds.), San Mateo, CA, Morgan
Kaufmann, 1990.
26
Overview
Introduction
Combining EBL with Version Spaces
Induction over Unexplained
Guiding Induction by Domain Theory
Plausible Justification Trees
Research Issues
Basic references
27
MTL-JT Multistrategy Task-adaptive
Learning based on Plausible Justification Trees
Deep integration of learning strategies Integrat
ion of the elementary inferences that are
employed by the single-strategy learning methods
(e.g. deduction, analogy, empirical inductive
prediction, abduction, deductive generalization,
inductive generalization, inductive
specialization, analogy-based generalization).
Dynamic integration of learning strategies The
order and the type of the integrated strategies
depend of the relationship between the input
information, the background knowledge and the
learning goal.
Different types of input (e.g. facts, concept
examples, problem solving episodes)
Different types of knowledge pieces in the
knowledge base (e.g. facts, examples, implicative
relationships, plausible determinations)
28
MTL-JT assumptions
Input correct (noise free) one or several
examples, facts, or problem solving episodes
Knowledge Base incomplete and/or partially
incorrect may include a variety of knowledge
types (facts, examples, implicative or causal
relationships, hierarchies, etc.)
Learning Goal extend, update and/or improve
the knowledge base so as to integrate new input
information
29
Plausible justification tree
A plausible justification tree is like a proof
tree, except that some of individual inference
steps are deductive, while others are
non-deductive or only plausible (e.g. analogical,
abductive, inductive).
30
Learning method

For the first positive example I1 - build a
plausible justification tree T of I1 - build the
plausible generalization Tu of T - generalize
the KB to entail Tu For each new positive
example Ii - generalize Tu so as to cover a
plausible justification tree of Ii - generalize
the KB to entail the new Tu For each new
negative example Ii - specialize Tu so as not
to cover any plausible justification of Ii -
specialize the KB to entail the new Tu without
entailing the previous Tu Learn different
concept definitions - extract different concept
definitions from the general justification tree
Tu
31
MTL-JT illustration from Geography
Knowledge Base
Facts terrain(Philippines, flat),
rainfall(Philippines, heavy), water-in-soil(Phil
ippines, high) Examples (of fertile
soil) soil(Greece, red-soil) ? soil(Greece,
fertile-soil) terrain(Egypt, flat)
soil(Egypt, red-soil) ? soil(Egypt,
fertile-soil) Plausible determination rainfall(
x, y) gt water-in-soil(x, z) Deductive
rules soil(x, loamy) ? soil(x, fertile-soil)
climate(x, subtropical) ? temperature(x, warm)
climate(x, tropical) ? temperature(x, warm)
water-in-soil(x, high) temperature(x, warm)
soil(x, fertile-soil) ? grows(x, rice)
32
Positive and negative examples of "grows(x, rice)"
Positive Example 1 rainfall(Thailand, heavy)
climate(Thailand, tropical) soil(Thailand,
red-soil) terrain(Thailand, flat)
location(Thailand, SE-Asia) ? grows(Thailand,
rice) Positive Example 2 rainfall(Pakistan,
heavy) climate(Pakistan, subtropical)
soil(Pakistan, loamy) terrain(Pakistan, flat)
location(Pakistan, SW-Asia) ? grows(Pakistan,
rice) Negative Example 3 rainfall(Jamaica,
heavy) climate(Jamaica, tropical)
soil(Jamaica, loamy) terrain(Jamaica, abrupt)
location(Jamaica, Central-America) ?
grows(Jamaica, rice)
33
Build a plausible justification of the first
example
Example 1 rainfall(Thailand, heavy)
soil(Thailand, red-soil) terrain(Thailand,
flat) location(Thailand, SE-Asia)
climate(Thailand, tropical) ? grows(Thailand,
rice)
34
Build a plausible justification of the first
example
Example 1 rainfall(Thailand, heavy)
soil(Thailand, red-soil) terrain(Thailand,
flat) location(Thailand, SE-Asia)
climate(Thailand, tropical) ? grows(Thailand,
rice)
Justify the inferences from the above tree
analogy
Facts terrain(Philippines, flat),
rainfall(Philippines, heavy), water-in-soil(Phili
ppines, high) Plausible determination rainfall(
x, y) gt water-in-soil(x, z)
35
Build a plausible justification of the first
example
Example 1 rainfall(Thailand, heavy)
soil(Thailand, red-soil) terrain(Thailand,
flat) location(Thailand, SE-Asia)
climate(Thailand, tropical) ? grows(Thailand,
rice)
Justify the inferences from the above tree
deduction
Deductive rules soil(x, loamy) ? soil(x,
fertile-soil) climate(x, subtropical) ?
temperature(x, warm) climate(x, tropical) ?
temperature(x, warm) water-in-soil(x, high)
temperature(x, warm) soil(x, fertile-soil) ?
grows(x, rice)
36
Build a plausible justification of the first
example
Example 1 rainfall(Thailand, heavy)
soil(Thailand, red-soil) terrain(Thailand,
flat) location(Thailand, SE-Asia)
climate(Thailand, tropical) ? grows(Thailand,
rice)
Justify the inferences from the above tree
inductive prediction abduction
Examples (of fertile soil) soil(Greece,
red-soil) ? soil(Greece, fertile-soil)
terrain(Egypt, flat) soil(Egypt, red-soil) ?
soil(Egypt, fertile-soil)
37
Multitype generalization
38
Multitype generalization
Justify the generalizations from the above tree
generalization based on analogy
Facts terrain(Philippines, flat),
rainfall(Philippines, heavy), water-in-soil(Phili
ppines, high) Plausible determination rainfall(
x, y) gt water-in-soil(x, z)
39
Multitype generalization
Justify the generalizations from the above tree
Inductive generalization
Examples (of fertile soil) soil(Greece,
red-soil) ? soil(Greece, fertile-soil)
terrain(Egypt, flat) soil(Egypt, red-soil) ?
soil(Egypt, fertile-soil)
40
Build the plausible generalization Tu of T
41
Positive example 2
Instance of the current Tu corresponding to
Example 2
Plausible justification tree T2 of Example 2
42
Positive example 2
The explanation structure S2
The new Tu
43
Negative example 3
Instance of Tu corresponding to the Negative
Example 3
The new Tu
44
The plausible generalization tree corresponding
to the three input examples
45
Learned knowledge
New facts water-in-soil(Thailand,
high) water-in-soil(Pakistan, high)
Why is it reasonable to consider these facts to
be true?
46
Learned knowledge
New plausible rule soil(x, red-soil) ? soil(x,
fertile-soil)
Examples (of fertile soil) soil(Greece,
red-soil) ? soil(Greece, fertile-soil)
terrain(Egypt, flat) soil(Egypt, red-soil) ?
soil(Egypt, fertile-soil)
47
Learned knowledge
Specialized plausible determination rainfall(x,
y) terrain(x, flat) gt water-in-soil(x, z)
Facts terrain(Philippines, flat),
rainfall(Philippines, heavy), water-in-soil(Philip
pines, high)
Positive Example 1 rainfall(Thailand, heavy)
climate(Thailand, tropical) soil(Thailand,
red-soil) terrain(Thailand, flat)
location(Thailand, SE-Asia) ? grows(Thailand,
rice) Positive Example 2 rainfall(Pakistan,
heavy) climate(Pakistan, subtropical)
soil(Pakistan, loamy) terrain(Pakistan, flat)
location(Pakistan, SW-Asia) ? grows(Pakistan,
rice) Negative Example 3 rainfall(Jamaica,
heavy) climate(Jamaica, tropical)
soil(Jamaica, loamy) terrain(Jamaica, abrupt)
location(Jamaica, Central-America) ?
grows(Jamaica, rice)
48
Learned knowledge concept definitions
Operational definition of "grows(x,
rice)" rainfall(x,heavy) terrain(x,flat)
climate(x,tropical) ? climate(x,subtropical)
soil(x,red-soil) ? soil(x,loamy) ?
grows(x, rice) Abstract definition of "grows(x,
rice)" water-in-soil(x, high) temperature(x,
warm) soil(x, fertile-soil) ? grows(x, rice)
49
Learned knowledge example abstraction
Abstraction of Example 1 water-in-soil(Thailand,
high) temperature(Thailand, warm)
soil(Thailand, fertile-soil) ? grows(Thailand,
rice)
50
Features of the MTL-JT method and reference
Is general and extensible Integrates
dynamically different elementary inferences
Uses different types of generalizations Is able
to learn from different types of input Is able
to learn different types of knowledge Exhibits
synergistic behavior May behave as any of the
integrated strategies
Tecuci, G., "An Inference-Based Framework for
Multistrategy Learning," in Machine Learning A
Multistrategy Approach Volume 4, R.S. Michalski
and G. Tecuci (Eds.), San Mateo, CA, Morgan
Kaufmann, 1994.
51
Features of the MTL-JT method
Justify the following feature
Integrates dynamically different elementary
inferences
52
Features of the MTL-JT method
Justify the following features
May behave as any of the integrated strategies
What strategies should we consider for the
presented illustration of MTL-PJT?
Explanation-based learning Multiple-example
explanation-based learning Learning by
abduction Learning by analogy Inductive learning
from examples
53
MTL-JT as explanation-based learning
Assume the KB would contain the knowledge
"x, rainfall(x, heavy) ? water-in-soil(x,
high) "x, soil(x, red-soil) ? soil(x,
fertile-soil)
54
MTL-JT as abductive learning
Assume the KB would contain the knowledge
"x, rainfall(x, heavy) ? water-in-soil(x, high)
55
MTL-JT as inductive learning from examples
56
MTL-JT as analogical learning
Let us suppose that the KB contains only the
following knowledge that is related to Example
1 Facts rainfall(Philippines, heavy),
water-in-soil(Philippines, high) Determination
rainfall(x, y) --gt water-in-soil(x, z) Then the
system can only infer that "water-in-soil(Thailand
, high)", by analogy with "water-in-soil(Philippin
es, high)". In this case, the MTL-JT method
reduces to analogical learning.
57
Overview
Introduction
Combining EBL with Version Spaces
Induction over Unexplained
Guiding Induction by Domain Theory
Plausible Justification Trees
Research Issues
Basic references
58
Research issues in multistrategy learning
Comparisons of learning strategies New ways
of integrating learning strategies Synergistic
integration of a wide range of learning
strategies The representation and use of
learning goals in multistrategy systems Dealing
with incomplete or noisy examples Evaluation of
the certainty of the learned knowledge General
frameworks for multistrategy learning More
comprehensive theories of learning
Investigation of human learning as multistrategy
learning Integration of multistrategy learning
and knowledge acquisition Integration of
multistrategy learning and problem solving

59
Exercise

• Compare the following learning strategies
• -Rote learning
• Inductive learning from examples
• Explanation-based learning
• Abductive learning
• Analogical learning
• Instance-based learning
• Case-based learning
• From the point of view of their input, background
  knowledge, type of inferences performed, and
  effect on systems performance.

60
Exercise
Identify general frameworks for multistrategy
learning, based on the multistrategy learning
methods presented.
61
Basic references
Proceedings of the International Conferences on
Machine Learning, ICML-87, , ICML-04, Morgan
Kaufmann, San Mateo, 1987- 2004. Proceedings of
the International Workshops on Multistrategy
Learning, MSL-91, MSL-93, MSL-96, MSL-98.
Special Issue on Multistrategy Learning, Machine
Learning Journal, 1993. Special Issue on
Multistrategy Learning, Informatica, vol. 17.
no.4, 1993. Machine Learning A Multistrategy
Approach, Volume IV, Michalski R.S. and Tecuci G.
(Eds.), Morgan Kaufmann, San Mateo, 1994.