Ch. 19

1
Ch. 19 Knowledge in Learning

• Supplemental slides for CSE 327
• Prof. Jeff Heflin

2
Current Best Hypothesis Search

• function CURRENT-BEST-LEARNING(examples) returns
  a hypothesis H ? any hypothesis consistent with
  the first example in examples for each remaining
  example in examples do if e is false positive
  for H then H ? choose a specialization of H
  consistent with examples else if e is false
  negative for H then H ? choose a
  generalization of H consistent with
  examples if no consistent specialization/genera
  lization can be found then fail return H
• Note here choose is a special operator that
  allows you to backtrack to a previous choice and
  select another option when the search fails. An
  actual implementation would probably use
  depth-first search instead.
• From Figure 19.2, p. 681

3
Example Learning Problem
(current best hypothesis search)
Training Set
Example Descriptions Classifications
X1 Color(X1,Red) ? Size(X1,Large) ? Shape(X1,Circle) Q(X1)
X2 Color(X2,Blue) ? Size(X2,Large) ? Shape(X2,Square) ?Q(X2)
X3 Color(X3,Red) ? Size(X3,Small) ? Shape(X3,Square) ?Q(X3)
X4 Color(X4,Green) ? Size(X4,Large) ? Shape(X4,Triangle) ?Q(X4)
X5 Color(X5,Red) ? Size(X5,Small) ? Shape(X5,Circle) Q(X5)
Only consider candidate definitions that are
positive conjunctive sentences
4
Current-Best Hypothesis Search
1
True
hypothesis
X1 ok
example status
2
FP false positive
True
FN false negative
X2 FP
3
Color(x,Red)
Shape(x,Circle)
X3 FP
X3 ?
4
6
Color(x,Red) ? Size(x,Large)
Color(x,Red) ? Shape(x,Circle)
X4 ok
X4 ok
Color(x,Red) ? Size(x,Large)
Color(x,Red) ? Shape(x,Circle)
5
7
X5 FN
X5 ok
5
Version Space Learning

• function VERSION-SPACE-LEARNING(examples) returns
  a version space local variables V, the version
  space (the set of all hypotheses)
• V ? the set of all hypotheses for each example
  e in examples do if V is not empty then V ?
  VERSION-SPACE-UPDATE(V,e) return V
• function VERSION-SPACE-UPDATE(V,e) returns an
  updated version space
• V ? h ? V h is consistent with e return V
• From Figure 19.3, p. 683

6
Version Space Update Details
function VERSION-SPACE-UPDATE(G,S,e) returns an
updated G-set and S-set (version space) for
each g in G if e is a false positive for g G
? G g G ? G ? h h is the most general
specialization of g that is consistent
with e and h is more general than some member of
S else if e is a false negative for g G ? G
g for each s in S if e is a false positive
for s S ? S s else if e is a false
negative for s S ? S s S ? S ? h h is
the most specific generalization of s that is
consistent with e and h is more specific
than some member of G return G,S
7
Example Learning Problem
(version space learning)
Training Set
Descriptions Classifications
Size(X1,Large) ? Shape(X1,Circle) ? Color(X1,Red) Q(X1)
Size(X2,Large) ? Shape(X2,Square) ? Color(X2,Blue) ?Q(X2)
Size(X3,Small) ? Shape(X3,Circle) ? Color(X3,Red) Q(X3)
Size(X4,Small) ? Shape(X4,Circle) ? Color(X4,Blue) ?Q(X4)
Size(X5,Large) ? Shape(X5,Square) ? Color(X5,Red) ?Q(X5)
Only consider candidate definitions that are
positive conjunctive sentences