true error less

1
(No Transcript)
2
(No Transcript)
3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
true error less
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
true error
training error
degree of overfitting
19
What if H is not finite?

• Cant use our result for finite H
• Need some other measure of complexity for H
• Vapnik-Chervonenkis dimension!

20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27

• Initialize VS to all hypotheses in H
• For each training example,
• remove from VS all hyps. that misclassify this
  example

28
(No Transcript)
29
Weighted Majority Algorithm
when b0, equivalent to the Halving algorithm
30
Even algorithms that learn or change over time