Title: Probably Approximately Correct Learning
1Probably Approximately Correct Learning pac Model
fixed but unknown
distribution
according to an
- When we evaluate the quality of a hypothesis
(classification function)
we should take the
into account
unknown
distribution
error or expected error
)
made by the
- We call such measure risk functional and denote
it as
2 Generalization Error of pac Model
3Probably Approximately Correct
or
4Find the Hypothesis with Minimum Expected Risk?
should has the smallest
expected risk
Unrealistic !!!
5Empirical Risk Minimization (ERM)
are not needed)
(
and
- Only focusing on empirical risk will cause
overfitting
6VC Confidence
(The Bound between )
7Capacity (Complexity) of Hypothesis Space
VC-dimension
8 Shattering Points with Hyperplanes in
Can you always shatter three points with a line in
?
9Definition of VC-dimension
- The Vapnik-Chervonenkis dimension,
, of
hypothesis space
defined over the input space
is the size of the (existent) largest finite
subset
shattered by
of