Linear SVM

1
Linear SVM
From perceptron to kernel methods
Following Scholkopf, see SVM site
Course website http//horn.tau.ac.il/course06.htm
l
2
(No Transcript)
3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
Cluster Analysis

• The clustering problempartition a data-set into
  a number of groups such that points within each
  group are alike in some sense.
• Approaches
• Hierarchical algorithms - cut the dendrogram at a
  certain level.
• Partitional algorithms.
• Parametric methods.
• K-means.
• Mixture models.
• Nonparametric methods.
• Graph theoretic methods.
• Information theoretic.
• Algorithms based on statistical physics.
• Density estimation methods.

46
Support Vector Clustering

• Given points x in data space, define images in
  Hilbert space.
• Require all images to be enclosed by a minimal
  sphere in Hilbert space.
• Reflection of this sphere in data space defines
  cluster boundaries.
• Two parameters width of Gaussian kernel and
  fraction of outliers

Ben-Hur, Horn, Siegelmann Vapnik. JMLR 2 (2001)
125-127
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
Variation of q allows for clustering solutions on
various scales
q1, 20, 24, 48
51
(No Transcript)
52
Example that allows for SVclustering only in
presence of outliers. Procedure limit ß
ltC1/pN, where pfraction of assumed outliers in
the data.
q3.5 p0
q1 p0.3
53
Similarity to scale space approach for high
values of q and p. Probability distribution
obtained from R(x).
q4.8 p0.7
54
(No Transcript)
55
(No Transcript)