?????????????Power Set Kernel ??????

1
?????????????Power Set Kernel ??????

• ?????????????
• ???????
• ?? ?
• ?? ??

2
??

• SVM ??????? Kernel Method ????????
• ?????????????
• ?????
• ??????
• ????
• Kernel Method ???????

3
Kernel Method ????

• ???????????
• ????????????
• ?????(???????)????????????? (?????)
• ??????????
• Kernel Method ???????????????????????????????????
• ???????? 0.5 1 ?

GOAL ??2???????
4
??????

• Kernel Method
• Power Set Kernel
• Power Set Kernel ??????
• PSKB (??????)
• PSKI (???? 1)
• PSKE (???? 2)
• ????????????????
• ????????

5
??????

• Kernel Method
• Power Set Kernel
• Power Set Kernel ??????
• PSKB (??????)
• PSKI (???? 1)
• PSKE (???? 2)
• ????????????????
• ????????

6
Kernel Method

• ?????????? Kernel ??????
• ????????????????
• ?? x ??????? Kernel ???
• (??, ????, ??, ?, ???)

7
Kernel Method ????

• ??? O(Lm) ???? (m ? K(,)????)
• ???????????????, ??????
• ?????

8
??????

• Kernel Method
• Power Set Kernel
• Power Set Kernel ??????
• PSKB (??????)
• PSKI (???? 1)
• PSKE (???? 2)
• ????????????????
• ????????

9
Power Set

• ????????????????????(Power Set)???
• ?? X ? Power Set ? P(X)???

• ? Xa,b,c
• P(X)f,a,b,c,a,b,a,c,
• b,c,a,b,c

10
(Special) Power Set Kernel

• ????????? Kernel
• X, Z ???
• X ????? X???
• K(X, Z) P(X) n P(Z)

? X a,b,c, Za,b,d P(X)
f,a,b,c,a,b,a,c,b,c,a,b,c P(Z)
f,a,b,d,a,b,a,d,b,d,a,b,d P(X)nP(Z
)f,a,b,a,b K(X,Z) P(X) n
P(Z) 4
11
Power Set Kernel (PSK)
(??????)
(?) X a,b,c, Za,b,d, C02, C13, C21,
C30 P(X) f,a,b,c,a,b,a,c,b,c,a,b,
c P(Z) f,a,b,d,a,b,a,d,b,d,a,b,d
P(X)nP(Z)f,a,b,a,b K(X,Z)
21 32 11 00 9
12
PSK ?????

• X,Z??????, ??????????, ? PSK ???
• X,Z??????, ? n????????
  ??????????,
  ?PSK???
  
• ???? PSK K(X,Z) ?, XnZ??????????
• Cr ?????????, PSK ??????

13
???Kernel
??2????, Power Set Kernel
? d2 ??? C0 1, C13, C22 d3 ??? C0
1, C17, C212, C3 6
??? dk ???
14
RBF Kernel
XZ?? (??????) ???
??2????, Power Set Kernel
15
??????

• Kernel Method
• Power Set Kernel
• Power Set Kernel ??????
• PSKB (??????)
• PSKI (???? 1)
• PSKE (???? 2)
• ????????????????
• ????????

16
Power Set Kernel ????

• PSKB
• ???????(??????)
• PSKI (Inverted representation)
• ?? ????????????
• PSKE (Expanded representation)
• ??? Power Set ??????

17
PSKB (??????)
3
K(X,Z) (XnZ1)
X
a
a, b, c a, b, d b, c, d
K(X,Z)
1 0.5 -2
??????? Za,c,e
1 2 3
3
3
3
f(Z)1(21) 0.5(11) - 2 (11)
15 ?????? O(LZ)
18
PSKI (Inverted Representation)
3
K(X,Z) (XnZ1)
X
a
Inverted Representation
??????? Z a, c, e
a b c d
1,2 1,2,3 1,3 2,3
a, b, c a, b, d b, c, d
1 0.5 -2
1 2 3

3
3
3
f(Z)1(21) 0.5(11) - 2 (11) 15

• ????, O(BZL), ?? O(LZ)
• ???????????(????????)

19
PSKE (Expanded Representation) (1/2)

• PSK ???? ?, ?? X ?, Power Set ?
• ?????????????????????
• w ????????????, ???????
• (Power Set ??)?????????
• ?? 2002 ???????

20
PSKE (Expanded Representation) (2/2)
C
w
Expansion Table
3
1
f a b c d a,b a,c a,d b,c b,d
c,d a,b,c a,b,d a,c,d b,c,d
-0.5 10.5 -3.5 -7 -10.5 12 12 6 -12 -18 -24
6 3 0 -12
K(X,Z) (XnZ1)
??????? Za,c,e
7
C01, C17, C212, C36
X
a
12
P(Z)f,a,c, e,a,c,a,e,
c,e,a,c,e
??
a, b, c a, b, d b, c, d
1 0.5 -2
1 2 3
F(z)-0.510.5-712 15
6

• ???? O(P(Z)), ?????????
• ????????????
• d????? Kernel ? d????????

21
PSKE???

• Expansion Table ???
• ??? d????????????????????
• ??2002? 2????????????
• w?, ????????????????. w?????????????(??)
• ?????????????????
• Expansion Table ???
• ??????????? TRIE ???

22
??????????????
w?s ??????????? ???? ???? ????
C
w
?
1
-0.5 10.5 -3.5 -7 -10.5 12 12 6 -12 -18 -24
6 3 0 -12
f a b c d a,b a,c a,d b,c b,d
c,d a,b,c a,b,d a,c,d b,c,d
C
w
7
a d a,b a,c b,c b,d c,d b,c,d
10.5 -10.5 12 12 -12 -18 -24 -12
7
s10
12
12
6
6
23
??????????(1/3)

• a????, ???????????
• ??(a?0),??(alt0)???????????????????
• ???? p ??? w ? (?????)-(?????)
• ??,????????????s???????????????, ????????????????
• ??? r ????????? Cr ????
• ????? Cmaxmax(C0,C1,..,Cx) ???, ??????
  Cmax?????
• ??????, ????p ???? Cp/ Cmax ?

24
?????????? (3/3)
3
?????????? (Apriori, PrefixSpan)
K(X,Z) (XnZ1)
Cmaxa? ??????
C01, C17,C212 C36, Cmax12
f 18 a 18 b 18 c 12 a,b 18 a,c
12 b,c 12
X
??
merge
??
s15 s??10 s??5
a, b, c a, b, d
12 6
1 2
a 10.5 d 10.5 a,b 12 a,c 12 b,c
-12 b,d -18 c,d -24 b,c,d -12
Minsup10
X
a
f 24 b 24 c 24 d 24 b,c 24 b,d
24 c,d 24 b,c,d 24
??
X
??
a, b, c a, b, d b, c, d
1 0.5 -2
1 2 3
b, c, d
24
3
w (f??-f??) Cp/Cmax
(?) b ??? W(18-24)7/1210.5
Minsup5
25
TRIE?????
root
w
C
a d a,b a,c b,c b,d c,d b,c,d
10.5 -10.5 12 12 -12 -18 -24 -12
7
a
d
b
c
10.5
-10.5
12
c
c
d
d
b
-24
12
12
-18
-12
6
d
-12

• ?? Prefix ???
• TRIE ??????????????

26
??????

• Kernel Method
• Power Set Kernel
• Power Set Kernel ??????
• PSKB (??????)
• PSKI (???? 1)
• PSKE (???? 2)
• ??? ?????, ?????????????
• ??, ?????

27
??

• ??? ?????/??????
• 3????? Kernel
• ??????PrefixSpan Pei? 2001

??? ????? ????
??? 265,413 110,355
SV? 57,672 34,996
????? 11,643 28,157
Xj??? 11.73 17.63
XEON 2.4GHz, Linux, C ?????
28
????(?????)
s ???? (?/?) ???() ?????(k?)
PSKB - 0.8535 97.94 -
PSKI - 0.4989 97.94 -
PSKE 0.1 0.0009 96.09 21
PSKE 0.05 0.0014 97.36 84
PSKE 0.01 0.0028 97.93 1,228
PSKE 0.005 0.0033 97.95 2,327
PSKE 0.001 0.0038 97.94 4,392
PSKE 0.0005 0.0041 97.94 4,820
PSKE 0.0001 0.0042 97.94 5,206
29
????(??????)
s ???? (?/?) ???() ?????(k?)
PSKB - 0.2848 89.29 -
PSKI - 0.0811 89.29 -
PSKE 0.1 0.0013 82.02 7
PSKE 0.05 0.0020 86.27 30
PSKE 0.01 0.0050 88.91 739
PSKE 0.005 0.0067 89.05 1,924
PSKE 0.001 0.0092 89.26 6,686
PSKE 0.0005 0.0097 89.29 8,262
PSKE 0.0001 0.0101 89.29 9,846
30
??

• PSKI ? ? 3-12?,
  PSKE ? ? 30-280??? ??
• s0.0005????????????? 826?, TRIE????? 391MB
  (??????)
• ????????????
• ?????????????????????????(1,2,3..)??????????,
  ?????
• ??????? ??????????????????????
• s0.0005????, ???????

31
??????????????(????)
? ???? (?/?) ???() ?????(k?) TRIE????(MB)
PSKB - 0.2848 89.29 - -
PSKI - 0.0810 89.29 - -
PSKE s0.005 1 0.0097 82.29 8,262 391.1
PSKE s0.005 2 0.0074 89.34 2,450 118.0
PSKE s0.005 3 0.0069 89.31 1,360 66.4
PSKE s0.005 4 0.0065 89.25 945 46.8

• ????????????????1/3?
• ????????????????????????

32
???

• Power Set Kernel ???????
• Power Set Kernel ????
• PSKI (?????????????)
• PSKE (Power Set ??????)
• PSKI ? 3-12???? PSKE ?
  30-280??????????

33
?????

• ?????? Cr ???
• Cr ?????? ? Cr???? Kernel
• ??????????????????
• ????????????

34
PSKE???????????? 3?????
? ???-??-?????-????, ???-???-??-?,
???-??-?????-???? ????? ? ????????? ?
???-???-??-?, ???-??-??-??, ???-???-??-?
?? ???? ??????, ???????????????????
? ???-???-??-??, ???-???-??-??,
???-???-??-?? ??? ??? ??????
35
???

• PSKI ???? ?? ????? YamCha ?, ??????????? CaboCha
  ????????
• PSKE ?????????, ?????????????? (?????)

36
?????? Cr (1/2)

• ???r ?????(r?????????)???????????
• ???? ?? r ?????

root
r0
???

• ????? r ???
• Cr? ???

r1
a
b
c
d
Cr

b
c
d
r2
???
r
r3
0 1 2 3 4
37
?????? Cr (2/2)
Cr
??? K(X,Z)(XnZ1) Quasi-RBF K(X,Z)exp
(2sXnZ)
d
r
s?
Cr
s?
r

• ??? Kernel ??????Cr???
• ????? d ???????????
• RBF ??????(??)???????
• ??????????, RBF ?? ??? Kernel ?
• ???????????????