IBM 704 (1954)

1
?????????????????????

• ???????????????????????
• ????

2
?????????????????

• IBM 704 (1954)
• ????CPY?????
• ????????????????
• FORTRAN I (1956)
• ?Hollerith field??FORMAT????
• ????????????
• ?????????????

3
???(string)???

• ALGOL 60 (1960)
• BNF???????
• ?string??1????????????

ltproper stringgt ltany sequence of basic
symbols not containing or gtltemptygt ltopen
stringgt ltproper stringgtltopen stringgtltopen
stringgtltopen stringgt ltstringgt ltopen stringgt
????????string??????????
4
?????????

• IBM System/360 (1964)
• ??????8bit1byte?
• 1??1byte (EBCDIC)
• PL/I (1964)
• CHARACTER????
• ???????????????

5
??????????????

• Morris-Pratt (1970)
• ?????????????????????
• Knuth-Morris-Pratt (1974)
• Morris-Pratt??????
• Aho-Corasick (1975)
• ????????????????

6
Aho-Corasick??????

• ex) ITALY?TIE?TULIA???

INSTITUTE FOR RESEARCH IN HUMANITIES
7
Aho-Corasick??????

• ex) ITALY?TIE?TULIA???

INSTITUTE FOR RESEARCH IN HUMANITIES
8
Aho-Corasick??????

• ex) ITALY?TIE?TULIA???

INSTITUTE FOR RESEARCH IN HUMANITIES
9
Aho-Corasick??????

• ex) ITALY?TIE?TULIA???

INSTITUTE FOR RESEARCH IN HUMANITIES
10
Aho-Corasick??????

• ex) ITALY?TIE?TULIA???

INSTITUTE FOR RESEARCH IN HUMANITIES
11
Aho-Corasick??????

• ex) ITALY?TIE?TULIA???

INSTITUTE FOR RESEARCH IN HUMANITIES
12
Aho-Corasick??????

• ex) ITALY?TIE?TULIA???

INSTITUTE FOR RESEARCH IN HUMANITIES
13
Aho-Corasick??????

• ex) ITALY?TIE?TULIA???

INSTITUTE FOR RESEARCH IN HUMANITIES
14
Aho-Corasick??????

• ex) ITALY?TIE?TULIA???

INSTITUTE FOR RESEARCH IN HUMANITIES
15
???????????

• Boyer-Moore (1977)
• ???????????????????
• ????????????????????
• Commentz-Walter (1979)
• ??????????Boyer-Moore???

16
Commentz-Walter??????

• ex) ITALY?TIE?TULIA???

L 1
I 1
T 2
U 3
??? ??? 3
INSTITUTE FOR RESEARCH IN HUMANITIES
17
Commentz-Walter??????

• ex) ITALY?TIE?TULIA???

L 1
I 1
T 2
U 3
??? ??? 3
INSTITUTE FOR RESEARCH IN HUMANITIES
18
Commentz-Walter??????

• ex) ITALY?TIE?TULIA???

L 1
I 1
T 2
U 3
??? ??? 3
INSTITUTE FOR RESEARCH IN HUMANITIES
19
Commentz-Walter??????

• ex) ITALY?TIE?TULIA???

L 1
I 1
T 2
U 3
??? ??? 3
INSTITUTE FOR RESEARCH IN HUMANITIES
20
Commentz-Walter??????

• ex) ITALY?TIE?TULIA???

L 1
I 1
T 2
U 3
??? ??? 3
INSTITUTE FOR RESEARCH IN HUMANITIES
21
Commentz-Walter??????

• ex) ITALY?TIE?TULIA???

L 1
I 1
T 2
U 3
??? ??? 3
INSTITUTE FOR RESEARCH IN HUMANITIES
22
????????????

• 1???1byte
• ex) ????1??????EUC???

? ? ? 1 ? B0 C2 B2 AC
A4 CE 31 C2 B2
????????????????????
23
????????????

• 1???1byte
• ??????????
• ex) ?????????
• ?????????????????????

??????? ????????? ??????? ???????
???????????????
24
????????????

• ??-?? (1985)
• Aho-Corasick???????????
• ex) ???EUC???????????????

25
??????????????

• ???EUC????JIS?????
• UTF-8???(1993)
• ISO 10646 (Unicode)??????
• 1???16??????
• 1??????007F?C0FE???
• 2????????80BF???

? ? ? 1 ? E5 AE 89
E5 B2 A1 E3 81 AE 31 E6 97 8F
? ? ? 1 ? E5 AE 89
E5 B2 A1 E3 81 AE 31 E6 97 8F
26
UTF-8??????????

• Commentz-Walter?????
• ? ???????????????????
• ex) ???????????????

88 1
96 2
A4 5
A6 1
A7 4
AD 2
B8 1
E5 3
E6 3
? 6
27
????????????

• 1???1byte
• ??????????
• ??????1???

28
????????1???

• ???????????

??????
29
????????1???

• ???????????

??????
30
????????1???

• ???????????

??????
31
????????1???

• ???????????

32
????????1???

• ???????????

• ?????????????

????(????)????????
????(????)????????
???
33
????????1???

• ???????????

• ?????????????

????(????)????????
????(????)????????
??????
34
????????1???

• ???????????

• ?????????????

????(????)????????
35
????????????????

• ??????1???????????
• XML/XHTML????
• ex) XHTML????Ruby Annotation (2001)

ltpgt??ltruby xmllangjagt ltrbcgtltrbgt?lt/rbgtltrbgt?lt/rb
gtltrbgt?lt/rbgtltrbgt?lt/rbgtlt/rbcgt ltrtcgtltrtgt??lt/rtgtltrtgt??
lt/rtgtltrtgt??lt/rtgtltrtgt??lt/rtgtlt/rtcgt lt/rubygt???lt/pgt
36
ltpgt??ltruby xmllangjagt ltrbcgtltrbgt?lt/rbgtltrbgt?lt/rb
gtltrbgt?lt/rbgtltrbgt?lt/rbgtlt/rbcgt ltrtcgtltrtgt??lt/rtgtltrtgt??
lt/rtgtltrtgt??lt/rtgtltrtgt??lt/rtgtlt/rtcgt lt/rubygt???lt/pgt
37
?????????????????
38
????????????????

• ??????1???????????
• Directed Acyclic Graph?????????

??????????
39
DAG?????????????

• Aho-Corasick???????
• ????????????
• ????????????????
• Commentz-Walter???????
• ?????????
• ?????????????????
• ??????????????????

40
DAG?????????????

• ??????????????
• ??????????
• Aho-Corasick???????
• ????????????????
• Commentz-Walter???????
• ??????????????????

41
?????

• DAG????????????????
• ??????????????
• ???????????tight??????
• ???????????????????

42
?????????????????????

• ???????????????????????
• ????