Learning Patterns on the World Wide Web

1
Learning Patterns on the World Wide Web

• Andrew Hogue
• Advisor David Karger
• October 17, 2003

2
Agenda

• What is a pattern?
• How do we make one?
• How do we use it?
• Why do you want one?
• Demo

3
What is a pattern?

• Objects in the world have certain semantic
  properties
• A pattern is a way of recognizing the semantic
  properties of an object weve seen before
• A pattern is a structure with semantic slots to
  be filled in

4
Example Books

• Define an objects semantics (ontology)
• Class Book
• Property Author
• Property Title
• Property Price
• Property Publisher
• Property ISBN
• . . .

5
Example - Books
Class Book Property Author Property
Title Property Price Property
Publisher Property ISBN . . .
?
?
6
Example - Books
Class Book Property Author Property
Title Property Price Property Publisher
Property ISBN . . .
7
Example - Books
Class Book Property Author Property
Title Property Price Property Publisher
Property ISBN . . .
8
Creating a Pattern

• Choose positive examples

9
Creating a Pattern
10
Creating a Pattern
11
Creating a Pattern

• Choose positive examples
• Find best mapping between examples

12
Creating a Pattern
13
Creating a Pattern

• Choose positive examples
• Find best mapping between examples
• Merge mapped elements and assign semantic labels

14
Creating a Pattern
15
Creating a Pattern
16
Creating a Pattern
17
Creating a Pattern
18
Creating a Pattern

• Choose positive examples
• Find best mapping between examples
• Merge mapped elements and assign semantic labels
• Eliminate unmapped elements

19
Creating a Pattern
20
Creating a Pattern
21
Matching Patterns

• Given a pattern with slots and a page to search
• Look for items on page with same structure
• Map pattern slots to page text

22
Matching Patterns
23
Matching Patterns
24
Applications

• Extract search engine results
• Extract and email news headlines
• Watch sites for updates
• Reformat sites for easier reading
• Monitor bank account balances

25
Demo
26
More Information

• http//haystack.lcs.mit.edu
• ahogue_at_mit.edu