OpenCalais

1
CS221 Fall 2008, Final Project Architecture
Alexander Behm Arjun Satish
WebServer (Tomcat)
Berkeley DB
Hadoop Distributed File System MapReduce
Word Index
Title Index
Word Index
Title Index
Wikipedia Dump (XML)
Title Index Basis (Articles)
One Time Build
Title Index Basis (Images)

• HTTP Servlet
• Call OpenCalais
• Run Queries for Entities
• Print Results

Inverted Index (Articles)
Per Request
Inverted Index (Images)
HTTPPOST

• MapReduce Jobs
• Build Inverted Indexes
• Build Title Index Basis

AJAX
OpenCalais
Client (Browser)
XUL Sidebar
WebService - Extract Entities
Glossary Inverted Index Big Files Containing
PostingsLists Title Index Basis List of (DocID,
WikiTitle) Pairs Word Index Word ?
PostingsListLocation (File Offset) Title Index
DocID ? WikiTitle