Index and Query Techniques

1
Index and Query Techniques

• pb
• 11/21/2009

2
Review

• What is Information Retrieval?

• A person has a goal to accomplish
• Find the nearest hotel/restaurant
• Write your research paper
• Keep informed about the cheapest Nokia mobile
  phone on the market
• Find a job
• Along the way, he needs to find information to
  accomplish the goal.
• IR engine is a tool to support searching. Its
  important to make it effective, efficient, and
  flexible.
• (User View)

3
Review

• What is Information Retrieval?

• IR is the study of
• Representation
• Storage
• Organization
• Access
• of information items
• articles, books, web pages, CDs, movies ...
• for people who are interesting in them.
• (System View)

4
(No Transcript)
5
Review

• What does IR model mean?

• ???D, Q, F, R(qi, dj)
• D ????????
• Q ?????????
• F ??????????????????????
• R(qi, dj) query qi ?document dj??relevance????
• ?????????
• ????????????????
• ????????????????????
• ????????????????

6
Review

• Some basic ideas in IR
• Bag of words captures much of the meaning
• Similarity in using vocabulary between objects
  means they are related
• Use probabilistic or statistical techniques that
  reflect semantics without actually understanding

7
What we talk about today?

• How can those approaches be made better?
• What can we do to make system work quickly?
• How do we decide whether it works well?
• What else can we do with the same approaches?

8
Outline

• Text processing (indexing)
• Text indexing is the process of deciding what
  will be used to represent a given document
• What selected call index terms
• File organization (indexes)
• File organizations or indexes are used to
  increase performance of system
• Query processing

9
Who are they?
SIGIR Gerard Salton Award This award honors
those who have made "... significant, sustained
and continuing contributions to research in
information retrieval".
W. Bruce Croft
Tefko Saracevic
Stephen Robertson
William Cooper
Gerard Salton
Karen Sparck Jones
Cyril Cleverdon
10
Bruce Croft. Information Retrieval and Computer
Science An Evolving Relationship

• Search engines have become the infrastructure for
  much of information access in our society. IR has
  provided the basic research
• IR championed the statistical approach to
  language long before other researchers.
  Statistical NLP is now mainstream
• IR focused on evaluation as a research area, and
  developed an evaluation methodology based on
  large, standardized testbeds...
• IR has always acknowledged the importance of the
  user and interaction as a part of information
  access. interfaces and learning techniques based
  on user feedback.