Passage Retrieval using HMMs

1
Passage Retrieval using HMMs

• HARD 2004
• University of Illinois at Urbana-Champaign
• Jing Jiang ChengXiang Zhai

2
Motivation Variable Length Passages
Nokia, the worlds biggest acquired Sega
Japanese video game maker,
its mobile N-Gage game
features of a cell phone, MP3-player
Nokia is the cell phone market leader

Nintendo Co.s now works as a videophone

which makes mobile and Internet
equipment
Nintendo has sold more than 10 million Game Boy
APE20030911.0887
APE20030922.0156
3
Motivation Variable Length Passages

• document-dependent

Nintendo Co.s now works as a videophone

which makes mobile and Internet
equipment
Nintendo has sold more than 10 million Game Boy
Nokia, the worlds biggest acquired Sega
Japanese video game maker,
its mobile N-Gage game
features of a cell phone, MP3-player
Nokia is the cell phone market leader

HARD-422
video game crash
APE20030911.0887
APE20030922.0156
4
Motivation Variable Length Passages

• query-dependent

Nintendo Co.s now works as a videophone

which makes mobile and Internet
equipment
Nintendo has sold more than 10 million Game Boy
Nokia, the worlds biggest acquired Sega
Japanese video game maker,
its mobile N-Gage game
features of a cell phone, MP3-player
Nokia is the cell phone market leader

HARD-422
video game crash
HARD-443
hand-held electronics
APE20030911.0887
APE20030922.0156
5
Research Question

• Passage length is
• document-dependent
• query-dependent

How to detect variable-length passages?
6
Previous Work on Passage Retrieval

• Structural or semantic boundary
• Passage is not query-specific.
• Fixed-length
• Passage length is not query-specific.
• Passage content may not be coherent.
• Arbitrary MultiText
• Only query words are considered.
• Heuristics are used to reduce search space.
• HMM-based
• The method is promising, but previous work didnt
  fully explore its potential.

7
HMM-Based Method
document
8
HMM-Based Method
Q hand-held electronics
relevant passage
document
9
HMM-Based Method
Q hand-held electronics
relevant passage
document
B
R

B
B

R
R
R
R
B

B
R
10
Constructing the HMM
11
Constructing the HMM
end-of-doc state
12
Constructing the HMM
end-of-doc state
0.01
0.005
smoothing achieved by transitions
0.99
13
Constructing the HMM
end-of-doc state
0.01
0.005
expanded query LM to incorporate feedback
smoothing achieved by transitions
0.99
14
Constructing the HMM
transition probabilities trained for each document
end-of-doc state
0.01
0.005
expanded query LM to incorporate feedback
smoothing achieved by transitions
0.99
15
Passage Extension
16
Retrieval Approach 1
17
Retrieval Approach 1
ranking
18
Retrieval Approach 1
passage extraction
ranking

19
Retrieval Approach 2
20
Retrieval Our Approach
21
Passage-Level Results

• Overall, baseline was the best.

22
Effectiveness of HMM method

• HMM method improved performance over
  fixed-length passages
• Less improvement if fixed-length closer to
  optimal length

23
Diagnosis Runs
KL-divergence works poorly on passages
non-optimal parameter setting
HMM improves boundaries
24
Discussions and Conclusions

• HMM method improved the performance over
  fixed-length passages
• LM (KL-divergence) method gives worse
  performance on passage ranking than on document
  ranking

25
The End

• Questions?