Semantic Inference at th

1
Textual Inference A Textual Entailment Perspective
Ido Dagan Bar-Ilan University, Israel
2
Lets look at state-of-the-art common practice
3
NLP Applications need morphology

• So we use an available morphological engine!
• Namely a morphological analyzer
• Input word (in context)
• Output possible morphological analyses
  (scored/ranked)
• But there are lots of morphological phenomena to
  model
• Yes, but they are all embedded within the
  morphological analyzer

4
NLP Applications need syntax

• So we use an available syntactic engine!
• Namely a parser
• Input sentence
• Output possible parse trees (scored/ranked)
• But there are lots of syntactic phenomena to
  model
• Yes, but they are all embedded within the
  parser
• ? Frameworks which embed modeling under a
  coherent task

5
NLP Applications need semantics

• So. what do we do?
• Use NER, WordNet, SRL, WSD, statistical
  similarities, syntactic matching, detect
  negations, etc, etc, etc.
• Assemble and implement bits and pieces
• ? Scattered redundant application-dependent
  research
• ? Focus critical mass lacking, s l o w
  progress
• Can we have a generic semantic inference
  framework?
• Semantic interpretation into logic? But hardly
  adopted (why?), so I put it aside till later

6
What is reasoning/inference ?

• From dictionary.com
• reasoningthe process of forming conclusions, jud
  gments, or inferences from facts or premises.
• inferringto derive by reasoning conclude or jud
  ge from premises or evidence.  ?
  Inferring new from the old (given)

7
Textual Inference

• Inferring new textual expressions from given ones
• Captures two types of inference
• Inferences about the extra-linguistic world
• it rained yesterday gt it was wet yesterday
• Inferences about language variability
• I bought a watch gt I purchased a watch
• No definite boundary between the two

8
The Textual Entailment Task
9
Applied Textual Entailment

• A directional relation between two text
  fragments Text (t) and Hypothesis (h)

t entails h (t?h) if humans reading t will infer that h is most likely true

• Operational (applied) definition
• Human gold standard
• Entailment judgment matches applications
  judgments
• Assuming common background knowledge
• Language world knowledge

10
Textual Entailment for Applications
QA Question What affects blood pressure? Salt
causes an increase in blood pressure
IRQuery symptoms of IBS IBS is characterized
by vomiting
IBM Coremetrics
Google reMail
Yahoo Overture
11
Entailment for Multi-document Summarization
Hypothesis
Obama gave a speech last night in the Israeli
lobby conference
In his speech at the American Israel Public
Affairs Committee yesterday, the president
challenged
Barack Obamas AIPAC address yesterday ...
Texts
12
Appeal of textual entailment definition

• Main motivation models applications inferences
• Became prominent view on textual inference
• RTE 1-7 1950 hits in Google Scholar
• Much more precise than
• paraphrase
• May be viewed as bi-directional entailment /
  equivalence
• May be extended to partial highly-covering
  entailment?
• similarity
• Very vague (non-scientific?) notion
• Additional textual inference types may be defined
• But they should be defined, reasonably precisely

13
Evaluation PASCAL RTE Challenges
TEXT HYPOTHESIS TASK ENTAIL-MENT
1 Regan attended a ceremony in Washington to commemorate the landings in Normandy. Washington is located in Normandy. IE False
2 Google files for its long awaited IPO. Google goes public. IR True
3 a shootout at the Guadalajara airport in May, 1993, that killed Cardinal Juan Jesus Posadas Ocampo and six others. Cardinal Juan Jesus Posadas Ocampo died in 1993. QA True
4 The SPD got just 21.5 of the vote in the European Parliament elections, while the conservative opposition parties polled 44.5. The SPD is defeated by the opposition parties. IE True

• Created utilizing (or simulating) reductions
  from real systems output

14
The RTE Challenges

• Annually, since 2005
• Very successful challenges, world wide
• Dozens of participating groups so far (20 each
  year)
• Hundreds of downloads
• Since RTE-4 (2008) under NIST
• New Text Analysis Conference
• Current trend
• Derive RTE datasets directly from other
  applications datasets (particularly within TAC)
• Reflect RTE potential usage by these applications
• Example ideas update summarization task, KBP
  slot filling

15
Initial use of RTE systems in applications

• QA
• Harabagiu Hickl, ACL-06
• Answer Validation Exercise (AVE) at CLEF
• QallMe (FBK-irst, Magnini et al.)
• Relation extraction
• Romano et al., EACL-06
• Educational applications
• Nielsen et al., ACL-08 education workshop
• Summarization (question-directed, DUC)
• Harabagiu et al. 2007, Information Processing and
  Management
• MT evaluation and paraphrasing for MT (two
  ACL-2009 papers)