NLP 1 An Introduction to Pragmatics in NLP

1
NLP 1An Introduction to Pragmatics in NLP

• GSLT,
• Göteborg, March 2006

Barbara Gawronska, Högskolan i Skövde
2
Reading list

• Jurafsky Martin, part IV
• Mitkov, R. 2000. "Towards a more consistent and
  comprehensive evaluation of anaphora resolution
  algorithms and systems." Proceedings of the
  Discourse Anaphora and Anaphora Resolution
  Colloquium (DAARC-2000), 96-107, Lancaster, UK
  (pdf)
• http//clg.wlv.ac.uk/papers/Lancaster2000.PDF
• Mitkov, R. and Barbu, C. 2002. "Using corpora to
  improve pronoun resolution." Languages in
  context, 4(1). (pdf )
• http//clg.wlv.ac.uk/papers/mitkov02.pdf
• Hutchins, J. 2003. "Has Machine Translation
  improved?" An expanded version PDF, 288KB of a
  paper presented at MT Summit IX Proceedings of
  the Ninth Machine Translation Summit, New
  Orleans, USA, September 23-27, 2003, 181-188.
  East Stroudsburg, PA AMTA. PDF, 191KB
• http//ourworld.compuserve.com/homepages/WJHutchi
  ns/HasMTimproved-exp.pdf

3
Outline

• The notion Pragmatics
• Pragmatics vs. Semantics
• Pragmatics and NLP Discourse Processing
• Anaphora resolution
• NL Generation
• Information Extraction and Text Summarization
• Machine Translation
• CALL
• Future directions

4
Pragmatics vs. Semantics (1)

• Austin 1962 Pragmatics the study of "how to
  do things with words
• Leech Weisser 2003 Pragmatics the branch
  of linguistics which seeks to explain the meaning
  of linguistics messages in terms of their context
  of use ,
• while
• Semantics investigates meaning as part of the
  language system irrespective of wider context

5
Pragmatics vs. Semantics (2)

• Classical work on pragmatics (Austin 1962, Searle
  1969, Grice 1975) problems as
• Discourse referents what entities does a given
  message refer to?
• What background knowledge is needed to understand
  a given message?
• How do the beliefs of speaker and hearer interact
  in the interpretation of a message?
• What is a relevant answer to a given question?

6
Pragmatics vs. Semantics (3)

• This implies that the study object of semantics
  and pragmatics comprises interactions between
  entities on different levels of the linguistic
  structure. Pragmatics takes even interactions
  between the linguistic and the non-linguistic
  reality into account.
• E.g. identification of discourse referents
  (entities referred to) in spoken language
  requires an interplay between phonetic/phonologica
  l, morphological, syntactic, and semantic factors
  as well as the use of extralinguistic knowledge.

7
Problems with reference in spoken language
processing an example (from August, KTH)

• User and system have different background
  knowledge
• User Finns det en bra restaurang i närheten? (Is
  there a good restaurant nearby?)
• System Du måste ange gatan (You have to name the
  street)
• The system gives an answer that is true, but not
  relevant
• User Var är vi? (Where are we?)
• System Vi är ju här. (We are here.)

8
Pragmatics in NLP

• Discourse processing for
• Dialogue systems
• Natural Language Generation
• Reading Comprehension (e.g. in Q/A systems, in
  summarization systems)
• Machine Translation
• Multifunctional NLP systems
• Computer Assisted Language Learning (CALL)

9
Discourse processing (1)

• Discourse level beyond the sentence level
• Traditional distinctions
• Spoken/written discourse
• Monologue/dialogue
• New discourse types related to new ways of
  communicating SMS, chatting, e-mail...

10
Discourse processing (2)

• The main aspects
• Anaphora resolution
• Cohesion and coherence
• Discourse structure

11
Anaphora resolution (1)

• Theoretical work Karttunen 1976, Kamp 1979,
  1981, Grosz and Sidner 1986, Hobbs 1978, 1982,
  Dagan Itai 1990, Lappin Leass 1944, Mitkov
  and Barbu 2000, 2002...)
• Basic notions
• Anaphora
• Antecedent
• Discourse referent
• Coreference chain

12
Anaphora resolution (2)

• Sources of knowledge
• Syntactic and morphosyntactic constraints
  (boundedness, gender, number, grammatical roles)
• Mary met John. He/She/They decided...
• She helped her/herself
• Semantic features, selectional restrictions
• I bought a bottle of wine, sat down on a stone,
  and drank it

13
Anaphora resolution (3)

• Ontological knowledge, domain knowledge
• in interaction with semantic and grammatical
  constraints
• My friends have a greyhound. They are really huge
  beasts
• They prohibited them from demonstrating because
  they feared violence
• They prohibited them from demonstrating because
  they advocated violence
• (Winograd 197233)

14
Algorithms for anaphora resolution (1)

• Based on parse trees (naïve)
• Left-to right, breadth-first search, starting
  with the sentence containing the pronoun
• Based on syntactic roles The centering algorithm
  (Grosz et al 1995, Lappin and Leass 1994)
• Based on lexical and collocational indicators
  Mitkovs knowledge poor approach (Mitkov 1998)
• Based on so-kalled pragmatic functions the
  Mental Space model (Fauconnier 1985,1998)

15
Algorithms for anaphora resolution (2) The
centering algorithm

• Backward lookning center (CB) - the entity
  currently in focus
• Forward looking centers (CF) - an ordered list of
  entities
• Subject gt Predicative NP gt direct object gtoblique
  gt PP
• Preferred center (CP) - the highest ranked
  forward looking element
• A ranked set of transitions
• Continue CB CP CB of the previous utterance
• Retain CB\ CP CB CB of the previous
  utterance
• Smooth shift CB CP CB \ CB of the previous
  utterance
• Rough-shift CB \ CP CB \ CB of the previous
  utterance

16
Algorithms for anaphora resolution (3) The
knowledge-poor approach (Mitkov 1998, 2000)

• Input a text processed by a POS-tagger and an NP
  extractor
• Locate all NPs which precede the anaphor within a
  distance of 3 sentences
• Check number and gender agreement, filter out
  NPs that do not fulfil agreement conditions
• Apply boosting and impeding indicators to the
  remaining NPs

17
Algorithms for anaphora resolution (4) The
knowledge-poor approach (Mitkov 1998)

• Boosting indicators (some examples)
• First NP in a sentence
• Lexical Iteration (NPs repeated twice or more in
  the papagraph before the pronoun)
• Section Heading Preference
• Collocation Pattern Preference (Press the key
  down and turn the volume up. Press it again)
• Term preference (terms characteristic for the
  genre
• For an interesting approach to collocations
  (collostructional analysis), see Gries, Hampe,
  Schönefeld 2005

18
Algorithms for anaphora resolution (4) The
knowledge-poor approach (Mitkov 1998)

• Impeding indicators (some examples)
• Indefiniteness
• Complement of a preposition
• Referential distance
• Evaluation
• Success rate Number of sucessfully resolved
  anaphors/Number of all anaphors
• (Different variants paying atention to trivial
  and non-trivial anaphors)

19
The Theory of Mental Spaces (Fauconnier1985,
Fauconnier and Sweetser 1996 focus on beliefs
and attitudes)
20
The Theory of Mental Spaces (2) (Fauconnier 1985,
Sweetser Fauconnier 1996, Sanders Redeker
1996)
21
Natural Language Generation (1)

• Discourse planning
• Templates partially pre-defined text frames
• Algorithms based on discourse theories (e.g.
  Rhetorical Strucure Theory (RST) Mann
  Thompson
• Sentence planning (sentence aggregation,
  generation of referring expressions, lexical
  selection)
• Surface realization (word order and agreement
  control, graphic realization)

22
Natural Language Generation (2)

• Main issues cohesion and coherence
• Cohesion establishing anaphoric connections (the
  reverse of anaphora resolution)
• Coherence nucleus-satellite relations (RST)
• e.g. result, cause, elaboration, contrast,
  parallel...

23
Natural Language Generation (3)

• Some examples of coherence relations
• John bought a dog. His wife went furious (result)
• John hid Bills car keys. Bill had drunk too much
  (explanation)
• John bought a Mercedes. Bill leased a BMW
  (parallel)
• An insufficiently cohesive/coherent text
• I saw a little dog. Dogs like bones. Bones are
  white. White is my favourite colour...

24
Sentence aggregation an example (visit
http//www.iccs.informatics.ed.ac.uk/jbos/anna/
for more demos)

• TEXT WITHOUT AGGREGATIONIcelandIceland is
  situated in the north Atlantic. Iceland has a
  coastline of 5 000 km. Iceland has an area of 103
  000 sq km. The highest point is 2 119 m. The
  highest point is Hvannadalshnukur. Iceland has a
  temperate climate. Iceland has mild, windy
  winters and cool, damp summers. Iceland has 280
  000 inhabitants. The population density is 3
  people/sq km. The life expectancy is 79 years.
  The fertility rate is 2 children. The official
  language is Icelandic. Icelandic is a germanic
  language. Iceland is a constitutional republic.
  The capital is Reykjavik. Reykjavik has 107 000
  inhabitants. The national holiday is June 17.

25

• TEXT WITH AGGREGATIONIcelandIceland is
  situated in the north Atlantic.
• Iceland has a coastline of 5 000 km and an area
  of 103 000 sq km. The highest point,
  Hvannadalshnukur, is 2 119 m. Iceland has a
  temperate climate with mild, windy winters and
  cool, damp summers. Iceland has 280 000
  inhabitants and the population density is 3
  people/sq km. The life expectancy is 79 years and
  the fertility rate is 2 children.
• The official language is Icelandic, a germanic
  language. Iceland is a constitutional republic.
  The capital, Reykjavik, has 107 000 inhabitants.
  The national holiday is June 17.

26
Document and Text Retrieval vs. Information
Extraction
27
Text Summarization Types of summaries

• With respect to content
• Indicative provide an idea what the text is
  about
• Informative shortened versions of the text
• With respect to the way of creating
• Extracts reused portions of the text (text
  retrieval)
• Abstracts re-generated text reflecting the
  important content (information extraction and
  text generation)
• Compressed texts (Knight Marcu 2000)
  compressing parse trees in order to get a shorter
  text
• Dialogue summarization selecting successful
  dialog transactions
• Non-verbal summaries (e.g. graphical
  representations)

28
Abstract creation

• Template-based summarization (templates, sketchy
  frames, extraction patterns frames containing
  slots with constraints and variables relay on
  prior domain knowledge) some examples
• DeJong 1982 FRUMP (Fast Reading Understanding
  and Memory Program)
• Rilloff 1996 CIRCUS (terrorism domain)
• McKeown and Radev 1999 SUMMONS (SUMMarizing
  Online NewS articles)
• Plot units (selecting causal relations Lehnert
  1981)

29
Template based summarization- general architecture
30
The outline of the summarization process in
Newspeak (Gawronska et al. 2004)
31
Some principles for selection of claims to be
rendered

• 1) Informatives
•  
• Neutral, the sender is not marked for high
  status officials said, the news agency reported,
  reportedlyA claim p introduced by a neutral
  informative is rendered in the summary the
  source is omitted if there are no denials or
  confirmations of p in the text and if the source
  is not marked for high status, like President
• Neutral, the sender marked for high status, and
  declarations the President saidthe government
  condemnedThe source is rendered if it is marked
  for high status
• Affirmative confirmations of explicit claims
  Israeli sources confirmed thatConfirmations of
  previous explicit claims are omitted in the
  summary
• Affirmative confirmations of claims that are
  not explicitly mentionedBoth the information
  source and the claim, including the type of the
  speech act phrase, are rendered in the summary,
  if the speech act is a confirmation of a claim
  not present in the news report

32
Some principles for selection of claims to be
rendered

• 1) Informatives
•  
• Negative, or neutral followed by denied claims
  The president denied, The Israeli source said
  that it is not trueBoth the initial claim and
  its denial are rendered in the summary together
  with the information about the senders
•  

33
Some principles for selection of claims to be
rendered (2)

• 2) Utterance refusal, negated speech act phrases,
  hypotheses, commissives, interpretations The
  Israeli sources neither denied or confirmed, the
  minister did not say, if, the defense secretary
  declined to say, the government had no immediate
  comments
• Utterance refusals or negated speech act phrases
  related to an explicit claim are omitted
• If a source refuses to confirm/deny a claim that
  has not been explicitly mentioned in the previous
  part of the text, the whole speech act is
  rendered, inclusive the type of the speech act
• Hypotheses and commissives are rendered together
  with their sources and marked for unsure
  epistemic status

34
Some principles for selection of claims to be
rendered (3)
3) Epistemic spaces e. g. no one knows if the
device was planted deliberately or if it was
leftover from New Years Eve If two claims
would exclude each other in the same mental
space, and if no source in the text takes
responsibility for any of these claims, both
claims are to be rendered as hypotheses
35
Some challenges
RAMALLAH, West Bank -- Palestinian leader Yasser
Arafat said Thursday that elections as part of a
reform of the Palestinian Authority will be held
this winter, whether or not Israeli forces
withdraw from the Palestinian territories. That
represented a change of course from Arafat, who
said last week that no elections would be held
until the Israelis pulled back. Shortly after
Arafat's announcement, a committee he had
appointed to set up elections resigned, according
to Israel Radio, because Arafat would not agree
to a specific date for the elections. Other
Palestinian leaders said the resignations were a
procedural matter. Arafat also condemned
Wednesday's suicide bombing in the Israeli town
of Rishon Letzion . Two Israelis were killed and
at least 37 others wounded when the bomber
detonated explosives in the center of a crowded
pedestrian district. The terror attack marked the
second time in two weeks a suicide bombing
directed at civilians has rocked Rishon Letzion,
a town about 15 miles southeast of Tel Aviv. On
May 8, a suicide attack at a pool hall killed 15
people and wounded dozens of others. "Suddenly
there was an explosion," 16-year-old Shmuel
Voller told The Associated Press on
Wednesday. The bombing occurred on Rothschild
Street in the heart of the town around 915 p.m.
(215 p.m. ET).
36
An example of non-verbal summaries extracting
pathway maps similar to those in Kyoto
Encyclopaedia of Genes and Genomes (KEGG) from
biomedical literature
37
(No Transcript)
38
A sample comprimed tree expressing a biological
process
39
Machine Translation combining NL Understanding
and NL Generation (1)

• 1940... the first attempts direkt word-to-word
  translation some morphosyntactic processing
  (e.g. case recognition in Russian)
• 1970...-syntax-based approaches interlingua and
  transfer
• 1990 Brown et al. foundation of stochastic MT
  (computing translation probabilities on the basis
  of parallel corpora)

40
Machine Translation (2)

• Knowledge Based Machine Translation KBMT
  Nirenburg et al., Hobbs, Wilks mm
• - knowledge stored in lexicons, onomasticons,
  and ontologies
• rule-based parsing and semantico-pragmatic
  analysis aimed at conceptuel representations
• Example Based MT EBMT - translation in analogy
  with best match in the corpus of previously
  translated texts
• Hybrid systems (e.g. Verbmobil Wahlster et al
  2000)

41
The multi engine architecture of the MT system
Verbmobil (a simplified version of Figure 11, p.
17 in 33)
42
MT evaluation some useful links

• Hutchins, John (2000) The IAMT Certification
  initiative and defining translation system
  categories. (Presented at EAMT Workshop,
  Ljubljana, May 2000)
• http//ourworld.compuserve.com/homepages/WJHutchi
  ns/IAMTcert.htm http//ourworld.compuserve.com/hom
  epages/WJHutchins/Compendium-4.pdf
• http//www.issco.unige.ch/projects/isle/femti/

43
MT, current trends

• Towards hybrid systems integration of rule-based
  approaches and stochastic approaches
• Spoken language translation
• Sign language translation
• Combined MT and Intormation Extraction
• Computer aided translation

44
Computer Assisted Language Learning (CALL) focus
on communicative competence
45
Conclusions Future?

• Pragmatics still a challenge for NLP
• Research needed on
• General vs. domain-specific resources and
  algorithms
• User models (beliefs, attitudes, etc.)
• The interplay between prosody, syntax, and
  semantics
• New means of communication, new types of
  discourse
• Synergy between rule-based and stochastic
  approaches