NLP 1 An Introduction to Pragmatics in NLP

1
NLP 1An Introduction to Pragmatics in NLP

• GSLT,
• Göteborg, March 2005

Barbara Gawronska, Högskolan i Skövde
2
Reading list

• Jurafsky Martin, part IV
• Mitkov, R. 2000. "Towards a more consistent and
  comprehensive evaluation of anaphora resolution
  algorithms and systems." Proceedings of the
  Discourse Anaphora and Anaphora Resolution
  Colloquium (DAARC-2000), 96-107, Lancaster, UK
  (pdf)
• http//clg.wlv.ac.uk/papers/Lancaster2000.PDF
• Mitkov, R. and Barbu, C. 2002. "Using corpora to
  improve pronoun resolution." Languages in
  context, 4(1). (pdf )
• http//clg.wlv.ac.uk/papers/mitkov02.pdf
• Hutchins, J. 2003. "Has Machine Translation
  improved?" An expanded version PDF, 288KB of a
  paper presented at MT Summit IX Proceedings of
  the Ninth Machine Translation Summit, New
  Orleans, USA, September 23-27, 2003, 181-188.
  East Stroudsburg, PA AMTA. PDF, 191KB
• http//ourworld.compuserve.com/homepages/WJHutchi
  ns/HasMTimproved-exp.pdf

3
Outline

• The notion Pragmatics
• Pragmatics vs. Semantics
• Pragmatics and NLP Discourse Processing
• Anaphora resolution
• NL Generation
• Text Summarization
• Machine Translation
• CALL
• Future directions

4
Pragmatics vs. Semantics (1)

• Austin 1962 Pragmatics the study of "how to
  do things with words
• Leech Weisser 2003 Pragmatics the branch
  of linguistics which seeks to explain the meaning
  of linguistics messages in terms of their context
  of use ,
• while
• Semantics investigates meaning as part of the
  language system irrespective of wider context

5
Pragmatics vs. Semantics (2)

• Classical work on pragmatics (Austin 1962, Searle
  1969, Grice 1975) problems as
• Discourse referents what entities does a given
  message refer to?
• What background knowledge is needed to understand
  a given message?
• How do the beliefs of speaker and hearer interact
  in the interpretation of a message?
• What is a relevant answer to a given question?

6
Pragmatics vs. Semantics (3)

• This implies that the study object of pragmatics
  comprises interactions between entities on
  different levels of the linguistic structure as
  well as interactions between the linguistic and
  the non-linguistic reality.
• E.g. identification of discourse referents in
  spoken language requires an interplay between
  phonetic/phonological, morphological, syntactic,
  and semantic factors as well as the use of
  extralinguistic knowledge.

7
Problems with reference in spoken language
processing an example (from August, KTH)

• User and system have different background
  knowledge
• User Finns det en bra restaurang i närheten? (Is
  there a good restaurant nearby?)
• System Du måste ange gatan (You have to name the
  street)
• The system gives an answer that is true, but not
  relevant
• User Var är vi? (Where are we?)
• System Vi är ju här. (We are here.)

8
Pragmatics in NLP

• Discourse processing for
• Dialogue systems
• Natural Language Generation
• Reading Comprehension (e.g. in Q/A systems, in
  summarization systems)
• Machine Translation
• Multifunctional NLP systems
• Computer Assisted Language Learning (CALL)

9
Discourse processing (1)

• Discourse level beyond the sentence level
• Traditional distinctions
• Spoken/written discourse
• Monologue/dialogue
• New discourse types related to new ways of
  communicating SMS, chatting, e-mail...

10
Discourse processing (2)

• The main aspects
• Anaphora resolution
• Cohesion and coherence
• Discourse structure

11
Anaphora resolution (1)

• Theoretical work Karttunen 1976, Kamp 1979,
  1981, Grosz and Sidner 1986, Hobbs 1978, 1982,
  Dagan Itai 1990, Lappin Leass 1944, Mitkov
  and Barbu 2000, 2002...)
• Basic notions
• Anaphora
• Antecedent
• Discourse referent
• Coreference chain

12
Anaphora resolution (2)

• Sources of knowledge
• Syntactic and morphosyntactic constraints
  (boundedness, gender, number, grammatical roles)
• Mary met John. He/She/They decided...
• She helped her/herself
• Semantic features, selectional restrictions
• I bought a bottle of wine, sat down on a stone,
  and drank it

13
Anaphora resolution (3)

• Ontological knowledge, domain knowledge
• in interaction with semantic and grammatical
  constraints
• My friends have a greyhound. They are really huge
  beasts
• They prohibited them from demonstrating because
  they feared violence
• They prohibited them from demonstrating because
  they advocated violence
• (Winograd 197233)

14
Algorithms and models for anaphora resolution (1)

• Based on parse trees (naïve)
• Left-to right, breadth-first search, starting
  with the sentence containing the pronoun
• Based on syntactic roles The centering algorithm
  (Grosz et al 1995, Lappin and Leass 1994)
• Based on lexical and collocational indicators
  Mitkovs knowledge poor approach (Mitkov 1998)
• Based on so-kalled pragmatic functions the
  Mental Space model (Fauconnier 1985,1998)

15
Algorithms for anaphora resolution (2) The
centering algorithm

• Backward lookning center (CB) - the entity
  currently in focus
• Forward looking centers (CF) - an ordered list of
  entities
• Subject gt Predicative NP gt direct object gtoblique
  gt PP
• Preferred center (CP) - the highest ranked
  forward looking element
• A ranked set of transitions
• Continue CB CP CB of the previous utterance
• Retain CB\ CP CB CB of the previous
  utterance
• Smooth shift CB CP CB \ CB of the previous
  utterance
• Rough-shift CB \ CP CB \ CB of the previous
  utterance
• For details and examples, see Jurafsky Martin
  pp. 691-696

16
Algorithms for anaphora resolution (3) The
knowledge-poor approach (Mitkov 1998, 2000)

• Input a text processed by a POS-tagger and an NP
  extractor
• Locate all NPs which precede the anaphor within a
  distance of 3 sentences
• Check number and gender agreement, filter out
  NPs that do not fulfil agreement conditions
• Apply boosting and impeding indicators to the
  remaining NPs

17
Algorithms for anaphora resolution (4) The
knowledge-poor approach (Mitkov 1998)

• Boosting indicators (some examples)
• First NP in a sentence
• Lexical Iteration (NPs repeated twice or more in
  the papagraph before the pronoun)
• Section Heading Preference
• Collocation Pattern Preference (Press the key
  down and turn the volume up. Press it again)
• Term preference (terms characteristic for the
  genre)

18
Algorithms for anaphora resolution (4) The
knowledge-poor approach (Mitkov 1998)

• Impeding indicators (some examples)
• Indefiniteness
• Complement of a preposition
• Referential distance
• Evaluation
• Success rate Number of sucessfully resolved
  anaphors/Number of all anaphors
• (Different variants paying atention to trivial
  and non-trivial anaphors)

19
The Theory of Mental Spaces (Fauconnier1985,
Fauconnier and Sweetser 1996 focus on beliefs
and attitudes)
20
The Theory of Mental Spaces (2) (Fauconnier 1985,
Sweetser Fauconnier 1996, Sanders Redeker
1996)
21
Natural Language Generation (1)

• Discourse planning
• Templates partially pre-defined text frames
• Algorithms based on discourse theories (e.g.
  Rhetorical Strucure Theory (RST) Mann
  Thompson
• Sentence planning (sentence aggregation,
  generation of referring expressions, lexical
  selection)
• Surface realization (word order and agreement
  control, graphic realization)

22
Natural Language Generation (2)

• Main issues cohesion and coherence
• Cohesion establishing anaphoric connections (the
  reverse of anaphora resolution)
• Coherence nucleus-satellite relations (RST)
• e.g. result, cause, elaboration, contrast,
  parallel...

23
Natural Language Generation (3)

• Some examples of coherence relations
• John bought a dog. His wife went furious (result)
• John hid Bills car keys. Bill had drunk too much
  (explanation)
• John bought a Mercedes. Bill leased a BMW
  (parallel)
• An insufficiently cohesive/coherent text
• I saw a little dog. Dogs like bones. Bones are
  white. White is my favourite colour...

24
Sentence aggregation an example (visit
http//www.iccs.informatics.ed.ac.uk/jbos/anna/
for more demos)

• TEXT WITHOUT AGGREGATIONIcelandIceland is
  situated in the north Atlantic. Iceland has a
  coastline of 5 000 km. Iceland has an area of 103
  000 sq km. The highest point is 2 119 m. The
  highest point is Hvannadalshnukur. Iceland has a
  temperate climate. Iceland has mild, windy
  winters and cool, damp summers. Iceland has 280
  000 inhabitants. The population density is 3
  people/sq km. The life expectancy is 79 years.
  The fertility rate is 2 children. The official
  language is Icelandic. Icelandic is a germanic
  language. Iceland is a constitutional republic.
  The capital is Reykjavik. Reykjavik has 107 000
  inhabitants. The national holiday is June 17.

25

• TEXT WITH AGGREGATIONIcelandIceland is
  situated in the north Atlantic.
• Iceland has a coastline of 5 000 km and an area
  of 103 000 sq km. The highest point,
  Hvannadalshnukur, is 2 119 m. Iceland has a
  temperate climate with mild, windy winters and
  cool, damp summers. Iceland has 280 000
  inhabitants and the population density is 3
  people/sq km. The life expectancy is 79 years and
  the fertility rate is 2 children.
• The official language is Icelandic, a germanic
  language. Iceland is a constitutional republic.
  The capital, Reykjavik, has 107 000 inhabitants.
  The national holiday is June 17.

26
Text Summarization Types of summaries

• With respect to content
• Indicative provide an idea what the text is
  about
• Informative shortened versions of the text
• With respect to the way of creating
• Extracts reused portions of the text
• Abstracts re-generated text reflecting the
  important content
• Compressed texts (Knight Marcu 2000)
  compressing syntactic parse trees in order to get
  a shorter text
• Dialogue summarization selecting successful
  dialog transactions

27
Abstract creation

• Template-based summarization (templates, sketchy
  frames, extraction patterns frames containing
  slots with constraints and variables relay on
  prior domain knowledge) some examples
• DeJong 1982 FRUMP (Fast Reading Understanding
  and Memory Program)
• Rilloff 1996 CIRCUS (terrorism domain)
• McKeown and Radev 1999 SUMMONS (SUMMarizing
  Online NewS articles)
• Plot units (selecting causal relations Lehnert
  1981)

28
Template based summarization- general architecture
29
Machine Translation combining NL Understanding
and NL Generation (1)

• 1940... the first attempts direkt word-to-word
  translation some morphosyntactic processing
  (e.g. case recognition in Russian)
• 1970...-syntax-based approaches interlingua and
  transfer
• 1990 Brown et al. foundation of stochastic MT
  (computing translation probabilities on the basis
  of parallel corpora)

30
Machine Translation (2)

• Knowledge Based Machine Translation KBMT
  Nirenburg et al., Hobbs, Wilks mm
• - knowledge stored in lexicons, onomasticons,
  and ontologies
• rule-based parsing and semantico-pragmatic
  analysis aimed at conceptuel representations
• Example Based MT EBMT - translation in analogy
  with best match in the corpus of previously
  translated texts
• Hybrid systems (e.g. Verbmobil Wahlster et al
  2000)

31
The multi engine architecture of the MT system
Verbmobil (a simplified version of Figure 11, p.
17 in 33)
32
MT evaluation some useful links

• Hutchins, John (2000) The IAMT Certification
  initiative and defining translation system
  categories. (Presented at EAMT Workshop,
  Ljubljana, May 2000)
• http//ourworld.compuserve.com/homepages/WJHutchi
  ns/IAMTcert.htm http//ourworld.compuserve.com/hom
  epages/WJHutchins/Compendium-4.pdf
• http//www.issco.unige.ch/projects/isle/femti/

33
MT, current trends

• Towards hybrid systems integration of rule-based
  approaches and stochastic approaches
• Spoken language translation
• Sign language translation
• Combined MT and Intormation Extraction
• Computer aided translation

34
Computer Assisted Language Learning (CALL) focus
on communicative competence
35
Conclusions Future?

• Pragmatics still a challenge for NLP
• Research needed on
• General vs. domain-specific resources and
  algorithms
• User models (beliefs, attitudes, etc.)
• The interplay between prosody, syntax, and
  semantics
• New means of communication, new types of
  discourse
• Synergy between rule-based and stochastic
  approaches