Natural Language Processing (NLP)

1
Natural Language Processing (NLP)

• Informally NLP computers handling ordinary
  language
• 6000-7000 languages exist. Important differences,
  more important similarities
• Applications of NLP to facilitate
• person-person communication Machine Translation
  (MT), Summarisation, ..
• person-machine communication Question-Answering,
  travel booking, car navigation, ...
• NLP - sometimes also includes speech processing.

Some slides from Kees Van Deemter
2
Language Technology
Meaning
Text
Text
Speech
Speech
3
Natural Language Understanding

• speech recognition (unless input is text)
• parsing
• word disambiguation
• determining overall meaning

4
Natural Language Generation

• Natural Language Generation
• what information to convey
• how to distribute information across sentences
• how to express information in a sentence
• determine sentence melody etc.

5
Dialogue systems

• Question-Answering, travel booking, customer
  service
• Dialogue systems perform understanding and
  generation
• they perform understanding to make sense of your
  utterances
• then they perform generation to produce a new
  utterance
• They are not very good!

6
European Association for Machine Translation, 1997
Machine Translation (MT)

• Translating texts from one natural language to
  another
• One of the very earliest pursuits in computer
  science
• MT has proved to be an elusive goal
• 1950s Much money in USA, USSR, Britain, Italy,
  France
• 1966 Any task that requires real understanding
  of natural language is too difficult for a
  computer - Bar-Hillel
• Today a number of systems are available which
  produce output which, if not perfect, is of
  sufficient quality to be useful in a number of
  specific domains.

7
Natural Language is Notoriously Ambiguous

• Squad helps dog bite victim.
• Helicopter powered by human flies.
• American pushes bottle up Germans.
• Once-sagging cloth diaper industry saved by full
  dumps.
• Portable toilet bombed police have nothing to go
  on.
• British left waffles on Falkland islands.
• Milk drinkers are turning to powder.
• Drunk gets nine months in violin case.
• Time flies like an arrow.

8
Natural Language is Notoriously Ambiguous

• (You should) time flies as you would (time) an
  arrow
• Time flies in the same way that an arrow would
  (time them)
• Time those flies that are like arrows
• Fruit flies like a banana
• each of above
• Time magazine travels straight when thrown

• Squad helps dog bite victim.
• Helicopter powered by human flies.
• American pushes bottle up Germans.
• Once-sagging cloth diaper industry saved by full
  dumps.
• Portable toilet bombed police have nothing to go
  on.
• British left waffles on Falkland islands.
• Milk drinkers are turning to powder.
• Drunk gets nine months in violin case.
• Time flies like an arrow.

9
Natural Language is Notoriously Ambiguous

• (You should) time flies as you would (time) an
  arrow
• Time flies in the same way that an arrow would
  (time them)
• Time those flies that are like arrows
• Fruit flies like a banana
• each of above
• Time magazine travels straight when thrown

• Squad helps dog bite victim.
• Helicopter powered by human flies.
• American pushes bottle up Germans.
• Once-sagging cloth diaper industry saved by full
  dumps.
• Portable toilet bombed police have nothing to go
  on.
• British left waffles on Falkland islands.
• Milk drinkers are turning to powder.
• Drunk gets nine months in violin case.
• Time flies like an arrow.

• Surprise for early researchers
• Almost every utterance is highly ambiguous
• Alternative interpretations often not apparent to
  native speaker

10
Which Nouns do Adjectives Apply to?

• pretty little girls' school
• Does the school look little?
• Do the girls look little?
• Do the girls look pretty?
• Does the school look pretty?

11
Natural Language is Notoriously Ambiguous

• mature students and staff
• (mature students) and staff
• rotten apples and oranges
• mature (students and staff)
• Two different syntactic analyses
• Two different MEANING REPRESENTATIONS
• Everyone can win a gold medal 1
  medalEveryone can take a chocolate 8
  chocolates
• This is not evidently a matter of syntax
• Two different MEANING REPRESENTATIONS
• Many nouns have many meanings Trunk, bank,
  battery

Syntactic Ambiguity
Semantic Ambiguity
12
Metonymy

• (one thing stands for another)
• Ive read Shakespeare
• Chrysler announced record profits

Metaphor

• More is up
• Prices have risen, climbed, skyrocketed
• Temperature has dipped, fallen
• Confidence has plummeted
• Popularity has jumped, soared
• Ive tried killing the process but it wont die.
  Its parent keeps it alive.

13
Anaphora

• Anaphora pronouns refer back to things already
  introduced
• We gave the monkeys the bananas because they were
  hungry.
• We gave the monkeys the bananas because they were
  over-ripe.
• After Mary proposed to John, they found a
  preacher and got married.
• For the honeymoon, they went to Hawaii
• Mary saw a ring through the window and asked John
  for it.
• Mary threw a rock at the window and broke it.

14
Anaphora

• Dana dropped the cup on the plate. It broke.
• Dana was quite fond of a special blue cup. The
  cup had been a present from a close friend.
  Unfortunately, one day while setting a place at
  the table, Dana dropped the cup on the plate. It
  broke.
• Discourse has structure above the level of a
  sentence

15
Discourse Understanding

• A funny thing happened yesterday
• Introduces new focus space and Evaluates it
• John went to a fancy restaurant
• Enables 3.
• He ordered the duck
• Causes 4.
• The bill came to 50
• 2-4 serve as Ground for the rest of the story
  implies John ate the duck
• John got a shock when he realized he had no money
• He had left his wallet at home
• Explains 5. 5-6 enable 7
• The waiter said it was all right to pay later
• 5-7 cause 8
• He was very embarrassed by his forgetfulness

16
Discourse Understanding (is hard!)

• Alice Lets go to the cinema.
• Bob I have an exam tomorrow.
• Kid Mommy, Im hungry.
• Mother Have you finished all your homework?

17
Tricks for Language Understanding

• Exploit many constraints
• meanings of individual words (lexicon)
• grammatical constraints (case roles and verb
  categories)
• Discourse coherence constraints
• Language model
• Speaker model
• World model
• Pretty good models available for all except the
  world model

18
Machine Translation Difficulties

• When languages similar, one can hope that
  word-by-word translation preserves ambiguity
• When languages are very different, this is often
  not the case
• The example open shows that the problems arises
  even in English/German
• on the door of a store (German offen)
• on a banner in front of the store (German neu
  eroeffnet)
• Open shop, market, question, position loose
  ice?
• Words dont map one to one
• It is necessary to model the situation in your
  mind (disambiguate), and then describe it in the
  other language
• So why not make the computer model it?
• Commonsense knowledge problem