Artificial Intelligence Communication by natural language

1
Artificial IntelligenceCommunication by natural
language

• Fall 2008
• professor Luigi Ceccaroni

2
Communication

• Communication is the intentional exchange of
  information
• brought about by the production and perception of
  signs drawn from a shared system of conventional
  signs.
• What sets humans apart from other animals and
  machines is the complex system of structured
  messages known as natural language.
• It enables us to communicate most of what we know
  about the world.

2
3
Natural language processing

• In contrast with formal languages, natural
  languages, such as Spanish, French and English,
  have no strict definition.
• They are used by a community of speakers.
• Natural language processing (NLP) treats natural
  languages as if they were formal languages
• to build computational systems able to understand
  and generate human language in all its forms.

3
4
Understanding speech acts

• The action of producing language is called speech
  act.
• The problem of understanding speech acts is much
  like other understanding problems
• such as understanding images or diagnosing
  illnesses.
• We are given a set of ambiguous inputs,
• from them we have to work backwards to decide
  what state of the world could have created these
  inputs.

4
5
Fundamentals of language

• A formal language is defined as a (possibly
  infinite) set of strings.
• Each string is a concatenation of terminal
  symbols, sometimes called words.
• Formal languages such as first-order logic and
  Java have strict mathematical definitions.
• A grammar is a finite set of rules that specifies
  a language.

5
6
Fundamentals of language

• Formal languages always have an official grammar,
  specified in some document.
• Natural languages have no official grammar.
• Linguists strive to discover properties of the
  language and then to codify their discoveries in
  a grammar.
• To date, no linguist has succeeded completely.

6
7
Fundamentals of language

• Linguists attempt to define a language as it is.
• Prescriptive grammarians try to dictate how a
  language should be.
• They create rules which are sometimes printed in
  style guides, but have little relevance to actual
  language usage.

7
8
Fundamentals of language

• Both formal and natural languages associate a
  meaning or semantics to each valid string.
• In natural languages, it is also important to
  understand the pragmatics of a string
• the actual meaning of the string as it is spoken
  in a given situation
• There are very different ways to say please.
• The meaning is not just in the words themselves,
  but in the interpretation of the words in situ.

8
9
Fundamentals of language

• Most grammar rule formalisms are based on the
  idea of phrase structure
• Strings are composed of substrings called
  phrases, which come in different categories.
• Examples of the category noun phrase, or NP
• the king
• the agent in the corner

9
10
Fundamentals of language

• Phrases usually correspond to natural semantic
  elements
• from which the meaning of an utterance can be
  constructed for example
• Noun phrases refer to objects in the world.
• Categorizing phrases helps us to describe the
  allowable strings of the language.
• Any of the noun phrases can combine with a verb
  phrase (or VP) such as is dead to form a phrase
  of category sentence (or S).

11
Fundamentals of language

• Without the intermediate notions of NP and VP, it
  would be difficult to explain why the king is
  dead is a sentence whereas king the dead is is
  not.
• Category names such as NP, VP and S are called
  nonterminal symbols.
• Grammars define nonterminals using rewrite rules
• S ? NP VP
• An S may consist of any NP followed by any VP.

12
Levels of analysis in NLP

• Lexico-morphological
• Detecting lexical units and their morphological
  information
• Syntactic
• Checking if a sentence is syntactically valid
• Semantic
• Extracting global meaning from individual
  meanings and from relations
• Pragmatic
• Relating a sentence to the line of discussion
• Illocutive
• Relating a sentence to intentions

13
Problems in NLP examples

• Lexical ambiguity
• reinventing the front wheel
• wheel can be a noun or a verb (part-of-speech
  tagging or POS-tagging)
• she saw the bank
• Building of a financial institution? Sloping
  land? Supply held in reserve for future use?
  (word sense disambiguation or WSD)

14
Problems in NLP examples

• Syntactic ambiguity
• He saw a man on the mountain top with
  binoculars
• Whos got the binoculars?
• The seller of newspapers of the neighborhood
• What is the prepositional-phrase attached to?
  (prepositional-phrase attachment or PP-attachment)

15
Problems in NLP examples

• Semantic ambiguity
• He gave the children a cake
• A cake in total or one to each child? (scope of
  the quantification)
• Colorless green ideas sleep furiously
• Sentence composed by Noam Chomsky in 1957 as an
  example of a sentence whose grammar is correct
  but whose meaning is nonsensical.
• It was used to show inadequacy of the
  then-popular probabilistic models of grammar, and
  the need for more structured models.

16
Problems in NLP examples

• References, ellipsis, pragmatics
• She gave him a book
• "We gave the monkeys the bananas because they
  were hungry
• "We gave the monkeys the bananas because they
  were over-ripe"
• Same surface grammatical structure. However, the
  pronoun they refers to monkeys in one sentence
  and bananas in the other, and it is impossible to
  tell which without a knowledge of the properties
  of monkeys and bananas.

17
Problems in NLP examples

• Illocution (Where is the stress? What
  intentions?)
• "I never said she stole my money" - Someone else
  said it, but I didn't.
• "I never said she stole my money" - I simply
  didn't ever say it.
• "I never said she stole my money" - I might have
  implied it in some way, but I never explicitly
  said it.
• "I never said she stole my money" - I said
  someone took it I didn't say it was she.
• "I never said she stole my money" - I just said
  she probably borrowed it.
• "I never said she stole my money" - I said she
  stole someone else's money.
• "I never said she stole my money" - I said she
  stole something, but not my money.

18
Statistical natural-language processing

• It uses stochastic, probabilistic and statistical
  methods to resolve some of the difficulties
  discussed above, especially those which arise
  because longer sentences are highly ambiguous
  when processed with realistic grammars, yielding
  thousands or millions of possible analyses.
• Methods for disambiguation often involve the use
  of corpora and Markov models.
• Statistical NLP comprises all quantitative
  approaches to automated language processing,
  including probabilistic modeling and information
  theory.
• The technology for statistical NLP comes mainly
  from machine learning and data mining, both of
  which are fields of artificial intelligence that
  involve learning from data.

19
Major tasks and applications in NLP

• Automatic summarization
• Foreign language reading aid
• Foreign language writing aid
• Information extraction
• Information retrieval (IR)
• IR is concerned with storing, searching and
  retrieving information.
• It is a separate field within computer science
  (closer to databases), but IR relies on some NLP
  methods (for example, stemming).
• Some current research and applications seek to
  bridge the gap between IR and NLP.

20
Major tasks and applications in NLP

• Machine translation
• Automatically translating from one human language
  to another.
• Named entity recognition (NER)
• Given a stream of text, determining which items
  in the text map to proper names, such as people
  or places.
• Although in English, named entities are marked
  with capitalized words, many other languages do
  not use capitalization to distinguish named
  entities.

21
Major tasks and applications in NLP

• Natural language generation
• Natural language understanding
• Optical character recognition (OCR)
• Question answering
• Given a human language question, the task of
  producing a human-language answer.
• The question may be a closed-ended (such as "What
  is the capital of Canada?") or open-ended (such
  as "What is the meaning of life?").

22
Major tasks and applications in NLP

• Speech recognition
• Given a sound clip of a person or people
  speaking, the task of producing a text dictation
  of the speakers.
• (The opposite of text to speech.)
• Spoken dialogue system
• Text simplification
• Text-to-speech
• Text-proofing

23
Resources

• Natural language processing (in Spanish)
  http//es.geocities.com/lenguajenatural/
• Introductory book http//www.gelbukh.com/clbook/
  
• Resources for text, speech and language
  processing http//www.cs.technion.ac.il/gabr/res
  ources/resources.html
• Natural language processing blog
  http//nlpers.blogspot.com/

24
Resources

• About Opinion, Language, and Blogs
  http//opinlab.wordpress.com/
• A comprehensive list of resources, classified by
  category http//www.proxem.com/
• ACL Wiki for natural language processing and
  computational linguistics http//aclweb.org/aclwi
  ki/index.php?titleMain_Page

25
Research and development groups

• IBM NLP Research Area http//domino.watson.ibm.co
  m/comm/research.nsf/pages/r.nlp.html
• Microsoft Research NLP http//research.microsoft
  .com/nlp/
• Language Technologies Institute at Carnegie
  Mellon University http//www.lti.cs.cmu.edu/
• Natural Language Group at the Information
  Sciences Institute http//www.isi.edu/natural-lan
  guage/
• Natural Language Generation Group at the Open
  University http//mcs.open.ac.uk/nlg/

26
Research and development groups

• Survey of the State of the Art in Human Language
  Technology http//cslu.cse.ogi.edu/HLTsurvey/
• University of Edinburgh Natural Language
  Processing Group http//www.iccs.informatics.ed.a
  c.uk/
• Natural Language and Information Processing Group
  at the University of Cambridge http//www.cl.cam.
  ac.uk/research/nl/
• Stanford Natural Language Processing Group
  http//nlp.stanford.edu/
• UPC center for research and technology
  development on language and speech processing
  (TALP) http//www.talp.cat/talp/