Why model language

1
Why model language?

• for application a model can predict the
  linguistic characteristics (pronunciation, parts
  of speech, structure, meaning) of an unknown
  string of words
• for theory a model can mimic human language
  behavior

2
Two Current Approaches in NLP

• Symbolic (our approach this semester)
• depends on systems of rules (syntactic,
  phonological, semantic, morphological) to analyze
  or generate data
• used in text understanding, QA, text editing,
  etc.
• the traditional approach
• Statistical
• depends on statistical analysis of large corpora
• used in speech recognition, information
  retrieval, author identification, etc.

3
Example of a statistical model Part of speech
tagging

• Text A The woman ate the cheese before the mouse
• could get across the floor to snatch it. The
  mouse
• was then throttled by the cat before it could
  escape
• into the hole in the wall. The poor mouse
  thought
• the end had come.
• Text B The mouse never imagined that he would
  see the _________.
• Assuming that Text B is like Text A, What is the
  probability that __________ in B is a noun?

4
Example of a statistical model (2)

• Text A The woman ate the cheese before the mouse
• could get across the floor to snatch it. The
  mouse
• was then throttled by the cat before it could
  escape
• into the hole in the wall. The poor mouse
  thought
• the end had come.
• Text B The mouse never imagined that he would
  see the _________.
• Text A In 9 cases, the is followed by a Noun.
• In 1 case, it is followed by an Adjective.

5
Example of a statistical model (3)

• The probability that _________ in B is a Noun is
  .9 - very high the likelihood that it is an
  adjective is .1 - very low.
• So a statistical part of speech tagger will tag
  _____ in Text B as a Noun.

6
Example of a statistical model (4)

• The model is a statistical formula
• P(nounarticle) P(noun ? article)/P(article)
• P 9/10 .9
• (P is probability)

7
Statistical Models

• require training texts (Text A)
• the training sets must be tagged by human judges
  TheArt mouseNoun thoughtVerb . . . .
• the training set must be large so there will be a
  significant number of cases of each part of
  speech
• usually tested on a test set of data that is like
  the training set.

8
Example of a symbolic model

• Rules
• np ? det (adj) n
• A Noun Phrase is a Determiner an
  adjective (optionally) a Noun
• A dictionary
• word(the,det).
• word(mouse,n).
• word(poor,adj).

9
A Symbolic Model has

• rules
• s ? np vp
• np ? det (adj) n
• vp ? v (np)
• a dictionary that specifies part of speech info
  for each word in the language.
• word(the,det).
• word(mouse,n).
• word(ran,v).