CPSC 503 Computational Linguistics

1
CPSC 503Computational Linguistics

• Context-Free Grammars
• Lecture 11
• Giuseppe Carenini

2
Today 25/2

• Syntax
• Context-Free Grammar
• Constituency
• Rules
• Trees
• Recursion
• More complex English constructs
• Problems

3
Knowledge-Formalisms Map(including probabilistic
formalisms)
State Machines (and prob. versions) (Finite State
Automata,Finite State Transducers, Markov Models)
Morphology
Syntax
Rule systems (and prob. versions) (e.g., (Prob.)
Context-Free Grammars)
Semantics

• Logical formalisms
• (First-Order Logics)

Pragmatics Discourse and Dialogue
AI planners
4
Syntax

• Def. The study of how sentences are formed by
  grouping and ordering words

Example Ming and Sue prefer morning flights
Ming Sue flights morning and prefer
5
Syntactic Notions so far...

• ngrams prob. distr. for next word can be
  effectively approximated knowing previous n words
  

• POS categories are based on
• distributional properties (what other words can
  occur nearby)
• morphological properties (affixes they take)

6
Syntax

• Why should you care?
• Grammar checkers
• Basis for semantic interpretation
• Question answering
• Information extraction
• Machine translation

7
Key Constituents (English)
(Specifier) X (Complement)

• Noun phrases
• Verb phrases
• Prepositional phrases
• Adjective phrases
• Sentences

• (Det) N (PP)
• (Qual) V (NP)
• (Deg) P (NP)
• (Deg) A (PP)
• (NP) (I) (VP)

Some simple specifiers Category Typical
function Examples Determiner specifier of N
the, a, this, no.. Qualifier specifier
of V never, often.. Degree word
specifier of A or P very, almost..
Complements?
8
Key Constituents Examples

• (Det) N (PP)
• the cat on the table
• (Qual) V (NP)
• never eat a cat
• (Deg) P (NP)
• almost in the net
• (Deg) A (PP)
• very happy about it
• (NP) (I) (VP)
• a mouse -- ate it

• Noun phrases
• Verb phrases
• Prepositional phrases
• Adjective phrases
• Sentences

9
CFG Example

• S -gt NP VP
• NP -gt Det NOMINAL
• NOMINAL -gt Noun
• VP -gt Verb
• Det -gt a
• Noun -gt flight
• Verb -gt left

• Non-terminal

• Terminal

10
CFG more complex Example

• Lexicon

• Grammar with example phrases

11
Rules key points

• S -gt NP VP
• says
• that there are units called S, NP, and VP in this
  language
• that an S consists of an NP followed immediately
  by a VP

• but does not say
• that thats the only kind of S
• that this is the only place that NPs and VPs occur

12
CFGs

• Define a Formal Language (un/grammatical
  sentences)
• Generative Formalism
• Generate strings in the language
• Reject strings not in the language
• Impose structures (trees) on strings in the
  language

13
CFG Formal Definitions

• 4-tuple (non-term., term., productions, start)
• (N, ?, P, S)
• P is a set of rules A?? A?N, ??(??N)

• A derivation is the process of rewriting ?1 into
  ? m (both strings in (??N)) by applying a
  sequence of rules ?1 ? ? m

• L G Ww?? and S ? w

14
Derivations as Trees
15
CFG Parsing

• It is completely analogous to running a
  finite-state transducer with a tape
• Its just more powerful
• Next class (Chpt. 10)

16
Context in CFG

• Nothing to do with the ordinary meaning of the
  word context in language.

A -gt B C

• A can be rewritten as a B followed by a C
  regardless of the context in which A is found

17
Other Options

• Regular languages (FSA/RegExp)
• Too weak (e.g., cannot deal with recursion in a
  general way)
• Recursion is quite common in a complete model of
  the NP
• CFGs
• Context-sensitive
• Can be computationally intractable
• Turing equiv.
• Too powerful / Computationally intractable

18
Recursion

• Rules such as the following where the
  non-terminal on the left also appears somewhere
  on the right (directly).

NP -gt NP PP
flights from Denver Flights from
Denver to Miami Flights from Denver
to Miami in February Flights from
Denver to Miami in February on a
Friday Flights from Denver to
Miami in February on a Fridayunder
300 Flights from Denver to Miami
in February on a Fridayunder 300with
lunch
19
Common Sentence-Types

• Declaratives A plane left
• S -gt NP VP
• Imperatives Leave!
• S -gt VP
• Yes-No Questions Did the plane leave?
• S -gt Aux NP VP
• WH Questions
• Which flights serve breakfast?
• S -gt WH NP VP
• When did the plane leave?
• S -gt WH Aux NP VP

20
NP more details

• NP -gt Specifiers N Complements

• NP -gt (Predet)(Det)(Card)(Ord)(Quant) (AP) Nom
• e.g., all the other cheap
  cars

• Nom -gt Nom PP (PP) (PP)
• e.g., reservation on BA456 from NY to YVR

Nom -gt Nom GerundVP e.g., flight arriving
on Monday Nom -gt Nom RelClause Nom RelClause
-gt(who that) VP e.g., flight that arrives
in the evening
21
Conjunctive Constructions

• S -gt S and S
• John went to NY and Mary followed him

• NP -gt NP and NP
• John went to NY and Boston

• VP -gt VP and VP
• John went to NY and visited MOMA

• In fact the right rule for English is
• X -gt X and X

22
Problems with CFGs

• Agreement
• Subcategorization

23
Agreement

• In English,
• Determiners and nouns have to agree in number
• Subjects and verbs have to agree in person and
  number

• Many languages have agreement systems that are
  far more complex than this (e.g., gender).

24
Agreement

• This dog
• Those dogs
• This dog eats
• You have it
• Those dogs eat

• This dogs
• Those dog
• This dog eat
• You has it
• Those dogs eats

25
Possible CFG Solution
OLD Grammar
NEW Grammar

• S -gt NP VP
• NP -gt Det Nominal
• VP -gt V NP

• SgS -gt SgNP SgVP
• PlS -gt PlNp PlVP
• SgNP -gt SgDet SgNom
• PlNP -gt PlDet PlNom
• PlVP -gt PlV NP
• SgVP -gtSgV Np

Sg singular Pl plural
26
CFG Solution for Agreement

• It works and stays within the power of CFGs
• But it doesnt scale all that well (explosion in
  the number of rules)

27
Subcategorization

• Def. expresses constraints that a predicate (verb
  here) places on the number and type of its
  arguments (see first table)

• John sneezed the book
• I prefer United has a flight
• Give with a flight

28
Subcategorization

• Sneeze John sneezed
• Find Please find a flight to NYNP
• Give Give meNPa cheaper fareNP
• Help Can you help meNPwith a flightPP
• Prefer I prefer to leave earlierTO-VP
• Told I was told United has a flightS

29
So?

• So the various rules for VPs overgenerate.
• They permit the presence of strings containing
  verbs and arguments that dont go together
• For example VP -gt V NP therefore
• Sneezed the book is a VP since sneeze is a
  verb and the book is a valid NP

30
Possible CFG Solution
OLD Grammar
NEW Grammar

• VP -gt IntransV
• VP -gt TransV NP
• VP -gt TransPPto NP PPto
• TransPPto -gt hand,give,..

• VP -gt V
• VP -gt V NP
• VP -gt V NP PP

This solution has the same problem as the one for
agreement
31
The Point

• CFGs cover most syntactic structure in English.
• But there are problems
• That can be dealt with adequately, although not
  elegantly, by staying within the CFG framework.
• There are simpler, more elegant, solutions that
  take us out of the CFG framework (beyond its
  formal power)
• Chpt 11 Features and Unification

32
For Next Time

• Read Chapter 10 (Parsing)
• If you have not done it yet, come and talk to me
  about your project (proposal and class
  presentation on Fri, Mar.5)