Computational Semantics

1
Computational Semantics

• CS224U 2007
• Christopher Manning
• Borrows slides from Mary Dalrymple, Jason Eisner,
  Jim Martin, and especially Aljoscha Burchardt,
  Alexander Koller, and Stephan Walters ESSLLI
  2004 course.

2
Shallow vs. deep semantics

• Weve looked at various notions of lexical
  meaning and various kinds of temporal and
  discourse relations
• But we havent really tried to work out what
  sentences mean
• But we cant do everything we would like
• Not all tasks can ignore detailed sentence
  structure
• Unsuitable if machine must act rather than
  relying on user to interpret material written by
  the author of the document
• You get what you pay for
• Cheap, fast, low-level techniques are appropriate
  in domains where speed and volume are more
  important than accuracy
• More computationally expensive, higher-level
  techniques are appropriate when high-quality
  results are required

3
Precise semantics. An early example Chat-80

• Developed between 1979 and 1982 by Fernando
  Pereira and David Warren became Pereiras
  dissertation
• Proof-of-concept natural language interface to
  database system
• Used in projects e.g. Shoptalk (Cohen et al.
  1989), a natural language and graphical interface
  for decision support in manufacturing
• Even used in an AppliedNLP-2000 conference paper!
  Asking about train routes and schedules
• Available in cs224n src directory
• Need sicstus prolog /usr/sweet/bin/sicstus

4
The CHAT-80 Database

• Facts about countries.
• country(Country,Region,Latitude,Longitude,
• Area (sqmiles), Population,
  Capital,Currency)
• country(andorra,southern_europe,42,-1,179,
  25000,andorra_la_villa,franc_peseta).
• country(angola,southern_africa,-12,-18,481351,
  5810000,luanda,?).
• country(argentina,south_america,-35,66, 1072067,
  23920000,buenos_aires,peso).
• capital(C,Cap) - country(C,_,_,_,_,_,Cap,_).

5
Chat-80 trace (illegibly small)

• Question What is the capital of Australia?
• Parse 0.0sec.
• whq
• VAR
• 1
• s
• np
• 3sin
• wh(B)
• verb(be,active,presfin,,pos)
• arg
• dir
• np
• 3sin

• np_head
• det(the(sin))
• capital
• pp
• prep(of)
• np
• 3sin
• name(australia)
• Semantics 0.0sec.
• answer(B) -
• capital(australia,B)
• canberra.

6
What Counts as Understanding?

• A somewhat difficult philosophical question
• We understand if we can respond appropriately
• throw axe at dwarf
• We understand statement if we can determine its
  truth
• We understand statement if we can use it to
  answer questions similar to above requires
  reasoning
• Easy John ate pizza. What was eaten by John?
• Understanding is the ability to translate
• English to Chinese? requires deep
  understanding?
• English to logic? deepest - the definition
  well use!
• all humans are mortal ?x human(x)
  ?mortal(x)
• We can represent truth in a model
• We assume we have logic-manipulating rules to
  tell us how to act, draw conclusions, answer
  questions

7
(Formal/Compositional) Computational Semantics

• Sentences John smokes.
• Everyone who smokes snores.
• Syntactic Analyses S
• NP VP
• John smokes
• Semantics Construction smoke(j)
• Inference ?x.smoke(x)?snore(x),smoke(j)
• gt snore(j)

8
Meaning Representations

• Meaning representations of NL sentences
• First Order Logic (FOL) as formal language
• John smokes.
• gt smoke(j)
• Sylvester loves Tweety.
• gt love(s,t)
• Sylvester loves every bird.
• gt ?x.(bird(x) ? love(s,x))

9
FOL Syntax (very briefly)

• FOL Formulae, e.g. ?x.(bird(x) ? love(s,x))
• FOL Language
• Vocabulary (constant symbols and
  predicate/relation symbols)
• Variables
• Logical Connectives
• Quantifiers
• Brackets, dots.

10
Logic Interesting Constants

• We have constants that name some of the
  entities and functions
• GeorgeWBush - an entity
• red a predicate on entities
• holds of just the red entities red(x) is true if
  x is red!
• loves a predicate on 2 entities
• loves(GeorgeWBush, LauraBush)
• Question What does loves(LauraBush) denote?
• Constants used to define meanings of words
• Meanings of phrases will be built from the
  constants

11
In the Background Model Theory

• ?x.(bird(x) ? love(s,x)) is a string again!
• Mathematically precise model representation,
  e.g. cat(s), bird(t), love(s,t), granny(g),
  own(g,s), own(g,t)
• Inspect formula w.r.t. to the model Is it true?
• Inferences can extract information Is anyone not
  owned by Granny?

12
Compositional Semantics

• Weve discussed what semantic representations
  should look like.
• But how do we get them from sentences???
• First - parse to get a syntax tree.
• Second - look up the semantics for each word.
• Third - build the semantics for each constituent
• Work from the bottom up
• The syntax tree is a recipe for how to do it
• Principle of Compositionality
• The meaning of a whole is derived from the
  meanings of the parts, via composition rules

13
A simple grammar of English(in Definite Clause
Grammar, DCG, form as in Prolog)

• sentence --gt noun_phrase, verb_phrase.
• noun_phrase --gt proper_noun.
• noun_phrase --gt determiner, noun.
• verb_phrase --gt verb, noun_phrase.
• Proper_noun --gt John verb --gt ate
• Proper_noun --gt Mary verb --gt kissed
• determiner --gt the noun --gt cake
• determiner--gt a noun --gt lion

14
Extending the grammar to check number agreement
between subjects and verbs

• S --gt NP(Num), VP(Num).
• NP(Num) --gt Proper_noun(Num).
• NP(Num) --gt det(Num), noun(Num).
• VP(Num) --gt verb(Num), noun_phrase(_).
• Proper_noun(s) --gt Mary. noun(s) --gt lion.
• det(s) --gt the. noun(p) --gt lions.
• det(p) --gt the. verb(s) --gt eats.
• verb(p) --gt eat.

15
A simple DCG grammar with semantics

• sentence(SMeaning) --gt noun_phrase(NPMeaning),
  verb_phrase(VPMeaning), combine (NPMeaning,
  VPMeaning, SMeaning).
• verb_phrase(VPMeaning) --gt verb(Vmeaning),
  noun_phrase(NPMeaning), combine (NPMeaning,
  VMeaning, VPMeaning).
• noun_phrase (NPMeaning) --gt name(NPMeaning).
• name(john) --gt john. verb(?x.jumps(x)) --gt
  jumps
• name(mary) --gt mary. verb(?y.?x.loves(x,y))
  --gtloves
• Combine(X, Y, Z) --gt apply(Y, X, Z)

16
Syntactic Analyses on trees with lambda
expressions

• Basis Context Free Grammar (CFG)
• Grammar Rules
• S ? NP VP
• VP ? TV NP
• TV ? love
• NP ? john Lexical Rules / Lexicon
• NP ? mary
• ...

17
Example Syntactic Analyses
18
Example Semantic Lexicon
19
Example Semantics Construction
20
Example Semantics Construction
21
Compositionality

• The meaning of the sentence is constructed from
• The meaning of the words john, mary, love(?,?)
  (lexicon)
• Paralleling the syntactic construction (semantic
  rules)

22
Systematicity

• How do we know that e.g. the meaning of the VP
  loves Mary is constructed as
• love(?,mary) and not as
• love(mary,?) ?
• Better How can we specify in which way the bits
  and pieces combine?

23
Systematicity (ctd.)

• Parts of formulae (and terms), e.g. for the VP
  love Mary?
• love(?,mary) bad not FOL
• love(x,mary) bad no control over free
  variable
• Familiar well-formed formulae (sentences)
• ?x.love(x,mary) Everyone loves Mary.
• ?x.love(mary,x) Mary loves someone.

24
Using Lambdas (Abstraction)

• Add a new operator to bind free variables
• ?x.love(x,mary) to love Mary
• The new meta-logical symbol ? marks missing
  information in the object language (?-)FOL
• We abstract over x.
• How do we combine these new formulae and terms?

25
Super Glue

• Gluing together formulae/terms with function
  application
• ?x.love(x,mary) john
• Often written as ?x.love(x,mary)(john)
• How do we get back to the familiar
  love(john,mary)?

26
Functional Application

• Glueing is known as Functional Application
• FA has the Form Functor(Argument)
• ?x.love(x,mary)(john)
• FA triggers a very simple operation
• Replace the ?-bound variable by the argument.
• ?x.love(x,mary)(john)
• gt love(john,mary)

27
?-Reduction/Conversion

• Strip off the ?-prefix,
• Remove the argument,
• Replace all occurrences of the ?-bound variable
  by the argument.
• ?x.love(x,mary)john
• love(x,mary)john
• love(x,mary)
• love(john,mary)

28
Semantics Construction with Lambdas

• S John loves Mary
• (?y?x.love(x,y)mary)john

NP John john
VP loves Mary ?y?x.love(x,y)mary
TV loves ?y?x.love(x,y)
NP Mary mary
29
Example Beta-Reduction

• (?y?x.love(x,y)mary)john
• ?gt (?x.love(x,mary))john
• ?gt love(john,mary)

30
Augmented CFG Rules

• We can also accomplish this just by attaching
  semantic formation rules to our syntactic CFG
  rules
• This should be read as the semantics we attach to
  A can be computed from some function applied to
  the semantics of As parts.
• The functions/operations permitted in the
  semantic rules are restricted, falling into two
  classes
• Pass the semantics of a daughter up unchanged to
  the mother
• Apply (as a function) the semantics of one of the
  daughters of a node to the semantics of the other
  daughters

31
In the Background

• ?-Calculus
• A logical standard technique offering more than
  ?-abstraction, functional application and
  ß-reduction.
• Other Logics
• Higher Order Logics
• Intensional Logics
• ...
• For linguistics Richard Montague (early
  seventies)

32
Adding more complex NPs

• NP A man gt ?x.man(x)
• S A man loves Mary
• gt love(?x.man(x),mary)
• How to fix this?

33
A disappointment

• Our first idea for NPs with determiner didnt
  work out
• A man gt ?z.man(z)
• A man loves Mary gt love(?z.man(z),mary)

But what was the idea after all? Nothing!
?z.man(z) just isnt the meaning of a man.
If anything, it translates the complete
sentence There is a man
Lets try again, systematically
34
A solution

• What we want is
• A man loves Mary gt ?z(man(z) ? love(z,mary))

What we have is man gt ?y.man(y) loves
Mary gt ?x.love(x,mary)
?z(man(z) ? love(z,mary))
How about
?z(man(z) ? love(z,mary))
?z(?y.man(y)(z) ? love(z,mary))
?z(?y.man(y)(z) ? love(z,mary))
?z(?y.man(y)(z) ? ?x.love(x,mary)(z))
?z(?y.man(y)(z) ? ?x.love(x,mary)(z))
Remember We can use variables for any kind of
term. So next
?z(?y.man(y)(z) ? ?x.love(x,mary)(z))
?z(?y.man(y)(z) ? ?x.love(x,mary)(z))
?Q.
Q(z)) ?x.love(x,mary)
?P(
P
?P(?Q.?z(P(z) ?Q(z)))
)?y.man(y)
lt A
35
But

• A man loves Mary

?P(?Q.?z(P(z) ?Q(z))) ?y.man(y)
?x.love(x,mary)
?Q.?z(man(z)?Q(z))
?z.man(z) ? ?x.love(x,mary)(z)
man(z) ? love(z,mary)
?P(?Q.?z(P(z)?Q(z)))_at_?y.man(y)
?x.love(x,mary)
_at_
fine!
John loves Mary
?x.love(x,mary)
john
not systematic!
?x.love(x,mary)
john
not reducible!
_at_
?x.love(x,mary)
?
better!
?P.P_at_john
?x.love(x,mary)_at_john
love(john,mary)
So John gt ?P.P(john)
36
Transitive Verbs
What about transitive verbs (like "love")?
"loves" gt ?y?x.love(x,y) ???
won't do
"Mary" gt ?Q.Q(mary)
?
"loves Mary" gt ?y?x.love(x,y)_at_?Q.Q(mary)
?x.love(x,?Q.Q(mary))
How about something a little more complicated
"loves" gt ?R?x(R?y.love(x,y))
The only way to understand this is to see it in
action...
37
"John loves Mary" again...
love(john,mary)
love(john,mary)
?x.love(x,mary)(john)
love(john,mary)
?x(?y.love(x,y)(mary))
?x.love(x,mary)
?R?x(R?y.love(x,y))
?P.P(mary)
?P.P(john)
loves
John
Mary
38
Summing up

• nouns man gt ?x.man(x)
• intransitive verbs smoke gt ?x.smoke(x)
• determiner a gt ?P(?Q.?z(P(z) ?Q(z)))
• proper names mary gt ?P.P(mary)
• transitive verbs love gt ?R?x(R?y.love(x,y)
  )

39
Success!

• What we can do now (and could not do before)
• Complex NPs (with determiners)
• Transitive verbs
• and all in the same way.
• Key ideas
• Extra ?s for NPs
• Variables for predicates
• Apply subject NP to VP

40
Quantifier Order

• Groucho Marx celebrates quantifier order
  ambiguity
• In this country a woman gives birth every 15 min.
  Our job is to find that woman and stop her.
• ?woman (?15min gives-birth-during(woman, 15min))
• ?15min (?woman gives-birth-during(15min, woman))
• Surprisingly, both are possible in natural
  language!
• Which is the joke meaning?
• (where its always the same woman)

41
How do things get more complex? (The former) GRE
analytic section

• Six sculptures C, D, E, F, G, H are to be
  exhibited in rooms 1, 2, and 3 of an art gallery.
• Sculptures C and E may not be exhibited in the
  same room.
• Sculptures D and G must be exhibited in the same
  room.
• If sculptures E and F are exhibited in the same
  room, no other sculpture may be exhibited in that
  room.
• At least one sculpture must be exhibited in each
  room, and no more than three sculptures may be
  exhibited in any room.
• If sculpture D is exhibited in room 3 and
  sculptures E and F are exhibited in room 1, which
  of the following may be true?
• Sculpture C is exhibited in room 1.
• Sculpture H is exhibited in room 1.
• Sculpture G is exhibited in room 2.
• Sculptures C and H are exhibited in the same
  room.
• Sculptures G and F are exhibited in the same room.

42
Scope Needs to be Resolved!

• At least one sculpture must be exhibited in each
  room.
• The same sculpture in each room?
• No more than three sculptures may be exhibited in
  any room.
• Reading 1 For every room, there are no more
  than three sculptures exhibited in it.
• Reading 2 Only three or less sculptures are
  exhibited ( the rest are not shown).
• Reading 3 Only a certain set of three or less
  sculptures may be exhibited in any room ( for the
  other sculptures there are restrictions in
  allowable rooms).
• Some readings will be ruled out by being
  uninformative or by contradicting other
  statements
• Otherwise we must be content with distributions
  over scope-resolved semantic forms

43
Ambiguity

• Some sentences have more than one reading, i.e.
  more than one semantic representation.
• Standard example "Every man loves a woman"
• Reading 1 the women may be different
• ?x(man(x) -gt ?y(woman(y) ? love(x,y)))
• Reading 2 there is one particular woman
• ?y(woman(y) ? ?x(man(x) -gt love(x,y)))
• What does our system do?

44
Logic Interesting Constants

• Generalized Quantifiers
• most a predicate on 2 predicates on entities
• most(pig, big) most pigs are big
• Equivalently, most(?x pig(x), ?x big(x))
• returns true if most of the things satisfying the
  first predicate also satisfy the second predicate
• similarly for other quantifiers
• all(pig,big) (equivalent to ?x pig(x) ? big(x))
• exists(pig,big) (equivalent to ?x pig(x) AND
  big(x))
• can even build complex quantifiers from English
  phrases
• between 12 and 75 a majority of all but
  the smallest 2

45
Coordination

• "John and Mary"
• (?X. ?Y.?P((XP) ? (YP)) ?Q.Q(john))?R.R(ma
  ry)

?P((?Q.Q(john)_at_P) ? (?R.R(mary)_at_P))
?P(P(john) ? P(mary))
"John and Mary walk"
?P(P(john) ? P(mary))_at_ ?x.walk(x)
?x.walk(x)john ? ?x.walk(x)mary
lambda(X,walk(X))john lambda(X,walk(X))mary