Inge Slingerland

1
Generation of Referring Expressions

• Inge Slingerland

Krahmer, E., van Erk, S. Verleg, A. (2003)
Graph-Based Generation of Referring Expressions.
Computational Linguistics, 29(1)5372.
Van Deemter, K. Krahmer, E. (2006) Graphs and
Booleans On the Generation of Referring
Expressions. In H.Bunt and R.Muskens (Eds.)
Computing Meaning, Volume 3, Studies in
Linguistics and Philosophy, Kluwer, Dordrecht.
2
Overview

• General info
• Graph based GRE
• Cost functions
• Extensions
• Linguistic realization

3
Generation of Referring Expressions (GRE)

• Find a combination of properties which let the
  Natural Language Generator refer uniquely to a
  (set of) object(s)

4
Content determination problem

• Target Object for which a referring expression
  is to be generated
• Confusables Object(s) from which the target
  needs to be distinguished
• To determine which set of properties is needed to
  single out the target object from the
  confusables.

5
Example
6
Older algorithms

• Full Brevity Algorithm
• The smallest amount of properties that
  distinguish the target from the confusables
• But NP-hard and psychologically unrealistic
• Incremental Algorithm
• Considers properties in predetermined order
• s1 conjoining ltSIZE, Biggt and ltTYPE, Musiciangt
• Because s1,s3 n s1, s2 s1
• Incrementally conjoining more and more
  properties, removing more and more confusables.

7
Advantages Graph-Based

• Graph structures have been studied extensively
• Many existing generation algorithms can be
  reformulated in terms of graphs
• The graph perspective allows us to solve a number
  of problems that have plagued earlier algorithms
  for the generation of referring expressions (f.e.
  the generation of relational expressions)

8
New formalism

• New formalism for expressing and implementing
  different GRE algorithms
• Labelled directed graphs
• G ltVG, EGgt labelled directed graph
• VG set of vertices (the possible referents)
• EG set of labelled directed edges

9
Graph based GRE
Scene graph S

• Properties are loops
• Relations are edges between nodes
• If s is the target object, the scene pair is S
  lts,Sgt

10
Description Graph (1)

• A description pair ? ltd,Dgt
• With D a description graph and d ? VD
• Task Construct a description pair which refers
  uniquely to a given scene pair

3 description graphs D
11
Description Graph (2)

• Target is s2 (the small musician with the
  trumpet)
• 1 refers to s2, but not uniquely (confusable is
  s1)
• 2 and 3 refer uniquely to s2

3 description graphs D
12
Multiple unique descriptions - Which one to take?

• Take the shortest unique description (Brevity
  Algorithm)
• Considering properties in fixed order, stop once
  a unique description is found (IA)
• Cost Functions

13
Cost Functions

• Every edge has a cost attached to it
• Graph with lowest cost is selected
• Monotonicity
• adding an edge e to a graph G should never result
  in a graph cheaper than G

14
Algorithm (1)

• Input is the scene pair
• Description pair is initialized with the target s
  and the initial description graph D whose only
  node is s
• bestGraph contains best solution so far
  initialized as the empty graph
• Systematically expand D by adding adjacent edges
• For each D check de set of confusables. If
  confusables is s we found a distinguishing
  description -gt store in bestGraph.
• Now only look for description graphs that are
  cheaper than bestGraph
• Output is the cheapest distinguishing description
  graph, following from the monotonicity
  requirement
• If the output is the null graph no uniquely
  referring description

15
Algorithm (2)
16
Costs - Incremental Algoritm

• IA uses preferred attributes lt type, color, size
  gt
• Give edges corresponding to absolute properties
  (type, color) lower costs than those
  corresponding to relative ones (size)
• Incremental aspect of IA
• Try the cheapest edges first
• And terminate when the first distinguishing graph
  is found.

17
Costs - Subsumption Hierarchy

• Chihuahua refer to by chihuahua or dog
• Dog is basic level value
• Preferenced by humans
• Basic level edges lowest costs, farthest levels
  away have highest costs

18
Costs Stochastic Cost Functions

• Frequently occurring properties are cheap
• Relatively rare properties are expensive
• cost (e) - 2log(P(e))
• P(e) Probability that e occurs in a
  distinguishing description
• polish owczarek nizinny sheepdog (type) costs
  more than brown (color)
• But co-occurrence of edges is not likely to be
  fully independent

19
Extensions Set as target

• A set as target
• Target s1,s2
• Conjoin properties till the set of Confusables
  only contains the target set
• Input no longer single node, but all the target
  nodes
• Outcome Musician

20
Extensions Gradable Properties

• Problems
• Small means different things for a trumpet than
  for a musician
• Smallest musician ? smallest object in the domain
  AND a musician
• List absolute values in KB
• Let GRE decide what counts as small in a given
  context
• Add derived properties to KB Size (x) gt value
• Only limited number of inequalities needs to be
  added

21
Extensions Gradable Properties

• Add derived inequalities as edges
• Size (s1) 185 cm
• Size (s2) 157 cm
• Size (s3) 185 cm
• Size (s4) 30 cm
• s2 Musician lt185cm
• Theoretical complexity doesnt change, but edges
  grows quadratically

22
Extensions Salience

• Earlier algorithms assumed all objects were
  equally salient not psychologically realistic
• Algorithm starts by restricting the domain to the
  set of objects that are at least as salient as
  the target set
• Confusables set of nodes that are at least as
  salient as the target
• So only properties that remove salient
  distractors will be added

23
Extensions Salience

• Add salience as a gradable property
• Properties like Salience (x) gt value
• Salience-properties are free w.r.t. to costs
• But
• only singular references
• no other gradable properties
• The big piano player
• The biggest of the piano players that are
  sufficiently salient?
• The most salient of the piano players that are
  sufficiently big?

24
Extensions Negations

• Target is s3,s4 (the technician and the
  trumpet)
• Not possible if only atomic properties are taken
  into account
• Negations
• Makes some characterizations possible
• all objects who are not Musicians
• Makes some characterizations more easy and
  psychologically realistic
• The person not holding an instrument
• Instead of The tall woman in the front row, with
  the pearl necklace and yellow coat.

25
Extensions Negations

• Make explicit what was implicit add edges for
  negations
• Following Closed World Assumption
• Theoretical complexity of the algorithm isnt
  altered, but the scene graph may grow drastically

26
Extensions Negations

• What should be the costs of adding a negated
  property?

27
Extensions Boolean Descriptions

• Also disjunctions possible with graph-based
  approach
• Target s1,s2,s3
• All either musicians or technicians
• Add edges labelled with binary disjunctions to
  the scene graph
• MusicianTechnician

28
Choosing costs

• What descriptions should be preferred (or have
  lower costs)?
• Adding a negation (The musicians that are not
  technicians)
• Adding a disjunction (The trumpettists and
  guitarists)
• Adding a relation with another object (The
  musicians in the front row)

29
Linguistic realization

• How should the selected properties be realized in
  natural language?
• Usually assumed once the content for a referring
  expression has been determined, a standard
  realizer such as KPML (Bateman 1997) or SURGE
  (Elhaded and Robin 1997) can convert the meaning
  representation to natural language.

30
Summary

• Graph based easy to use
• Very useful for extending the algorithms
• Gradable properties
• Boolean descriptions

31
Questions?