Completions and continuations in dialogue: a preliminary account

1
Completions and continuations in dialogue a
preliminary account

• Massimo Poesio (Uni Essex)
• Hannes Rieser (Uni Bielefeld)CATALOGBarcelona,
  July 2004

2
Sentence cooperations an example
Inst So, jetzt nimmst Du Well, now you
grasp Cnst eine Schraube a screw. Inst eine
lt-gt orangene mit einem Schlitz. an lt-gt orange
one with a slit Cnst Ja. Yes.
3
Sentence cooperations an informal definition
(Clark, 1996)

• SENTENCE COOPERATION At least two dialogue
  participants contribute to a sentence production
• COMPLETION sub-sentential structure continued by
  obligatory constituents
• CONTINUATION material added to already
  complete sentence

4
The significance of sentence cooperations

• Clear evidence that dialogue requires
  coordination at the sub-sentential level (see
  also Pickering and Garrod, in press)
• Provide insights into incrementality and
  compositionality issues
• A tool to investigate competing claims about
  coordination in dialogue
• Purely intentional models
• Pickering and Garrods IAM based on simpler
  alignment mechanisms.

5
Outline of the talk

• Sentence cooperations in the Bielefeld Toy Plane
  Corpus (BTPC)
• An introduction to
• PTT (Poesio and Traum 1997, 1998 Matheson et al,
  2000 Poesio, to appear)
• Intro to model of we-intentions we use, following
  Bratman (1992), Tuomela (2000) and Grosz and
  Kraus (1996)
• A PTT implementation of an intentional analysis
  of completions
• (May have time to sketch an IAM analysis)

6
The Bielefeld Toy Plane Corpus
7
The Bielefeld Toy Plane Corpus

• 22 video-filmed, speech recorded and transcribed
  dialogues
• two agents, Instructor and Constructor
• constructing a Baufix airplane
• different sight conditions total screen,
  half-screen, face to face
• 3675 contributions
• 160 sentence cooperations (4.34 )
• in most of them cooperation other-initiated (95)

8
Sentence cooperations in the BTPC (Skuplik, 1999)

• 126 sentence cooperations from the BTPC
• 54 completions (43)
• 72 continuations (57)
• Production of the completion / continuation 79
  Cnst, 21 Inst
• 84 of compl. / contin. accepted by previous
  speaker (41 implicitly)
• Release-turn signalled in 31 of cases

9
A few other observations

• Completions become more frequent as dialogue
  procedes (routinization?)

10
The example, revisited
Inst So, jetzt nimmst Du Well, now you
grasp Cnst eine Schraube a screw. Inst eine
lt-gt orangene mit einem Schlitz. an lt-gt orange
one with a slit Cnst Ja. Yes.
11
The example

• A CNST COMPLETION (70)
• By way of an OBLIGATORY NP (a screw ) (30)
• making up A SENTENCE when merged with Insts
  contribution (50)
• A REQUEST TO CHANGE SPEAKER signaled by prosodic
  means (lengthening of Du, level tone)

12
Outline of the talk

• The Bielefeld Toy Plane Corpus (BTPC)
• An introduction to
• PTT (Poesio, 1995 Poesio and Traum 1997, 1998
  Matheson et al, 2000 Poesio, to appear)
• Intro to model of we-intentions we use, following
  Bratman (1992), Tuomela (2000) and Grosz and
  Kraus (1996)
• A PTT implementation of an intentional analysis
  of completions

13
PTT

• A theory of semantics and interpretation in
  dialogue originally motivated by work on the
  TRAINS project
• Key characteristics
• Building on (Compositional) DRT (Muskens, 1996)
• Common ground as a record of the discourse
  situation (Barwise and Perry, 1983)
• An account of incremental semantic interpretation
• An account of GROUNDING
• So far, primarily concerned with aspects of
  dialogue driven by obligations

14
Common ground beyond assertion
A They have at their disposal enormous assets
// and their policy B //look can I just
come in on that// last year A //YES IN A MINUTE
IF YOU MAY AND WHEN IM FINISHED // then
youll know B // yes IM SO SORRY (Coulthard
1977)
15
Common ground beyond assertion
B Go to Elmhurst, pass the courthouse and go to
Elmhurst and then to Elmhurst, uh north.A
mm hum.B Towards Riverton, till you come to
that Avila HallA Oh yesB Dju know where
that//is?A //uh huhA Oh surelyB Avilla Hall
on the corner of Bor//donA //uh huhB Well
there, on Bordon you turn back to town,
left. (George Psathas, "Direction-giving in
Interaction," in Boden and Zimmerman, ed.)
16
From DRT to PTT

• a. A There is an engine at Avon.
• B It is hooked to a boxcar
• DRT x,w,y,u,s,s engine(x), Avon(w), s
  at(x,w), boxcar(y), shooked-to(u,y),
  ux

17
Common ground and discourse situation in PTT
ce1,ce2,K1,K2 K1x,w,s engine(x),
Avon(w), s at(x,w), ce1 assert(A,B,K1)
K2y,z,s boxcar(y), shooked-to(z,y),
zx, ce2 assert(B,A,K2)
18
Locutionary acts in the common ground
The fact that a speaker is speaking, saying the
words he is saying in the way he is saying them,
is a fact that is usually accessible to everyone
present. Such observed facts can be expected to
change the presumed common background knowledge
of the speaker and his audience in the same way
that any obviously observable change in the
physical surroundings of the conversation will
change the presumed common knowledge.
(Stalnaker, Assertion, p. 323)
19
The time-order of sentence processing

• GARDEN-PATH phenomena shows that parsing is
  INCREMENTAL (Bever, 1974 Frazier, 1987)
• Marslen-Wilson 1973, 1975 semantic information
  ALSO accessed immediately
• Swinney, 1979 lexical access incremental
• Just and Carpenter,1980 IMMEDIACY HYPOTHESIS
  (Every word encountered should be processed to
  the deepest level possible before the eye moves
  on to the next word)
• Eye-tracking work (Tanenhaus et al, 1995,
  tomorrow) really fine-grained incrementality

20
Alignment at all levels Pickering Garrod
21
Clarification questions (Ginzburg and Cooper,
Purver and Ginzburg)
A Did Bo leave?B BO? A Bo Smith.B Yes,
half an hour ago. Matthew It wasnt all that
bad. At least the pool was
clean.Lara MR POOL?Matthew The
pool.Lara Oh. ltlaughgt
(What is the intended content of your utterance
Bo?)
(Did you utter the words Mr. Pool?)
22
Micro conversational events (Poesio, 1995)
boxcar ? uuutter(A,boxcar), Noun(u),
sem(u)?x boxcar(x),
SYN INFO (NEXT)
umm? u,ce u utter(A,umm), ce
keep-turn(A), generate(u,ce)
23
MCEs in the example dialogue
mce1,ce1 mce1utter(Inst,so"), Adv(mce1),
ce1take-turn(Inst), generate(mce1,ce1)
mce2,ce2 mce2utter(Inst,jetzt"), Adv(mce2),
ce2keep-turn(Inst),
generate(mce2,ce2)
mce3 mce3utter(Inst,"nimmst"), Verb(mce3),
sem(mce3) ?Q?x(Q(?xe e grasp(x,
x)))
mce4 mce4utter(Inst,"Du"), Pro(mce4),
sem(mce4) ?P.P (you)
mce5 mce5utter(Cnst,"eine"), Det(mce5),
sem(mce5) ?P?P(y P(y) P(y))
mce6 mce6utter(Cnst,"Schraube"), Noun(mce6),
sem(mce6) ?v( screw(v)
24
Syntactic interpretation with MCEs (Poesio, 1996)
MCE2 CE2
MCE3
MCE4
25
Syntactic Interpretation with MCEs, II
MCE2 CE2
MCE3
MCE4
26
Semantic interpretation and compositionality
BINARY SEMANTIC COMPOSITION
27
Intentions and obligations
OBLIGATIONS o oOblCnst
(address(Cnst, ce1))
INTENTIONS i iIntInstCnst
(join(Cnst, wing1,fuselage1))
INTENTIONAL STRUCTURE GroszSidner-like
sp(i1) i2 dom(i1) i2
28
Grounding

• As in proposals such as Clark and Schaefer (1989)
  and Traum (1994), establishment of common ground
  (G) modeled in terms of CONTRIBUTIONS, or
  DISCOURSE UNITS, that may be ACKNOWLEDGED or
  REPAIRED

29
Discourse Units and Grounding Acts
30
MCEs in the example dialogue (simplified)
mce1,ce1 mce1utter(Inst,so"), Adv(mce1),
ce1take-turn(Inst), generate(mce1,ce1)
mce1 mce1utter(Inst,"nimmst"), Verb(mce1),
sem(mce1) ?Q?x(Q(?xe e grasp(x,
x))) mce2 mce2utter(Inst,"Du"),
Pro(mce2), sem(mce2) ?P.P
(you)mce3 mce3utter(Inst,"eine"),
Det(mce3), sem(mce3) ?P?P(y
P(y) P(y))mce4 mce4 utter(Inst,
orangene), Adj(mce4), sem(mce4)
?P?z( orange(z) P(z))mce5
mce5utter(Const,"Schraube"), Noun(mce5),
sem(mce5) ?v( screw(v)
31
The dynamics of discourse situations, I
AX-SP-SDRT Let p, r be core speech acts
K1 and K2 DRSs. Then ? ce1,ce2,A,B,C,D,K1,K2
,f,g ce1 p(A,B,K1), ce2 r(C,D,K2),
sp(ce2)ce1 (f,g) ? (? i, j,k
ce1,ce2 (f,i) ? K1(i,j) ? K2(j,k) ?
sp(ce2)ce1(k,g))
32
The dynamics of discourse situations, II
AX-DOM-MCE ? u, u, u, f, g, u ?
u, u ? u, sem(u) ???1..?m???,
sem(u) ???1..?n???, sem(u)
???1..?p???(f,g) ? (?
i,j, x1 . xm, ?( x1 )(x2). (xm)(i,j) ?
(? k,l,y1 yn, z1.. zp, i ? k ? l ? j
? ? (y1) (yn) (i,j) ? ?(
z1)( zp) (k,l)))
33
Anaphoric expressions and resource situations
the ? uuutter(A,the), Det(u),
sem(u)? K ? P ? P x
x ?y. K P(x) P(x) sem(the boxcar)
? ? K ? P x
x ?y. K boxcar(x) P(x)
34
What prompts the completion? Two accounts

• Intentional account
• Need to explain why help
• Alignment account
• What representation is aligned?

HR
35
Outline of the talk

• The Bielefeld Toy Plane Corpus (BTPC)
• An introduction to
• PTT (Poesio and Traum 1997, 1998 Matheson et al,
  2000 Poesio, to appear)
• Intro to model of we-intentions we use, following
  Bratman (1992), Tuomela (2000) and Grosz and
  Kraus (1996)
• A PTT implementation of an intentional analysis
  of completions

36
Public and private (partial) plans

• Inst has a fully specified plan for building the
  toy-airplane (drawing or model)
• The public plan among Inst and Cnst is usually
  underspecified, but gets more refined throughout
  the construction dialogue
• Besides the shared partial plan, the agents have
  private plans which overlap to some extent with
  the shared plan.
• The difference between the public plan and the
  private plans leads to discrepancies and
  negotiations

37
The partial shared plan before the example
 
     
Join wing and fuselage
38
We-intentions for shared cooperative activity

• Tuomelas (2000) modification of Bratmans 1993
  definition of SCA, adapted
• Inst and Cnst WE-INTEND that Cnst join wing and
  fuselage is equivalent to
• It is Insts and Cnsts mutual knowledge that
• Inst intends that Cnst join wingfuselage
  because Cnst intends that Cnst join wingfuselage
• and
• Cnst intends that Cnst join wingfuselage
  because Inst intends that Cnst join wingfuselage

39
Tuomelas definition of we-intention, formal
IntInstCnst(join(Cnst, WF)) ? MK((IntInst
join(Cnst, WF)) /r
IntCnst(join(Cnst, WF))) and
(IntCnstjoin(Cnst, WF)) /r
IntInst(join(Cnst, WF)))) (where /r is the
reason relation, which is factual)
40
Outline of the talk

• The Bielefeld Toy Plane Corpus (BTPC)
• An introduction to
• PTT (Poesio and Traum 1997, 1998 Matheson et al,
  2000 Poesio, to appear)
• Intro to model of we-intentions we use, following
  Bratman (1992), Tuomela (2000) and Grosz and
  Kraus (1996)
• A PTT implementation of an intentional analysis
  of completions

41
The situation before example 1
MP
42
The partial shared plan before the example
 
     
Join wing and fuselage
43
Reaching the intention to perform a directive
1. (partial) we-intention to join i
iIntInstCnst (join(Cnst, wing1,fuselage1))

• 2. intertwined discourse / domain
  planjoin(Cnst,Obj1,Obj2) gtAshMorr b c d
  e f,
• (b) 1. direct(Inst, Cnst, grasp(Cnst,
  Bolt)), 2. grasp(Cnst, Bolt), 3. tell(Cnst,
  Inst, grasp(Cnst, Bolt ))
• 1.direct(Inst, Cnst, grasp(Cnst, Nut)), 2.
  grasp(Cnst, Nut), 3. tell(Cnst, Inst,
  grasp(Cnst, Nut))
• put through (e) fasten.. (f) .feedback.

44
Deciding to perform a directive, II
3. Distributivity of we-intention ?
4. Achieve partial we-intention to perform
directivei1 i1IntInstCnst (K1,ce1K1
e,xbolt(x), egrasp(Cnst,x)
ce1direct(Inst, Cnst, K1))
5. Achieve partial Inst intention ( Cnst
intention)i2 i2IntInst (K1,ce1K1 e,
xbolt(x), egrasp(Cnst,x) ce1direct(Inst,
Cnst, K1))
45
Insts private plan
Assemble toy airplane
 
     
Join wing and fuselage
Assemble fuselage
Assemble wing
get 5h bar
get 3h bar
get 7h bar1
get 7h bar2
join
join
alignwingfuselage
getbolt

getbolt
getbolt
orange-bolt-with-slit
46
Cnsts private plan
bolt
47
Planning the directive
6. arrive at more specific intention (evidence
subsequent repair) i3 i3IntInst (K2,ce2K2
e, xbolt(x),xorange-slit-bolt,
egrasp(Const,x) ce2direct(Inst, Cnst,
K2))
7. develop plan to perform utterance that
generates directive i4 i4IntInst (u1.1
utterance(u1.1), sem(u1.1) K2,
generates(u1.1,ce2))
48
Micro-plan
8. plan to perform utterance in terms of
MCEs i5a i5aIntInst (u1.2 u1.2
sotake-turn)
i5b i5bIntInst (u1.3 u1.3jetztkeep-tur
n)
i5c i5cIntInst (u1.4 .. u1.7 S(u1.1) ,
u1.4nimmstV, u1.5DuNP,
u1.7VP, NP(u1.6), u1.5 ?u1.1, u1.7 ?u1.1,
u1.4 ? u1.7, u1.6 ? u1.7
9. lengthening signals problem - Inst doesnt
necessarily know which bolts are unused ?
possibly does not know how to refer to bolt
(NP type / content)
49
Possible motivations for the completion

• Interpret lengthening as request to continue
• obl(Cnst, cont(DU))
• Interpret lengthening as request for
  acknowledgment (also standard PTT)
• obl(Cnst, ack(DU))
• Cooperativeness
• Blurting out

50
A cooperativeness analysis
10. Cnst acquires intention to turn the directive
in a joint action (cfr. Tuomelas unrequired
contributory actions) i6 i6IntCnst
(K3,ce3K3 e,xbolt(x), egrasp(Const,x)
ce3direct(InstCnst, Cnst, K3))Derivation of
10 not axiomatized by Tuomela, but we assume here
is the result of an intention to help.
11. Cnst produces plan to perform action to
generate directive analogous to Insts, but
content (partial) K3 i7 i7IntCnst (u1.1a
utterance(u1.1a), sem(u1.1a) K3,
generates(u1.1a,ce3))
51
Further specification not possible
52
Micro-plan for Cnsts completion
12. most of actions in plan already performed by
Inst Cnst plans missing action i8 i8IntCnst
( K1.1d u1.6eine SchraubeNP,
K1.1dx1bolt(x1) sem(u1.6) K1.1d )
(Same action would be planned to continue
contribution and to acknowledge) 13. Instructor
begins repair due to his private plan
53
An alternative analysis the IAM model

• Successful dialogue involves the development of
  aligned representations at all levels
• Aligned representations the result of priming
  mechanisms at every level of linguistic
  representation
• Mental state reasoning an option but not basic

54
Alignment at all levels
55
An IAM-analysis of the BTPC example

• What leads Cnst to produce eine Schraube?
• What is a situation model in this domain?
• the key dimensions encoded in situation models
  are SPACE, TIME, CAUSALITY, INTENTIONALITY, and
  REFERENCE to the MAIN INDIVIDUALS (p. 7)

56
Two possible views of the situation model in BTPC
domain

• The partial shared plan (cfr. earlier analysis)
• Cfr. mention of intentionality
• Would make the IAM model much closer to the model
  presented earlier
• The state of Cnsts plane assembly

57
The state of assembly route

• Problems with this type of dialogues
• Situation models are clearly NOT aligned (they
  will only be so at the END of the conversation)
• Cannot assume implicit common ground
• Need to extend model to deal with directives

58
A proposal

• Based on the notion of priming
• Extend the notion of routine to non-linguistic
  actions
• During the dialogue, an AGGREGATE FORMATION
  ROUTINE gets established

59
Development of the AF routine
JOIN(SIDE-RUDDER, FUSELAGE)
JOIN(TAIL, FUSELAGE)
JOIN(TAIL, FUSELAGE)
60
The routine

• PARAMETERS
• 2 constituents to be joined
• through a port
• a fixing mechanism going through this port
  (currently underspecified)
• ROUTINE FOR AGGREGATE FORMATION
• align material to be joined (requires port
  identif.)
• obtain fixing mechanism
• put fixing mechanism through port
• fasten fixing mechanism
• yields aggregate

61
Generation ofeine Schraube

• Based not on global shared plan, but on local
  instantiation of AF-routine, primed in context
• Only one parameter is still underspecified the
  bolt
• Local execution of routine at the obtain fixing
  mechanism point
• Syntactic structure to realize action presumably
  also available through routine

62
Open problems with this account

• Some motivation for completion needed
• More in general, how is helping done?
• (Normal cooperativity axioms are based on beliefs
  intentions.)
• Also need to say something more about the choice
  of that particular realization
• Economy principle?

63
Preliminary conclusions

• PTT provides the technical tools to formalize a
  crucial feature of sentence cooperations
  coordination at the micro conversational event
  level
• Mind-reading always difficult, but Tuomelas
  theory of we-intention goes some way towards
  formalizing one of the possible motivations for
  completions, in terms of help
• A preliminary investigation of the alignment
  route also possible