Causes for Requesting Clarification in Dialogue

1
Causes for Requesting Clarification in Dialogue

• David Schlangen
• Universität Potsdam

2
Clarification Requests
3
Clarification Requests

• A I just saw your archenemy.

4
Clarification Requests

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

5
Clarification Requests

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

who/what did A say he saw? whats an
archenemy? who is A referring to? is A referring
to Peter as my archenemy? what did A say he
just did? what did A say? is A talking to me? is
A trying to scare me? whats A trying to do?
6
Clarification Requests

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

who/what did A say he saw? whats an
archenemy? who is A referring to? is A referring
to Peter as my archenemy? what did A say he
just did? what did A say? is A talking to me? is
A trying to scare me? whats A trying to do?
7
Clarification Requests

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

who/what did A say he saw? whats an
archenemy? who is A referring to? is A referring
to Peter as my archenemy? what did A say he
just did? what did A say? is A talking to me? is
A trying to scare me? whats A trying to do?
8
Clarification Requests

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

who/what did A say he saw? whats an
archenemy? who is A referring to? is A referring
to Peter as my archenemy? what did A say he
just did? what did A say? is A talking to me? is
A trying to scare me? whats A trying to do?
9
what this talk is not about

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

who/what did A say he saw? whats an
archenemy? who is A referring to? is A referring
to Peter as my archenemy? what did A say he
just did? what did A say? is A talking to me? is
A trying to scare me? whats A trying to do?
form
function
10
what this talk is not about

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

who/what did A say he saw? whats an
archenemy? who is A referring to? is A referring
to Peter as my archenemy? what did A say he
just did? what did A say? is A talking to me? is
A trying to scare me? whats A trying to do?
11
what this talk is about

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

who/what did A say he saw? whats an
archenemy? who is A referring to? is A referring
to Peter as my archenemy? what did A say he
just did? what did A say? is A talking to me? is
A trying to scare me? whats A trying to do?

• classification of problems that lead to a need
  for clarification
• computational model of utterance processing that
  can have these problems

12
Overview

• Three dimensions for classification
• level of action / extent / severity
• Dimension1 Clarks / Allwood SDRT
• Dimension 3 computational model
• Generalised confidence scores
• Further Work, Conclusions

13
Dimension 1 Level of Action

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

is A talking to me? who/what did A say he
saw? whats an archenemy? who is A referring
to? is A referring to Peter as my
archenemy? what did A say he just did? what did
A say? is A trying to scare me? whats A trying
to do?
14
Dimension 1 Level of Action

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

is A talking to me? who/what did A say he
saw? whats an archenemy? who is A referring
to? is A referring to Peter as my
archenemy? what did A say he just did? what did
A say? is A trying to scare me? whats A trying
to do?
15
Dimension 1 Level of Action

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

is A talking to me? who/what did A say he
saw? whats an archenemy? who is A referring
to? is A referring to Peter as my
archenemy? what did A say he just did? what did
A say? is A trying to scare me? whats A trying
to do?
16
Dimension 1 Level of Action

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

is A talking to me? who/what did A say he
saw? whats an archenemy? who is A referring
to? is A referring to Peter as my
archenemy? what did A say he just did? what did
A say? is A trying to scare me? whats A trying
to do?
17
Dimension 1 Level of Action

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

is A talking to me? who/what did A say he
saw? whats an archenemy? who is A referring
to? is A referring to Peter as my
archenemy? what did A say he just did? what did
A say? is A trying to scare me? whats A trying
to do?
18
Dimension 1 Level of Action

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

is A talking to me? who/what did A say he
saw? whats an archenemy? who is A referring
to? is A referring to Peter as my
archenemy? what did A say he just did? what did
A say? is A trying to scare me? whats A trying
to do?

• attention problem
• acoustic problem
• lexical problem
• (semantic) understanding
• problem
• (pragmatic) understanding
• problem

19
Dimension 2 Extent

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

is A talking to me? who/what did A say he
saw? whats an archenemy? who is A referring
to? is A referring to Peter as my
archenemy? what did A say he just did? what did
A say? is A trying to scare me? whats A trying
to do?

• problem caused by
• whole utterance
• or
• part of utterance

20
Dimension 2 Extent

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

is A talking to me? who/what did A say he
saw? whats an archenemy? who is A referring
to? is A referring to Peter as my
archenemy? what did A say he just did? what did
A say? is A trying to scare me? whats A trying
to do?

• problem caused by
• whole utterance
• or
• part of utterance

21
Dimension 2 Extent

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

is A talking to me? who/what did A say he
saw? whats an archenemy? who is A referring
to? is A referring to Peter as my
archenemy? what did A say he just did? what did
A say? is A trying to scare me? whats A trying
to do?

• problem caused by
• whole utterance
• or
• part of utterance

22
Dimension 3 Severity

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

is A talking to me? who/what did A say he
saw? whats an archenemy? who is A referring
to? is A referring to Peter as my
archenemy? what did A say he just did? what did
A say? is A trying to scare me? whats A trying
to do?
Does B maintain a hypothesis, or not?
23
Dimension 3 Severity

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

is A talking to me? who/what did A say he
saw? whats an archenemy? who is A referring
to? is A referring to Peter as my
archenemy? what did A say he just did? what did
A say? is A trying to scare me? whats A trying
to do?
Does B maintain a hypothesis, or not?
24
Dimension 3 Severity

• A I just saw your archenemy.
• My what?
• My arch-what?
• Who?
• Peter?
• B You just did what?
• Huh?
• Pardon?
• You trying to scare me?
• Why are you saying that?
• ...

is A talking to me? who/what did A say he
saw? whats an archenemy? who is A referring
to? is A referring to Peter as my
archenemy? what did A say he just did? what did
A say? is A trying to scare me? whats A trying
to do?
Does B maintain a hypothesis, or not?
25
Dim.1, current approaches

• inter alia (Larsson 2003 Gabsdil 2003)
• based on (Clark 1996) / (Allwood 1995)

Level Clark Allwood 4 proposal
con- reaction 2 evocative func. (prag.)
underst. prob. sideration
3 meaning under- understanding
(sem.) underst. prob. standing
2 presentation perception acoustic
prob. identification 1 execution
contact attention prob. attention
26
A fine-grained classification

• Clark / Allwood SDRT
• (Asher Lascarides 2003)

27
A fine-grained classification

• Clark / Allwood SDRT
• (Asher Lascarides 2003)

Level Description 4 recognising speakers
intentions 3c contextual relevance (
computing rhet. relation) 3b resolving
underspecification -i reference
-ii tense, scope, presuppositions, lex.
amb., etc. 3a parsing -i
determining a unique syntactic structure
-ii determining syntactic structure
-iii recognising all words 2 speech
recognition 1 establishing contact
A Max fell. John pushed him. B Are you saying
Max fell because John pushed him?
28
A fine-grained classification

• Clark / Allwood SDRT
• (Asher Lascarides 2003)

Level Description 4 recognising speakers
intentions 3c contextual relevance (
computing rhet. relation) 3b resolving
underspecification -i reference
-ii tense, scope, presuppositions, lex.
amb., etc. 3a parsing -i
determining a unique syntactic structure
-ii determining syntactic structure
-iii recognising all words 2 speech
recognition 1 establishing contact
A Every wire has to be connected to a power
source. B Each to a different one, or can it be
the same for all wires?
29
Dimension Extent

• e.g. "my arch-what?" vs. "huh?"
• (Clark Schaefer 1987)

State 4 3 2 1 0
I just saw your archenemy.
30
Dimension Extent

• e.g. "my arch-what?" vs. "huh?"
• (Clark Schaefer 1987)

State 4 3 2 1 0
I just saw your archenemy.
31
Dimension Severity

• is hypothesis maintained or not?
• ("Peter?" vs. "Who?")
• quality of hypothesis / confidence in it
• ... old news to people working on SDS confidence
  score of speech recognition

A I just met your archenemy.
reject confirm accept
explicit implicit
B Who? My archenemy?
32
Dimension Severity

• is hypothesis maintained or not?
• ("Peter?" vs. "Who?")
• quality of hypothesis / confidence in it
• ... old news to people working on SDS confidence
  score of speech recognition

A I just met your archenemy.
reject confirm accept
explicit implicit
e.g. (San-Segundo et al. 2001)
33
Dimension Severity

• is hypothesis maintained or not?
• ("Peter?" vs. "Who?")
• quality of hypothesis / confidence in it
• ... old news to people working on SDS confidence
  score of speech recognition

A I just met your archenemy.
reject confirm accept
explicit implicit
B Who? My archenemy?
34
Dimension Severity

• quality of hypothesis / confidence in it
• needed at all levels of processing!
• (here reference resolution)

A I just met your archenemy.
reject confirm accept
explicit implicit
B   Whos that? Peter?
35
Dimension Severity

• quality of hypothesis / confidence in it
• needed at all levels of processing!
• (here reference resolution)

A I just met your archenemy.
reject confirm accept
explicit implicit
B   Whos that? Peter?
36
Interdependencies

• not always in discrete state..

State 4 3 2 1 0
I just saw your archenemy.
37
Interdependencies

• snowballing wrong hypothesis at one level will
  lead to dubious hyps. at higher levels.

State 4 3 2 1 0
I just saw your archenemy.
38
Interdependencies

• SR confidence NLU features
• (Duff, Gates LuperFoy 1996), (Walker, Wright
  Langkilde 2000), (Gabsdil 2004)
• generalised to other levels
• (Paek Horvitz 1999)

39
A Computational Model

• RUDIclar, based on RUDI
• (Resolving Underspecification using Discourse
  Information, (Schlangen et al. 2000))
• RUDI
• implementation of SDRT
• computes rhetorical relations
• bridging relations

A Let's meet next week. B How about Monday?
Monday of next week.
Q-Elab
40
A Computational Model

• RUDIclar, based on RUDI
• (Resolving Underspecification using Discourse
  Information, (Schlangen et al. 2000))
• RUDI implements SDRT, computes
• rhetorical relations
• bridging relations

A Let's meet next week. B How about Monday?
Monday of next week.
Q-Elab
41
A Computational Model

• RUDIclar, based on RUDI
• (Resolving Underspecification using Discourse
  Information, (Schlangen et al. 2000))
• RUDI implements SDRT, computes
• rhetorical relations
• bridging relations

A Let's meet next week. B How about Monday?
Monday of next week.
Q-Elab
42
RUDI
attach.sites antecedent
speech acts
resolve under- specification
(LKB)
43
RUDI
attach.sites antecedent
speech acts
resolve under- specification
(LKB)
? Let's meet next week. x1 ? How about
Monday? x2
44
RUDI
attach.sites antecedent
speech acts
resolve under- specification
(LKB)
?
? Let's meet next week. x1 ? How about
Monday? x2
45
RUDI
attach.sites antecedent
speech acts
resolve under- specification
(LKB)
attach ? antec x1
?
? Let's meet next week. x1 ? How about
Monday? x2
46
RUDI
attach.sites antecedent
speech acts
resolve under- specification
(LKB)
attach ? antec x1
Q-Elab(?, ?) ??, ?? ? ?? gt Q-Elab(?, ?)
?
? Let's meet next week. x1 ? How about
Monday? x2
47
RUDI
attach.sites antecedent
speech acts
resolve under- specification
(LKB)
attach ? antec x1
temp_inc(t?,t?) Q-Elab(?, ?) ? temp_inc(t?,t?)
Q-Elab(?, ?) ??, ?? ? ?? gt Q-Elab(?, ?)
?
? Let's meet next week. x1 ? How about
Monday? x2
48
RUDIclar
attach.sites antecedent
speech acts
resolve under- specification
(LKB)
49
RUDIclar
attach.sites antecedent
speech acts
resolve under- specification
(LKB)
(Gabsdil Bos 2003)
50
RUDIclar
attach.sites antecedent
speech acts
resolve under- specification
(LKB)
51
RUDIclar
attach.sites antecedent
speech acts
resolve under- specification
score clarify
(LKB)
or ? accept( ) clar_acous( )
52
RUDIclar
attach.sites antecedent
speech acts
resolve under- specification
score clarify
(LKB)
or ? accept( ) clar_acous( )
53
RUDIclar
attach.sites antecedent
speech acts
resolve under- specification
score clarify
(LKB)
A We should arrange another meeting. B I am
free on Monday Sunday the 15th.
54
RUDIclar
attach.sites antecedent
speech acts
resolve under- specification
score clarify
(LKB)
A We should arrange another meeting. now
1st March 2004 B I am free on Monday Sunday
the 15th. Mon 15th 15th Mar.
'04 Sun 15th 15th Aug. '04
55
RUDIclar
attach.sites antecedent
speech acts
resolve under- specification
score clarify
(LKB)
A We should arrange another meeting. now
1st March 2004 B I am free on Monday Sunday
the 15th. Mon 15th 15th Mar.
'04 Sun 15th 15th Aug. '04
Rclar Monday the 15th?
56
RUDIclar
attach.sites antecedent
speech acts
resolve under- specification
score clarify
(LKB)
A We should arrange another meeting. now
1st February 2004 B I am free on Monday
Sunday the 15th. Mon 15th 15th
Mar. '04 Sun 15th 15th Feb. '04
57
RUDIclar
attach.sites antecedent
speech acts
resolve under- specification
score clarify
(LKB)
A We should arrange another meeting. now
1st February 2004 B I am free on Monday
Sunday the 15th. Mon 15th 15th
Mar. '04 Sun 15th 15th Feb. '04
Rclar Sorry, Monday the 15th or Sunday
the 15th?
58
RUDIclar
attach.sites antecedent
speech acts
resolve under- specification
score clarify
(LKB)
A Let's meet next month. B OK. How about
Monday? at least 4 possible
59
RUDIclar
attach.sites antecedent
speech acts
resolve under- specification
score clarify
(LKB)
A Let's meet next month. B OK. How about
Monday? at least 4 possible
Rclar Which Monday?
60
Conclusions

• multi-dimensional classification of problems that
  lead to a need for clarification
• level of action
• extent
• severity
• computational model of utterance processing that
  can have these problems

61
Conclusions

• multi-dimensional classification of problems that
  lead to a need for clarification
• level of action
• extent
• severity
• computational model of utterance processing that
  can have these problems

Peter finagled a raise
62
Conclusions

• multi-dimensional classification of problems that
  lead to a need for clarification
• level of action
• extent
• severity
• computational model of utterance processing that
  can have these problems, using a principled
  theory of dialogue semantics pragmatics

Peter finagled a raise
63
Conclusions

• multi-dimensional classification of problems that
  lead to a need for clarification
• level of action
• extent
• severity
• computational model of utterance processing that
  can have these problems, using a principled
  theory of dialogue semantics pragmatics

Peter finagled a raise
64
Further work

• learn best combination of conf. scores
• not only model interactions btw. levels, but also
  model costs (cost of misunderst. vs. cost of
  clarifying)
• "I'm busy on Monday, because ..."

reject confirm accept
explicit implicit
65
Further work

• learn best combination of conf. scores
• not only model interactions btw. levels, but also
  model costs (cost of misunderst. vs. cost of
  clarifying)
• "I'm busy on Monday, because ..."

reject confirm accept
explicit implicit
66
Further work

• learn best combination of conf. scores
• not only model interactions btw. levels, but also
  model costs (cost of misunderst. vs. cost of
  clarifying)
• "I'm busy on Monday, because ..."

reject confirm accept
explicit implicit
67
Further work

• learn best combination of conf. scores
• not only model interactions btw. levels, but also
  model costs (cost of misunderst. vs. cost of
  clarifying)
• evaluation!!
• (different grammar, more robust, ...)

68

• Thank you!
• Acknowledgements
• The Dialogue-Group at Potsdam
• Deutsche Forschungsgemeinschaft

69
A formal semantic approach

• Ginzburg et al.s analysis
• N.B. their focus is interpretation of
  clarification ellipsis!
• (Ginzburg Cooper 2001 Purver, Ginzburg
  Healey, 2001)
• A Did Bo leave? -- B Bo?
• clausal reading Are you asking whether Bo left?
• constituent reading Whos Bo?

70
A formal semantic approach

• Ginzburg et al.s analysis
• N.B. their focus is interpretation of
  clarification ellipsis!
• (Ginzburg Cooper 2001 Purver, Ginzburg
  Healey, 2001)
• A Did Bo leave? -- B Bo?
• clausal reading Are you asking whether Bo left?
• constituent reading Whos Bo?

(semantic) understanding no hypothesis part
of utterance
(semantic) understanding hypothesis part of
utterance
71
A formal semantic approach

• Ginzburg et al.s analysis
• N.B. their focus is interpretation of
  clarification ellipsis!
• (Ginzburg Cooper 2001 Purver, Ginzburg
  Healey, 2001)
• A Did Bo leave? -- B Who?
• clausal reading About whom are you asking if he
  left?
• constituent reading Whos Bo?

(semantic) understanding no hypothesis part
of utterance
(semantic) understanding no hypothesis part
of utterance
72
A formal semantic approach

• Ginzburg et al.s analysis
• N.B. their focus is interpretation of
  clarification ellipsis!
• (Ginzburg Cooper 2001 Purver, Ginzburg
  Healey, 2001)
• A Did Bo leave? -- B Who?
• clausal reading About whom are you asking if he
  left?
• constituent reading Whos Bo?

(semantic) understanding no hypothesis part
of utterance
(semantic) understanding no hypothesis part
of utterance
(acoustic) understanding no hypothesis part
of utterance