Dialogue Act Annotation

1
Dialogue Act Annotation

• Harry Bunt
• Tilburg University
• ISO-SIGSEM meeting Tilburg, January 9, 2007

2
Outline

• Introduction
• Theoretical background
• Empirical backing
• Data categories status and current issues
• Conclusions and future work

3
Introduction Motivation

• Assumption
• Dialogue acts are useful for the analysis of
  dialogues and for the design of dialogue systems,
  in particular for
• describing nonpropositional aspects of the
  meanings of dialogue utterances
• the design of dialogue management systems

4
Introduction context

• Part of preparatory work for establishing
  interoperable concepts for semantic annotation,
  in a joint effort of ISO and eContent project
  LIRICS
• Focus
• How to best support the annotation of
  dialogues with dialogue act information in an
  empirically and theoretically well-founded way.

5
Theoretical background

• The meaning of a dialogue utterance is the way in
  which the information state of a listener is
  changed by understanding the utterance (Dynamic
  Interpretation Theory, DIT).
• A dialogue act captures the speakers intention
  to change tye addressees information state plus
  the associated beliefs of the speaker about the
  addressee and about the world.
• Semantically, a dialogue can be viewed as a
  sequence of sets of dialogue acts.

6
Whats a Dialogue Act

• Definition (not just a speech act, used in
  dialogue)
• A dialogue act is a unit in the semantic
  description of communicative behaviour in
  dialogue, specifying how the behaviour is
  intended to change the information state of the
  addressee through his understanding of the
  behaviour.
• gt Appropriate for semantic annotation of
  dialogues

7
Whats a Dialogue Act

• Definition (not just a speech act, used in
  dialogue)
• A dialogue act is a unit in the semantic
  description of communicative behaviour in
  dialogue, specifying how the behaviour is
  intended to change the information state of the
  addressee through his understanding of the
  behaviour.
• Aspects of a dialogue act
• - communicative function
• - semantic content

8
Data categories for dialogue acts

• Definitions of communicative functions.
• Organization in the set of definitions?
• An unstructured list of CFs is unsatis-factory,
  since it doesnt show the relations between CFs
  and provides no guidance for adding CFs in a
  principled way.

9
Key to organization Multifunctionality

• U Can you tell me what time is the first
  train to the airport on Sunday?
• S On Sunday morning the first train to the
  airport is at 5.32
• U Thank you.

10
Multifunctionality

• U Can you tell me what time is the first
  train to the airport on Sunday?
• S On Sunday morning the first train to the
  airport is at 5.32
• U Thank you.
• - expression of thanks

11
Multifunctionality

• U Can you tell me what time is the first
  train to the airport on Sunday?
• S On Sunday morning the first train to the
  airport is at 5.32
• U Thank you.
• - expression of thanks
• - positive feedback
• (about understanding and acceptance)

12
Multifunctionality

• U Can you tell me what time is the first
  train to the airport on Sunday?
• S On Sunday morning the first train to the
  airport is at 5.32
• U Thank you.
• - expression of thanks
• - positive feedback
• (about understanding and acceptance)
• - indication of dialogue closure

13
Multifunctionality gt Multidimensional annotation

• Utterances have multiple functions gt
• annotation must be multidimensional, i.e.
• multiple annotation tags are required
• (or syntactically and semantically (!)
  complex tags -- cf. studies by Popescu-Belis)

14
Multidimensional annotation

• Usual informal notion of dimension
• Set of mutually exclusive tags, that can be
  assigned independent from the tags in other
  dimensions
• Not satisfactory... See multidimensional
  annotation according to DAMSL

15
Multidimensionality in DAMSL

• Dialogue Act Markup using Several Layers
• (Discourse Resource Initiative, 1996-1998)
• - 4 layers 1. Backward-Looking Functions (BLF)
• 2. Forward-Looking Functions
  (FLF)
• 3. Information Level
• 4. Information Status
• - 12 dimensions within BLF and FLF layers
• - Each utterance can have a FLF and a BLF
  function, or multiple functions

16
Dimensions in DAMSL

• FLF Dimensions
• Statement
• Info-request
• Influencing-addressee-future-action
• Committing-speaker-future-action
• Conventional Opening or Closing
• Explicit-performative
• Exclamation
• Other

17
Dimensions in DAMSL

• Definitions of some FLF Dimensions
• Statement Speaker makes a claim about the world
• Info-request Speaker requests Addressee to
  provide information
• Influencing-addressee-future-action
• Committing-speaker-future-action
• (.... 8)

18
Dimensions in DAMSL (2)

• BLF Dimensions
• Agreement
• Understanding
• Answer
• Information-relation

19
Dimensions in DAMSL (2)

• Definitions of some BLF Dimensions
• Agreement
• Understanding Utterances concerning the
  understanding between Speaker and Addressee
• Answer Speaker provides information requested by
  the Addressee
• Information-relation

20
Dimensions in DAMSL (3)

• Example
• A I hope youll have a good time!
• B Yeah, thanks.
• And youll be visiting friends in
  Italy.
• 3. A Thats right.

21
Dimensions in DAMSL (3)

• Example
• A I hope youll have a good time!
• B Yeah, thanks.
• And youll be visiting friends in
  Italy.
• 3. A Thats right.

22
Dimensions in DAMSL (3)

• Example
• A I hope youll have a good time!
• B Yeah, thanks.
• And youll be visiting friends in
  Italy.
• Statement? Question?
• 3. A Thats right.

23
Dimensions in DAMSL (3)

• Example
• A I hope youll have a good time!
• B Yeah, thanks.
• And youll be visiting friends in
  Italy.
• Statement? Question?
• DAMSL - statement
• - info-request
• ... but a speaker cannot at the same time state
  something and question its truth!

24
Dimensions in DAMSL (4)

• Conclusion
• Question and statement are mutually
  exclusive tags gt having them in different
  dimensions is wrong

25
Dimensions in DAMSL (4)

• Conclusion
• Question and statement are mutually
  exclusive tags gt having them in different
  dimensions is wrong
• Can they be alternatives in the same dimension?

26
Dimensions in DAMSL (4)

• Can question and answer be alternatives in
  the same dimension?
• Consider
• 1. S Did you ask me something?
• 2. U Can I change the contrast?

27
Dimensions in DAMSL (4)

• Can question and answer be alternatives in
  the same dimension?
• Consider
• 1. S Did you ask me something?
• 2. U Can I change the contrast?
• - question

28
Dimensions in DAMSL (4)

• Can question and answer be alternatives in
  the same dimension?
• Consider
• 1. S Did you ask me something?
• 2. U Can I change the contrast?
• - question
• - answer

29
Dimensions in DAMSL (4)

• Can question and answer be alternatives in
  the same dimension?
• Consider
• 1. S Did you ask me something?
• 2. U Can I change the contrast?
• - question
• - answer
• So question and answer can co-occur gt question
  and answer cannot be in the same dimension

30
Dimensions in DAMSL (4)

• Can question and answer be alternatives in
  the same dimension?
• 1. S Did you ask me something?
• 2. U Can I change the contrast?
• - question about what the task doman
• - answer about what the speaker said
• gt The task domain and what the speaker said are
• different dimensions an utterance can be
  a question in
• one dimension and an answer in another.

31
Dimensions in DAMSL (5)

• Example
• A And what possibilities do you have on
• Tursday?
• B Did you say Thursday?
• - function in (DAMSLs) Understanding
• dimension, but which?
• - Signal-understanding?
• - Signal-non-understanding?
• - Check (function in DAMSLs Info-
• request dimension)

32
Dimensions for dialogue acts

• Questions, answers, statements, requests,
  instructions, offers,... do not belong to any
  dimension they are general purpose functions
  they can be used in any dimension.
• DAMSL dimensions like Info-request and Answer
  are not proper dimensions.

33
General-purpose functions

• Example Informs in various dimensions
• The KL204 leaves at 12.30. Task/domain
• I see what you mean. Auto-feedback
• You misunderstood me. Allo-feedback
• I would like to hear Peters opinion. Turn
  management
• Im listening. Contact management
• ... I mean Toronto. Own communication management
• We should also discuss the agenda. Topic
  management
• I would like to ask you something. Discourse
  structuring
• Im very grateful for you help. Social obligation
  management

34
General-purpose functions

• Applicable in any dimension are
• Information-seeking functions
• WH-question, YN-question,
  Alternatives-question, Check,..
• Information-providing functions
• Inform, WH-Answer, YN-Answer, Confirmation,
  Disconfirmation, Agreement, Correction,..
• Commissive functions
• Offer, Promise, AcceptRequest,..
• Directive functions
• Instruct, Request, Suggest,..

35
Dialogue dimensions

• A dimension is an aspect of participating in a
  dialogue such that
• It can be addressed by means of dialogue acts
  that have a function for this purpose
• It can be addressed independently of other
  aspects.

36
Multidimensionality in dialogue

• Possible multifunctionality of dialogue
  utterances an utterance can have a function in
  more than one dimension
• An utterance has at most one function in any
  given dimension
• The function of an utterance in one dimension is
  in principle independent of those in other
  dimensions

37
Dimensions of communication

• Performing a certain task or activity through or
  with support from the communication
• Monitoring the interaction
• - providing and eliciting feedback
• - managing the turn-taking
• - managing the use of time
• - managing contact and attention
• - managing the opening and closing of (sub-)
• dialogues and thematic progression
• Deaing with social obligations greeting,
• thanking, apologizing,

38
Dialogue acts in different dimensions
39
Interaction Management acts

• Aspects of monitoring the interaction
• Feedback
• 1a. Auto-feedback about the speaker
• 1b. Allo-feedback about the addressee
• Turn management
• Time management
• Contact management
• Own communication management
• Partner communication management
• Topic management
• Dialogue structuring
• ...

40
Social dialogue acts

• Social obligations management
• Greeting
• Introducing oneself
• Apologizing
• Thanking
• Valediction
• ...

41
Dimension-specific functions

• Examples
• Auto-feedback OverallPositive Okay.
• Allo-feedback EvaluationElicitation Okay?
• Turn management TurnGiving Yes
• Time management Stalling Well, you
  know,..
• Contact mant ContactChecking
  Hello?
• Own comm. mant Self-correction I
  mean...
• Partner comm.man. Completion ...
  completion
• Topic management TopicShiftAnnounc. Something
  else.
• Dialogue structuring DA-announcement
  Question
• Social oblig. mant Valediction
  Bye
• Task/domain OpenMeeting I open this meeting

42
Multidimensional annotation scheme for dialogue
acts

• Two parts
• taxonomies of dimension-specific communicative
  functions for each dimension
• taxonomy of general-purpose functions

43
DA tag components

• ltGeneral-purpose CF, dimension namegt
• e.g. ltConfirm, Feedbackgt
• ltDimension-specific CFgt
• e.g. ltTurn givinggt

44
Data categories Current status

• Multidimensional DIT taxonomy, based on DAMSL,
  DIT, Verbmobil, TRAINS, TRINDI..., See
  http//ls0143.uvt.nl/dit/
• 35 general-purpose functions
• 52 dimension-specific functions, in 11 dimensions
• Subset of 45 core dialogue act types as LIRICS
  data categories

45
Communicative functions in DIT
46
Communicative functions in DIT (part)
47
Empirical backing

• Based on DAMSL, DIT, Verbmobil, Trindi,
  TRAINS,...
• Pilot test of DIT taxonomy (see
  http//ls0143.uvt.nl/dit/) for usability by
  annotators (Geertzen Bunt, SIGDIAL 2006
  encouraging results).
• More extensive testing planned, also for Italian
  (Claudia Sora, Pisa).
• Application of multidimensional taxonomy on
  multimodal AMI dialogues (Petukhova Bunt,
  IWCS-7) results comparing favourably with using
  AMI or DAMSL annotation schemes.
• Development of annotated dialogues using LIRICS
  data categories.

48
Software support and application

• Annotation tool for multidimensional DA
  annotation (Geertzen, IWCS-7).
• Dialogue act recognition and dialogue management
  software using the same concepts (in PARADIME
  dialogue manager, within STEVIN IMIX programme
  Keizer Bunt, SIGDIAL06).

49
Current Issues

• See Bunt Schiffrin, IWCS-7 paper
• Multidimensional segmentation
• Partial dialogue acts
• Negation and modification of communicative
  functions
• Empirical validation
• Manual annotation efforts
• Machine learning experiments (see also Petukhova
  Bunt, IWCS-7 paper)