Grounding%20in%20Conversational%20Systems

1
Grounding in Conversational Systems

• Dan Bohus
• January 2003
• Dialogs on Dialogs Reading Group
• Carnegie Mellon University

2
Overview

• Early grounding theories
• Discourse Contributions - Clark Schaefer
• Conversational acts Traum
• A Computational Framework (Horvitz, Paek)
• Principles
• Systems
• Grounding in RavenClaw

3
Clark Schaefer

• In discourse, humans collaborate to
  establish/maintain mutual ground
• Discourse is structured in contributions
• Contribution Presentation Acceptance
• Grounding criterion
• A and B mutually believe that the partners have
  understood what A said to a criterion sufficient
  for the current purposes

4
Clark Schaefer (2)

• Evidence of understanding
• Display
• Demonstration
• Acknowledgement
• Initiating the next relevant contribution
• Continued attention
• Display/Demonstration order challenged

5
Clark Schaefer (3)

• Infinite recursion avoided by Strength of
  Evidence Principle
• 4 possible states of non-understading
• L did not notice Ss utterance
• L notices it but does not hear it correctly
• L hears it correctly but does not understand it
• L understands it

6
Traum

• Conversational acts, extension of speech acts
  theory
• Turn-taking
• Grounding
• Initiate, Continue, Cancel, ReqAck, Ack,
  ReqRepair, Repair
• Core speech acts
• Argumentational acts
• Eliminates infinite recursion by ack.s dont
  need further ack.s

7
Traum (2)

• Later work, the following computational model is
  introduced
• Finally, Brennan ( Clark)
• another computational formulation
• studies the different types of grounding
  behaviors in different media

8
Criticisms

• These models are by-and-large descriptive.
• Cant be used to determine whats the next best
  thing to do to achieve the grounding criterion.
• Moreover, they dont describe quantitatively/make
  use of the uncertainty in contributions
• Are insensitive to differences in channels,
  content, populations, etc
• Cannot be used for guidance
• Decision Theory to the rescue ! ! !

9
Decision Theory

• Action under uncertainty
• Given a set of states S s, evidence e, and a
  set of actions A a, if
• P(se) is a probabilistic model of the state
  conditioned on the evidence
• U(a,s) the utility of taking action a when in
  state s.
• Take action that maximizes the expected utility
• EU(ae) ?S U(a,s)p(se)

10
Conversation under Uncertainty

• Conversation action under uncertainty
• Example I want to fly to Pittsburgh
• States grounded, not_grounded
• Unaccessible, but describable by a probabilistic
  model
• P(g e) P(Pittsburgh e) confidence annot.
• Actions explicit_confirm, implicit_confirm,
  continue_dialog
• Utilities
• U(ec,g) lt U(ic,g) lt U(cd,g)
• U(ec,ng) gt U(ic,ng) gt U(cd,ng)

11
I want to fly to Pittsburgh (2)

• States
• NotGrounded (ng)
• Grounded (g)
• Actions
• ExplicitConfirm (ec)
• ImplicitConfirm (ic)
• ContinueDialog (cd)
• Utilities
• U(ec,g) lt U(ic,g) lt U(cd,g)
• U(ec,ng) gt U(ic,ng) gt U(cd,ng)

ng
g
12
Overview

• Early grounding theories
• Discourse Contributions - Clark Schaefer
• Conversational acts Traum
• A Computational Framework (Horvitz, Paek)
• Principles
• Systems
• DeepListener
• Bayesian Receptionist (Quartet architecture)
• Presenter (Quartet architecture)
• Grounding in RavenClaw

13
DeepListener - Domain

• Domain
• Provides spoken command-and-control functionality
  for LookOut
• Respond to offers of assistance (Yes/No)
• Small domain, but illustrates the core ideas very
  well

14
DeepListener - States

• States 5 possible intentions of the user
• Acknowledgement
• Negation
• Reflection
• Unrecognized Signal
• No Signal
• State model P(SE) temporal bayesian network.
• E Users Actions, Content, ASR Results and
  Reliability at time -1

15
DeepListener - Actions

• Actions
• Execute the service
• Repeat
• Note a hesitation and try again
• Was that meant for me?
• Try to get the users attention
• Apologize for the interruption and forego the
  service
• Troubleshoot the overall dialog

16
DeepListener - Utilities

• Utilities
• Elicited through psychological experiments
• Elicited through slidebars
• Works when you have 2, 3 grounding actions, and a
  clear/small state-space design, but how about
  when the problem gets more complex ?
• Example (paper)

17
Bayesian Receptionist, Presenter

• Bayesian Receptionist performs the tasks of a
  receptionist at a MS front desk
• Im here to see Rashid
• Bathroom?
• Beam me to 25 please
• 32 goals
• Presenter command control interface to
  PowerPoint presentations.
• Both based on Quartet architecture

18
Quartet

• Uses DT and BN to ensure grounding at 4 different
  levels
• Signal
• Channel
• Intention
• Conversation
• The actual DM task is encapsulated in the same
  framework at the Intention level
• Different domains different intention levels

19
Quartet Signal Channel

• At each level infer a distribution over possible
  states. Key variables
• Signal level signal identified (low/med/hi)
• Channel level users focus of attention
• Maintenance module integrates Signal Channel
  levels -gt Maintenance Status
• Channel x Signal NoChannel, NoSignal,
  ChannelButNoSignal, SignalButNoChannel, Signal

20
Quartet Intention Level

• Domain is mostly goal inference
• Hierarchical decomposition on levels, where lower
  levels refine the goals into more specific needs
• Use BN to model p(goal e) at leach level
• Psychological studies to identify key variables
  and utilities
• Visual cues
• Linguistic variables both syntactic and semantic

21
Quartet Intention Level

• To move between levels, compare probability of
  goal to
• p-progress
• (above do it)
• p-guess
• (above search confirmation)
• (below search more info via VOI)
• p-backtrack
• used on return nodes
• Use Value-Of-Information analysis to infer whats
  the variable that should be queried next.

22
Comments on Intention level

• What is the size of the learning problem? (How
  many BN needed?) How much data needed for
  training?
• Not very clear
• how to deal with attribute/value, with rich
  ranges (e.g. which bus station ?)
• how to deal with basically richer dialog
  mechanisms (beyond CC applications)
• focus shifts, mixed initiative
• providing help

23
Quartet Conversation Level

• See image. Use Intention and Maintenance Status
  to infer
• Grounding diagnoses mutual understanding
• Okay, ChannelFailure, IntentionFailure,
  ConversationFailure
• Activity goal measures if the user is engaged or
  not in an activity with the system
• Compute expected utility for each action
  (utilities elicited through psychological studies)

24
Bayesian Receptionist, Presenter

• Runtime behavior (section 3)
• Presenter
• The Signal Channel level allow a uniform
  treatment in the same framework of continuous
  listening
• Experiments show that its better than random,
  but significantly less so than humans
• But then again, the experiments were not very
  fair, being performed only at that level (i.e. no
  engaging in dialog allowed)

25
My Research

• Deal with misunderstandings
• Use probabilistic modeling and decision theory to
  make grounding decisions (but not task decisions)
• I want a room tomorrow morning (0.73)
• States time correctly understood/not
• Grounding Actions no_action, expl_conf,
  impl_conf, reject
• Utilities try to learn them by relating the
  actions to an overall dialog/grounding metric

26
RavenClaw Dialog Task / Grounding
27
States and Actions

• Actions Strategies.xls
• States (have to keep it small!!!)
• Single state-space model
• What are the variables? Which are observable and
  which are stochastically modeled?
• Multiple state-space models
• First 5 strategies state amount of grounding
  on each concept
• What should state be for the rest? What are the
  indicators? Which are fully observable and which
  are not?
• How to combine decisions from different spaces

28
Utilities

• Learn them! How ?
• Idea 1 POMDPs, maybe this small they are
  tractable
• Idea 2 Regression to some overall dialog metric
• What should that be?
• (hmm) amount of non-null grounding actions taken