IUT de Montreuil

1
Emotion in Interaction Embodied Conversational
Agents
Catherine Pelachaud

• IUT de Montreuil
• Université Paris 8

2
Emotion in Interaction Humaine

• Three domains of study
• Perception domain how certain aspects related to
  perception may influence agents actions
• Interaction domain how to create relations
  between users and agents how the agent can
  provide feedback
• Generation domain how to show expressive
  behaviours consistently and naturally across
  modalities

3
Perception and Attention

• Non obtrusive agent based on the interest shown
  by the addressee decides
• when to start a communication
• how long to maintain it
• when to finish it
• Perception of attentive and emotional behaviors
• where to look at
• how to interpret others behaviors

4
Perception and Attention

• Endow an ECA with the capabilities to
• have a visual attention and perception model
• show attention and interest
• attract the attention of others

5
Interaction

• Interaction two or more parties exchange
  messages.
• Interaction is by no means a one way
  communication channel between parties.
• Within an interaction, parties take turns in
  playing the roles of the speaker and of the
  addressee.

6
Interaction

• Speaker and addressee adapt their behaviors to
  each other
• Speaker monitors addressees attention and
  interest in what he has to say
• Addressee selects feedback behaviors to show the
  speaker that he is paying attention

7
Interaction

• Agent as adressee ought to have the capabilities
• attention pay attention to the signals produced
  by speaker to perceive, process and memorize them
• perception of signals
• comprehension understand meaning attached to
  signals
• internal reaction the comprehension of the
  meaning may create cognitive and emotional
  reaction
• decision decision to communication the internal
  reaction
• generation display behaviors

8
Interaction

• Providing feedback is a fundamental aspect of
  engagement
• Backchannel signals of
• 1-     perception
• 2-     attention -- interest
• 3-     understanding
• 4-     attitude belief (T/F) like (T/F)
• 5- acceptance
• positive/negative
• any combination of the above
• pay attention but not understand understand but
  not believe

9
Expressive Behaviors

• Behaviors encode
• content information (the What is communicating)
• expressive information (the How it is
  communicating)
• highly synchronized
• multimodal

10
Expressive Behaviors

• Basic emotions
• Anger, fear, sadness, happiness, disgust,
  surprise
• Universally recognized (Ekman)
• Basic emotion family of related states (Ekman
  75)
• Real life emotions are often complex and involve
  several emotions

11
Expressive Behaviors

• Blends may occur either as
• superposition of emotions
• masking of one emotion by another emotion
• quick succession of different emotions
• suppression / overacting of an emotion
• Superposition of 2 emotions evaluation of an
  event can happen under different angles
• Mask an emotion by another one consideration of
  social context

12
Examples

• Disappointment Masked by Joy
• Superposition of Sadness and Joy

Disappointment
joy
Disappointment masked by joy
Sadness
Joy
Sadness and Joy
13
Callas

• Creation of a virtual agent able to
• perceive and be attentive
• be aware of and adapt to the surroundings
  (physical and social context)
• interact with users/other agents
• communicate and listen
• show real emotions
• use expressive gestures

14
Callas

• Extension and refinement of the results obtained
  in Humaine
• Real-Time integrated system
• Integration of virtual and real perception models
• Real situations