Enhancing Information Awareness Through Speech Induced Anthropomorphism

1
Enhancing Information Awareness Through Speech
Induced Anthropomorphism

• PhD Examination
• Nuno Jorge Gonçalves de Magalhães Ribeiro
• External examiner Professor Philip Barker
• Internal examiner Dr. Alistair Edwards
• Department of Computer Science
• University of York
• 06 February 2002

2
Structure of the Examination(follow
presentation, 4 blocks, discussion at the end of
each block)
3
Motivation

• Problems with current interfaces
• for information awareness
• (lack of humanized interaction)
• Finding visually hidden information is
  distracting (e.g. information hidden behind
  opened windows, dynamic information)
• Lack of semantics in alerting cues (e.g. blinking
  icons, sounds), or visually intrusive displays
  (e.g. windows that obscure info. being accessed)
• Lack of awareness and co-ordination between
  high-priority and unimportant alerts to be
  reported (e.g. junk e-mail, status info.)
• Lack of a unified interruption mechanism for
  notifications (imminent and just occurred events)
  and reminders (past, but relevant, events)

4
How to Humanize Information Awareness?

• One way is to exploit speech to deliver alerting
  messages (notifications and reminders)
• BUT
• (because listening to speech cues anthropomorphic
  feelings)
• The reception of speech demands that the
  interface reflects human social norms
• at the delivery level
• design should humanize the delivery of
  speech-based messages
• at the content level
• design should humanize the contents of
  speech-based messages
• 3 questions Why speech? How to humanize the
  delivery? How to humanize the content?

5
Why Speech?

• There is an urgent need to find out the
  situations in which
• speech enhances the user interface and the
  situations in
• which it is detrimental to the user interface
• This thesis suggests that speech can be used with
  advantages to
• enable less disruptive interruptions
• convey more meaningful notifications
• design of an integrated notification mechanism

6
How to Humanize Message Delivery?(in order to
reinforce social acceptability of the speech
interface)

• This thesis suggests that there are 3 fundamental
  design aspects
• Attentive interruptions (minimise user
  disruption)
• When to interrupt precedence lt prioritiy and
  urgency,
• How to interrupt get attention, modulate the
  audio channel when it is already being used
• Linguistic variation (avoid irritating repetitive
  utterances while maintaining the semantics)
• Sources lexical, syntactical, phonetic
• Politeness (attenuate irritating effect of
  interruptions)
• Simple courtesy (e.g. Excuse me, Im sorry)
• Design based on politeness strategies (e.g.
  direct, positive, negative)

7
How to Humanize Message Contents?(in order to
enhance the awareness of information that is
hidden from view)

• This thesis suggests that there are 2 fundamental
  design aspects
• Short alerting messages just relevant content
  (because speech is a serial channel)
• Context-based cues extracted from items
  (because context plays a major role in recalling
  information)
• Forward Cues Kind, Time, Quantity, People,
  Nature, Description, Place

8
End of block 1

• Questions
• Discussion

9
Humanized Experimental EnvironmentIn presenting
the design, the thesis illustrates

• How to split a complex environment into a number
  of separate software agents that perform specific
  monitoring tasks.
• How to develop a software architecture that is
  based on a collection of specialised software
  agents.
• How to design the internal architecture of each
  software agent, in such a way that it collects
  and conveys context information.
• How to communicate collected information between
  the software agents comprising the environment.
• How to transform collected information into
  appropriate spoken messages to be delivered to
  the user.
• How to deliver the spoken information following
  typical social norms.

10
Experimental Environment Architecture
11
Steps in the Humanization Process

1. Deciding when to interrupt (hum.
   delivery) (specialised agents I.L. lt urgency,
   priority, user presence)
2. Extracting context cues (hum. content) (specialis
   ed agents forward cues)
3. Including linguistic variation (hum. content and
   delivery) (speaking agent templates for lexical
   and syntactical variations prosody markers for
   phonetic variation)
4. Including politeness (hum. delivery)
   (speaking agent attention-grabbers, templates
   for linguistic style)
5. Deciding how to interrupt (hum.
   delivery) (speaking agent attention-grabbing,
   audio channel modulation)

12
Experimental Environment Prototype

• Windows environment (familiar, limitations)
• Implements specialised agents
• Environmental awareness, context extraction,
  message composition
• E-mail (MAPI), Diary (TXT), Printer (Default
  printer queue)
• Implements communication protocol
• Comprises purposefully designed message types and
  contents
• Relies on transport mechanism based on Dynamic
  Data Exchange (DDE)
• Implements speaking agent
• MS Agent client
• Creates spoken reminders and notifications
  including linguistic variation and politeness
• Interrupts by grabbing the user attention, fading
  audio channel
• Uses TTS synthesiser and Speech Recognition engine

13
End of block 2

• Questions
• Discussion

14
Empirical Study Objectives (1/3)

• Assess users perceptions of the adequacy of
  anthropomorphic behaviours

15
Empirical Study Objectives (2/3)

• Assess users perceptions of the usefulness and
  humanization of the system
• Useful helpful, convenient, comfortable.
• Humanized pleasant, unobtrusive, satisfying,
  reliable, efficient, likeable.

16
Empirical Study Objectives (3/3)

• Assess differences in those users perceptions
  according to psychological (MBTI) and other
  personal characteristics
• Heuristic (N, F) versus Analytic (S, T)
  psychological types
• Intuition (N) / Sensing (S) dimension gt
  preferences for info. Acquisition (awareness -
  first step in the acquisition of info.)
• Feeling (F) / Thinking (T) dimension gt
  preferences for info. processing
• Gender, O.S. proficiency, familiarity with TTS

17
Empirical Study Evaluation Method

• Quasi-experimental model
• Extensive laboratory test that simulated closely
  a real world situation.
• 14 specific research questions
• IV subject variables DV subjective users
  opinions and perceptions
• Participants
• 15 participants selected from the population of
  the C.S. Department
• Measurement of variables
• Attitude measurement through the use of
  questionnaires
• 5-point scales 1Strongly Disagree, 3Neither
  disagree nor agree, 5Strongly agree
• 7-point scales 1Not at all, 4 Undecided, 7
  Very much so
• 7-point semantic differential scales e.g. 1
  unpleasant, 7 pleasant

18
Empirical Study Experimental Design and Procedure
19
Empirical Study Summary of Results (1/2)

• Speech is useful to convey information that is
  hidden from view (but content must be humanized
  through context cues)
• The prototype was perceived as supporting the
  awareness of visually hidden information.
• It was considered particularly useful by
  heuristic information gatherers speech-based
  alerts allowed participants to appreciate the
  immediate relevance of an informational event,
  that is the relationship and connection between
  the alerting information and their personal
  interests and activities.
• Information awareness is an arena where using
  speech is a valid and beneficial approach
• Participants indicated that speech-based
  interruptions were not overly disruptive and
  allowed them to decide when to deal with
  information.

20
Empirical Study Summary of Results (2/2)

• The anthropomorphic behaviours were considered to
  be important and adequate to reinforce the social
  acceptability of the speech-based system
• Attentive interurptions, linguistic variation and
  politeness combine to avoid socially incorrect
  behaviours, such as respectively unnecessary
  interruptions, irritating repetitions and
  rudeness.
• Urgency and priority-based interruptions were
  perceived as relevant.
• Attention-grabbing and audio modulation were
  perceived as natural (or expected) behaviours.
• When messages were varied they did not irritate
  participants. They were found so natural that
  some people did not even notice the variation.
• Polite behaviour is appropriate to avoid
  rudeness when getting a persons attention, but
  it invokes the perception of a personality and
  thus must be used in a way so that it suggests a
  consistent personality.

21
End of block 3

• Questions
• Discussion

22
Summary of the Research Achievements

• How to reinforce the social acceptability of a
  speaking system
• Humanize interruptions, vary language, use
  politeness
• How to enhance information awareness
• Support speech-based alerts including contextual
  cues
• Experimental environment design and prototype
• Agent-based approach, protocol, delivery of
  spoken alerts
• Empirical study
• Validated the anthropomorphic speech-based
  approach for info. awareness
• Practical guidelines for the design of
  speech-based systems for information awareness.

23
Contributions (1/4)

• The thesis contributes to understand the
  importance of, and how to exploit, speech to
  enhance the awareness of personal information
  that is hidden from view in a desktop-based
  environment
• How to humanize the delivery of speech-based
  alerts
• How to humanize the contents of speech-based
  alerts

24
Contributions (2/4)

• The thesis contributes with a unique experimental
  environment that integrates three different
  anthropomorphic characteristics for the delivery
• Attentive interruptions, linguistic variation,
  politeness
• They reinforce the social acceptability of the
  system - it was unexpectedly well accepted by
  users.

25
Contributions (3/4)

• The thesis contributes to the research community
  in that it provides an extensible agent-based
  architecture for an experimental environment
• Can be extended with additional specialised
  agents
• Can accommodate the incorporation of additional
  anthropomorphic behaviours (e.g. gender,
  personality, emotion)
• Can be used to support controlled experiments
  with larger and more heterogeneous user groups

26
Contributions (4/4)

• The thesis contributes to expand current
  knowledge about how to evaluate users
  perceptions of anthropomorphic systems that
  employ speech, and suggests a range of
  appropriate subjective measures
• It measured the adequacy of speech-induced
  anthropomorphic behaviours through specific
  questionnaires
• It measured the perceptions of usefulness and
  humanization through careful designed
  questionnaires