NERALLT 2006 October 2627, 2006 Harvard University Cambridge, MA

1
NERALLT 2006October 26-27, 2006Harvard
UniversityCambridge, MA
Humanizing CALL
The use of Pedagogical Agents as Language Tutors
Roberto G. Pérez Galluccio Florida State
University
2
The Origin Software Agents
A software agent is an entity that performs
certain tasks with little or no human
intervention.

• "Autonomous agents are computational systems
  that inhabit some complex dynamic environment,
  sense and act autonomously in this environment,
  and by doing so realize a set of goals or tasks
  for which they are designed.
• 
  (Pattie Maes, 1995)

• "Intelligent agents are software entities that
  carry out some set of operations on behalf of a
  user or another program with some degree of
  independence or autonomy, and in so doing, employ
  some knowledge or representation of the user's
  goals or desires.
• 
  (IBM,
  1996)

3
Software Agents Characteristics
Different authors propose different defining
characteristics of software agents

• autonomous they are self-contained and can
  make decisions
  
• persistent they run continuously
• reactive they perceive changes in the
  environment and react
  
• accordingly
• intelligent they learn from experience and
  adapt to new situations
  
• socially capable they communicate with,
  collaborate with, and relate to other
  agents and/or elements in the environment
  
• personality-bearing they have a believable
  character that expresses emotions
  and displays behavior patterns
  

4
The Visual Interface Animated Agents
Visual interfaces add a communicative persona
through image, voice, and sometimes facial
expressions and gestures.

• They mediate between the computer user and the
  computer program.
  
• Their physical appearance can vary from
  geometric figures, to cartoon style, to human
  appearance.
  

Coney
Peedy
Professor
5
Early Implementation The Clippy Effect
- The Microsoft Office assistant (a.k.a Clippy)
reminds me of Jason from the Friday the 13th
slasher flicks -- he just won't die. (By Nicci
Noteboom, Managing Editor, author of Die Clippy,
Die, ZDNet AnchorDesk Stories, 9/29/98)
- But please let Super Dog live (Madeleine Robe
rts)
6
The Clippy Effect An Informal Survey
As part of his 1998 article, Noteboom
administered the following survey among his
readers
7
The Clippy Effect Survey Results
The results show the negative effect of Clippy on
readers (this survey was not statistically
reliable, though)
Lesson learned the Desktop Assistant interface
was a complex implementation that needed
comprehensive, interdisciplinary research.
8
Animated Agents Research Areas

• Agent role desktop assistants, information
  presenters, mentors/coaches, learning companions,
  tutor/instructor.
  
• Agent appearance anthropomorphism, gender,
  ethnicity, age, attire.
  
• Agent voice computer-generated, human,
  accentedness.
  
• Agent motivation believability, credibility,
  intelligence, expertise, feedback, motivational
  messages.
  
• Agent emotion facial expressions, voice
  inflections.
  
• Agent communication gestural behaviors, verbal
  behaviors, multimodality.
  

9
Animated Agents Role

• Presentation agent a guide and presenter of
  information in a web-based instructional
  environment (Andre, Rist, Muller, 1998).
  
• Tutoring Agent a tutor that provides
  assistance with computer literacy content,
  answering questions and providing feedback
  (Graesser et al., 1999).
• MIMIC System a constructivist pedagogical
  agent working together with a direct instruction
  agent. (Baylor, 2000).
  
• Socratic Agent a Socratic questioning approach
  used by an agent to guide learners through a
  computer-simulated disassembly process (Perez
  Solomon, 2005).

10
Animated Agents Appearance

• Human/Shape a human shape was compared to a
  geometrical shape on intelligence with HS (King
  Ohya, 1996).
  
• Caricature/Realistic a cartoon version of the
  agent was compared to a photo realistic version
  with RC on intelligence (Koda Maes, 1996).
  
• White/AAme-Male/Female white and
  African-American agents (both male and female)
  were rated on persona (Baylor, Shen, Huang,
  2003).
• ASax/Asian an Anglo-Saxon and an Asian
  character were compared while presenting sales
  information in a web-based environment (Nass,
  2005).

11
Animated Agents Voice

• Muted/Expressive silent agent were compared
  to expressive agents on engagingness with EM
  (Lester Stone, 1977).
  
• Strong/Calm computer-generated and human
  voices (strong and calm) were used with agents
  controlling for gender, gestures, affect, and
  speech (Kim, Baylor, Reed, 2003).
• Accented/Unaccented an Anglo-Saxon and an
  Asian character providing sales information in a
  web-based environment were presented with
  accented and unaccented voices (Nass, 2006).

12
Animated Agents Motivation

• Advocates of animated agents assume that such
  agents render a computer system more human-like,
  engaging, and motivating
  
• (Dehn Van Mulken, 2000)
• Motivation has been studied more as an outcome
  than a design feature. More research on what
  features make agents motivating is needed.
  
• Expert/Motivator/Mentor agent speech and
  expressiveness were changed to validate each
  agent as an expert, a mentor, or a motivator
  (Baylor Kim, 2003).

13
Animated Agents Emotion

• Emotion recognition computer recognition of
  and reaction to a basic emotions (Picard, 1998).
  
• Emotion/Mood/Personality voice and speech
  content were used to display different affective
  states in order to validate emotional tags
  (Descamps Ishizuka, 2001).
• Friendliness/Unfriendliness a multi-agent
  system modeled friendly and unfriendly behaviors
  to users to validate emotional expressions
  (Prendinger Ishizuka, 2001).
• Empathy/Apology an animated agent delivers
  apologetic or empathetic messages to mitigate
  user frustration during computer problems (Baylor
  et al., 2005).

14
Animated Agents Communication

• Functions and behaviors a full-bodied Embodied
  Conversational Agent (ECA) uses functions (open
  interaction, greet) to produce behaviors (look at
  user, smile, wave) in a real state consultation
  (Cassell, 2000).
• Expressivity and Animation an ECA uses
  expressivity specifications (tags, hierarchies)
  and expressivity animations (facial, gestural,
  and gaze motor planners) for believable,
  context-dependent behavior (Mancini, Hartmann,
  Pelechaud, 2004).
• Deictic Gesture, Locomotion, and Speech a
  robotic ECA uses ambiguity appraisal and gesture,
  locomotion, and utterance planning
  (Lester,Voerman, Towns, Callaway, 1997).

15
Some Examples...

• AutoTutor complex conversational system.
• Baldi grammar and vocabulary practice.
• Comi!Chat natural language parsing for basic
  conversation.
  
• Virtual Language Tutor (VLT) pronunciation
  practice.
  

16
AutoTutor Overview

• Text-based conversation with an animated agent
  tutoring computer literacy, physics, and research
  methods.

• Agent accepts long answers to open-ended
  questions.
  
• Dialogue moves selected from positive/negative
  feedback, prompts, and hints.
  
• It includes a Dialogue Advancer Network
  (selects next move according to students last
  utterance) and a Latent Semantic Analysis
  (matches student answers with expectations).
• Research group Tutoring Research Group,
  University of Memphis.

17
Baldi Overview

• Part of the CSLU Toolkit, which integrates speech
  recognition, text-to-speech synthesis, and
  animation.
  
• Used initially with hearing impaired children, it
  now can teach English grammar and vocabulary.

• Interaction occurs via speech, typed input, or
  mouse clicks.
  
• It includes a Speech Performance and Assessment
  (SPAM) database to capture and analyze data
  produce during interactions.
  
• Research group Center for Spoken Language
  Understanding, University of Colorado.

18
Comi!Chara

• The system provides English conversation training
  for Japanese speakers.
  
• Student-agent interactions take place in the form
  of role-play.

• Social Intelligence is implemented with a
  social filter that determines acceptable forms
  of interaction.
  
• This web-based system simulates a coffee-shop
  environment where learners communicate with the
  agent.
  
• Research group School of Engineering, University
  of Tokio.
  

19
Virtual Language Tutor (VLT)

• Talking head offers lip-sync, frowning, nodding,
  and eyebrow movement.
  
• A Dialogue Manager (DM) determines what the
  response is based on user input.

• A Pronunciation Analyzer (PA) provides visual
  information on correct pronunciation.
  
• Research group Center for Speech Technology,
  Stockholm University, Sweden.
  

20
Some Demos...

• Peedys Pizza Palace Microsoft Agent
  Technology, HTML, and JavaScript.
  
• Online Storyteller Microsoft Agent Technology,
  HTML, and JavaScript.
  
• Paco, the Spanish Tutor Living Actor
  Technology, Director, and Lingo.
  
• Spelling Tutor Microsoft Agent Technology and
  VisualBasic.
  

21
o