Socially Intelligent Robots

1
Socially Intelligent Robots

• Cynthia Breazeal
• MIT Media Lab
• Robotic Life Group

2
Robots turn 85 years old Posted May 31st 2006
1138PM by Ryan Block Filed under Robots
Dear Robots, We're very sorry. It appears we
missed your 85th birthday two days ago -- the
anniversary of which is marked by the date Czech
writer Karel Capek debuted his play R.U.R.
(Rossum's Universal Robots) to its first audience
in Prague. Yes, we know the concept of the
automaton dates back much further, but we think
it's well agreed upon that Capek's play marks the
robot's entry into mass consciousness (as well as
marking the first use of the word "robot"). No
matter, we're just saying happy birthday, robots
-- not because we fear you'll one day you'll
subsume us in some dystopian nightmare of
artificial intelligence gone terribly wrong, but
because from Asimov to AIBO, from Roomba to
Ri-Man, from QRIO to ASIMO, we just love ya. So
happy birthday, happy birthday, happy birthday,
robots, and when the day of reckoning comes,
please remember Engadget and its readers are
your friends. All our love, Engadget
3

• Robots have
• explored ocean depths,
• mapped subterranean mines,
• rescued natural disaster victims,
• assisted surgeons with operations,
• driven autonomously across the desert,
• And even been to Mars

4
Whats Next?

• The next big frontiersociety at large

5
Everyday Life with People and Robots and its
implication for design
6
People and Robots

• Robots are not perceived as pure tools or
  appliances, but often as social actors -- over a
  wide range of morphologies and behaviors

7
Robots Evoke Human Social Responses
The Kismet Effect
New Scientist, 2005
8
Newsmaker My friend, the robot CNET news.com,
May 24, 2006
The PackBots have almost become members of
military units, Angle said, recalling an incident
when a U.S. soldier begged iRobot to repair his
unit's robot, which they had dubbed Scooby Doo.
"Please fix Scooby Doo because he saved my life,"
was the soldier's plea, Angle told the Future in
Review conference last week in Coronado, Calif.
For many reasons, people bond with robots in a
way they don't bond with their lawn mowers,
televisions or regular vacuum cleaners. At some
point, this could help solve the looming health
care problem caused by an enormous generation of
aging people. Not only could robots make sure
they take their medicine and watch for early
warning signs of distress, but they could also
provide a companion for lonely people and extend
their independence.
9
Social Robots Socio-emotive Factors
Social as relationship
Future applications require robots to address
Interactive Toys
Social as entertainment
the socio-emotive and psychological aspects of
people, in long-term relations
BANDAI elder toys
Professional Service Robots
Social as interface
NEC babysitters
OMRON pets
10
HRI, An Emerging Discipline
An important goal of Human-Robot Interaction
(HRI) is synergy of the human-robot system.
Robots bring their own abilities that complement
human strengths. It is not about equivalence
(replacement), but compatibility with a typical
human partner
11
Four Cornerstones of Social Robotics in HRI
Interdependence
Transparent Communication
Lasting Relationship
Teamwork
Social Learning
Social Intelligence
Cognitive Compatibility
Perspective Taking
User Studies, Psychology Social Development
12
Todays Focus
Robots, like humans, should leverage the social
and environmental constraints in the real world
to foster learning new skills and knowledge from
anyone.
13
Personalization agents, Adaptive user
interfaces Lashkari, Metral, Maes,
Collaborative Interface Agents, AAAI 1994 E.
Horovitz et al., The Lumiere project, UAI 1998
Active Learning, Learning with Queries Cohn,
Ghahramani, Jordan, Active learning with
statistical models, 1995 Cohn et al.,
Semi-supervised clustering with user feedback,
2003
Learning by Demonstration, Programming by
Example Voyles, Khosla, Programming robotic
agents by demonstration, 1998 Lieberman, Your
Wish is my Command, 2001 A. Billard, Special
Issue of RAS on Robot Programming by
Demonstration, 2006 Learning by Imitation S.
Schaal review in TICS 1999 K. Dautenhahn C.
Nehaniv, Imitation in Animals and Artifacts, 2002
Animal training techniques Stern, Frank,
Resner, Virtual Petz, Agents 1998 Blumberg
et al. Integrated learning for interactive
characters, SIGGRAPH 2002 Kaplan et al., Robot
clicker training, RAS 2002 Reinforcement
Learning with humans Isbell et al. Cobot a
social reinforcement learning agent, UAI
1998 Evans, Varieties of Learning, AI Game
Programming Wisdom, 2002 Clouse, Utgoff,
Teaching a Reinforcement Learner, ICML 1992
and many more
14
How Do Ordinary People Teach a RL Agent?
Most people dont have experience with Machine
Learning techniques, they have a lifetime of
experience with social learning interactions that
they bring to the table. We emphasize the need
to consider and design to support the ways that
people naturally approach teaching. And then
design algorithms and systems that take better
advantage of this
15
Experiments inSophies Kitchen

• A computer game - players teach a virtual robot
  to bake a cake, by sending various messages with
  a mouse interface.

Sophie learns via Q-Learning 30 steps 10,000
states 2-7 actions/state Allows us to run many
subjects on-line
16
Experiments inSophies Kitchen

• A computer game - players teach a virtual robot
  to bake a cake, by sending various messages with
  a mouse interface.

An object specific reward is about a particular
part of the world
17
Initial Experiment
Thomaz Breazeal RO-MAN 2006

• 18 people trained Sophie
• They are given a description of the cake task,
  and told they cant do actions but can help
  Sophie by sending FEEDBACK messages with the
  mouse
• System logs time of state changes, agent actions,
  and any human feedback. We analyze games logs to
  understand peoples teaching behavior

18
Findings Guidance

• People tried to use the object specific rewards
  as FUTURE directed guidance.

19

• Many object rewards not about the last object used

Each players Object Rewards about last object
20

• Almost everyone gave rewards to the bowl or tray
  sitting empty on the shelf...a guidance reward.

Number of People
Zero rewards to Empty Bowl
At least 1 reward to Empty Bowl
21
Findings People Adapt Teaching to their Mental
Model of Sophie

• People gave more rewards after realizing their
  feedback made a difference
• Interpreted Sophies behavior as being a staged
  learner
• Adapted their teaching strategy accordingly

22
human rewards agent actions
23
Guidance
Initial Experiment

Transparency
Asymmetry
24
Using Guidance in Sophies Kitchen
Interactive Q-Learning Algorithm, baseline system
slight delay to animate act and receive human
reward

25
Using Guidance in Sophies Kitchen
26
GuidanceExperiment
Thomaz Breazeal, AAAI 2006

• Hypothesis Non-expert teachers can use guidance
  to improve agents performance
• 27 subjects trained Sophie in two groups
• Using feedback only
• Using both feedback and guidance
• Again, system logs game play and logs are
  analyzed to understand teaching behavior

27
Effects of Guidance

gtgt
only

• 1-tailed T-tests show logs in guidance condition
  are significantly better than non-guidance

feedback only guidance feedback effect size
Number of Trials 28.5 14.6 49
Number of Actions 816.4 368 55
Number of Failures 18.89 11.8 38
Number Fails before 1st Goal 18.7 11 41
Number Unique States Visited 124.44 62.7 50
28
Guidance
Initial Experiment

Asymmetry
29
Transparency
Teachers structure the environment and the task
to help a learner succeed.
Learners contribute by revealing internal state
helping the teacher maintain a mental model to
make guidance more appropriate.

• How can machine learners be Transparent?

30
Sophies Gaze Behavior
Interactive Q-Learning Algorithm modified
to incorporate Guidance
31
Sophies Gaze Behavior
32
TransparencyExperiment
Thomaz et al., ICDL 2006

• 52 subjects trained Sophie in an online version
  
• Feedback and guidance, no gaze
• Feedback and guidance, Sophie gazing
• Hypothesis
• Learners can help shape their learning
  environment by communicating aspects of the
  internal process -- gaze will improve the humans
  guidance instruction

33
Sophies Gaze Behavior
Results Sophies gaze significantly improves the
guidance received - more when uncertainty high
and less when uncertainty is low.
Uncertainty high 3 or more choices
Uncertainty low 3 or less
34
Lessons

• People bring their own teaching and learning
  experience to the task
• Social factors of guidance and transparency
• Collaborative process between teacher and learner
  improves performance
• Agent can use transparency cues to improve its
  own learning environment by helping teacher form
  a better mental model
• Adding gaze significantly improves the humans
  Guidance

35
Summary