Towards Robot Theatre

1
Towards Robot Theatre
Marek Perkowski Department of Electrical and
Computer Engineering, Portland State University,
Portland, Oregon, 97207-0751
2

• Humanoid Robots and Robot Toys

3
Talking Robots

• Many talking robots exist, but they are still
  very primitive
• Work with elderly and disabled
• Actors for robot theatre, agents for
  advertisement, education and entertainment.
• Designing inexpensive natural size humanoid
  caricature and realistic robot heads

Dog.com from Japan
We concentrate on Machine Learning techniques
used to teach robots behaviors, natural language
dialogs and facial gestures.
Work in progress
4
Robot with a Personality?

• Future robots will interact closely with
  non-sophisticated users, children and elderly, so
  the question arises, how they should look like?
• If human face for a robot, then what kind of a
  face?
• Handsome or average, realistic or simplified,
  normal size or enlarged?

• The famous example of a robot head
• is Kismet from MIT.

• Why is Kismet so successful?
• We believe that a robot that will interact with
  humans should have some kind of personality and
  Kismet so far is the only robot with
  personality.

5
Robot face should be friendly and funny

• The Muppets of Jim Henson are hard to match
  examples of puppet artistry and animation
  perfection.

• We are interested in robots personality as
  expressed by its
• behavior,
• facial gestures,
• emotions,
• learned speech patterns.

6
Behavior, Dialog and Learning
Words communicate only about 35 of the
information transmitted from a sender to a
receiver in a human-to-human communication. The
remaining information is included in
para-language. Emotions, thoughts, decision and
intentions of a speaker can be recognized earlier
than they are verbalized. NASA

• Robot activity as a mapping of the sensed
  environment and internal states to behaviors and
  new internal states (emotions, energy levels,
  etc).
• Our goal is to uniformly integrate verbal and
  non-verbal robot behaviors.

7
(No Transcript)
8
Moritas Theory
9

• Our Base Model and Designs

10
Neck and upper body movement generation
11
Robot Head Construction, 1999
High school summer camps, hobby roboticists,
undergraduates
Furby head with new control
Jonas
We built and animated various kinds of humanoid
heads with from 4 to 20 DOF, looking for comical
and entertaining values.
12

• Mister Butcher

Latex skin from Hollywood
4 degree of freedom neck
13
Robot Head Construction, 2000
Skeleton
Alien
We use inexpensive servos from Hitec and Futaba,
plastic, playwood and aluminum. The robots are
either PC-interfaced, use simple
micro-controllers such as Basic Stamp, or are
radio controlled from a PC or by the user.
14
Technical Construction, 2001 Details
Marvin the Crazy Robot
Adam
15

• Virginia Woolf

2001
heads equipped with microphones, USB cameras,
sonars and CDS light sensors
16
2002
Max

• BUG (Big Ugly Robot)

Image processing and pattern recognition uses
software developed at PSU, CMU and Intel (public
domain software available on WWW). Software is
in Visual C, Visual Basic, Lisp and Prolog.
17
Visual Feedback and Learning based on
Constructive Induction
2002
Uland Wong, 17 years old
18
Professor Perky
2002, Japan
Professor Perky with automated speech recognition
(ASR) and text-to-speech (TTS) capabilities

• We compared several commercial speech systems
  from Microsoft, Sensory and Fonix.
• Based on experiences in highly noisy environments
  and with a variety of speakers, we selected Fonix
  for both ASR and TTS for Professor Perky and
  Maria robots.
• We use microphone array from Andrea
  Electronics.

1 dollar latex skin from China
19
Maria, 2002/2003
20 DOF
20
Construction details of Maria
location of head servos
skull
location of controlling rods
location of remote servos
Custom designed skin
21
Animation of eyes and eyelids
22
Cynthia, 2004, June
23
Currently the hands are not moveable.
We have a separate hand design project.
24
Software/Hardware Architecture

• Network- 10 processors, ultimately 100
  processors.
• Robotics Processors. ACS 16
• Speech cards on Intel grant
• More cameras
• Tracking in all robots.
• Robotic languages Alice and Cyc-like
  technologies.

25
Face detection localizes the person and is the
first step for feature and face recognition.
Acquiring information about the human face
detection and recognition, speech recognition and
sensors.
26
Face features recognition and visualization.
27
Use of Multiple-Valued (five-valued) variables
Smile, Mouth_Open and Eye_Brow_Raise for facial
feature and face recognition.
28
HAHOE KAIST ROBOT THEATRE, KOREA, SUMMER 2004
Czy znacie dobra sztuke dla teatru robotow?
Sonbi, the Confucian Scholar
Paekchong, the bad butcher
29
Editing movements
30
Yangban the Aristocrat and Pune his concubine
The Narrator
31
The Narrator
32
(No Transcript)
33
We base all our robots on inexpensive
radio-controlled servo technology.
34
We are familiar with latex and polyester
technologies for faces
Martin Lukac and Jeff Allen wait for your help,
whether you want to program, design behaviors,
add muscles, improve vision, etc.
35
New Silicone Skins
36
A simplified diagram of software explaining the
principle of using machine learning based on
constructive induction to create new interaction
modes of a human and a robot.
37

• Probabilistic and Finite State Machines

38
Probabilistic State Machines to describe emotions
you are beautiful / Thanks for a compliment
P1
Happy state
you are blonde! / I am not an idiot
P0.3
you are blonde! / Do you suggest I am an
idiot?
Unhappy state
P0.7
Ironic state
39
Facial Behaviors of Maria
Do I look like younger than twenty three?
Maria asks
Response

• no

• no

• yes

0.7
0.3
Maria smiles
Maria frowns
40
Probabilistic Grammars for performances
Speak Professor Perky, blinks eyes twice
P0.1
Speak Professor Perky
Where?
P0.3
Who?
P0.5
P0.5
P0.5
Speak in some location, smiles broadly
Speak In the classroom, shakes head
Speak Doctor Lee
What?
P0.1
P0.1
P0.1
Speak Was singing and dancing
P0.1
Speak Was drinking wine
.
41
Human-controlled modes of dialog/interaction
Human teaches
Thanks, I have a lesson
Hello Maria
Lesson finished
Robot performs
Robot asks
Question
Stop performance
Questioning finished
Command finished
Thanks, I have a question
Thanks, I have a command
Human commands
Human asks
42

• Dialog and Robots Knowledge

43
Robot-Receptionist Initiated Conversation
Human
Robot
What can I do for you?
Robot asks
This represents operation mode
44
Robot-Receptionist Initiated Conversation
Human
Robot
What can I do for you?
I would like to order a table for two
Robot asks
45
Robot-Receptionist Initiated Conversation
Human
Robot
Smoking or non-smoking?
Robot asks
46
Robot-Receptionist Initiated Conversation
Human
Robot
Smoking or non-smoking?
I do not understand
Robot asks
47
Robot-Receptionist Initiated Conversation
Human
Robot
Do you want a table in a smoking or non-smoking
section of the restaurant? Non-smoking section
is near the terrace.
Robot asks
48
Robot-Receptionist Initiated Conversation
Human
Robot
Do you want a table in a smoking or non-smoking
section of the restaurant? Non-smoking section
is near the terrace.
A table near the terrace, please
Robot asks
49
Human-Initiated Conversation
Human
Robot
Hello Maria
initialization
Robot asks
50
Human-Initiated Conversation
Human
Robot
Hello Maria
What can I do for you?
Robot asks
51
Human-Asking
Human
Robot
Question
Question
Robot asks
Human asks
52
Human-Asking
Human
Robot
Question
Yes, you ask a question.
Human asks
53
Human-Asking
Human
Robot
What book wrote Lee?
Yes, you ask a question.
Human asks
54
Human-Asking
Human
Robot
What book wrote Lee?
I have no sure information.
Human asks
55
Human-Asking
Human
Robot
Try to guess.
I have no sure information.
Human asks
56
Human-Asking
Human
Robot
Try to guess.
Lee wrote book Flowers.
Human asks
57
Human-Asking
Human
Robot
This is not true.
Lee wrote book Flowers.
Human asks
58
Human-Teaching
Human ends questioning
Human
Robot
Questioning finished
Human asks
Human teaches
Questioning finished
Robot asks
Thanks, I have a lesson
59
Human-Teaching
Robot enters asking mode
Human
Robot
Questioning finished
What can I do for you?
Human asks
Human teaches
Questioning finished
Robot asks
Thanks, I have a lesson
60
Human-Teaching
Human starts teaching
Human
Robot
Thanks, I have a lesson
What can I do for you?
Human asks
Human teaches
Questioning finished
Robot asks
Thanks, I have a lesson
61
Human-Teaching
Human
Robot
Thanks, I have a lesson
Yes
Human teaches
62
Human-Teaching
Human
Robot
I give you question-answer pattern
Yes
Human teaches
63
Human-Teaching
Human
Robot
Question pattern What book Smith wrote?
Yes
Human teaches
64
Human-Teaching
Human
Robot
Answer pattern Smith wrote book Automata
Theory
Yes
Human teaches
65
Human-Teaching
Human
Robot
Checking question What book wrote Smith?
Yes
Human teaches
66
Human-Teaching
Human
Robot
Checking question What book wrote Smith?
Smith wrote book Automata Theory
Human teaches
67
Human-Teaching
Human
Robot
I give you question-answer pattern
Yes
Human teaches
68
Human-Teaching
Human
Robot
Question pattern Where is room of Lee?
Yes
Human teaches
69
Human-Teaching
Human
Robot
Answer pattern Lee is in room 332
Yes
Human teaches
70
Human-Checking what robot learned
Human
Robot
Lesson finished
Human teaches
Lesson finished
Question
Robot asks
Human asks
71
Human-Checking what robot learned
Human
Robot
Lesson finished
What can I do for you?
Human teaches
Lesson finished
Question
Robot asks
Human asks
72
Human-Checking what robot learned
Human
Robot
Question
What can I do for you?
Human teaches
Lesson finished
Question
Robot asks
Human asks
73
Human-Asking
Human
Robot
Question
Yes, you ask a question.
Human teaches
Lesson finished
Question
Robot asks
Human asks
74
Human-Asking
Human
Robot
What book wrote Lee?
Yes, you ask a question.
Human asks
75
Human-Asking
Human
Robot
What book wrote Lee?
I have no sure information.
Human asks
76
Human-Asking
Human
Robot
Try to guess.
I have no sure information.
Human asks
77
Human-Asking
Human
Robot
Try to guess.
Lee wrote book Automata Theory
Observe that robot found similarity between Smith
and Lee and generalized (incorrectly)
Human asks
78
Behavior, Dialog and Learning

• The dialog/behavior has the following components
  
• (1) Eliza-like natural language dialogs based on
  pattern matching and limited parsing.
• Commercial products like Memoni, Dog.Com, Heart,
  Alice, and Doctor all use this technology, very
  successfully for instance Alice program won the
  2001 Turing competition.
• This is a conversational part of the robot
  brain, based on pattern-matching, parsing and
  black-board principles.
• It is also a kind of operating system of the
  robot, which supervises other subroutines.

79
Behavior, Dialog and Learning

• (2) Subroutines with logical data base and
  natural language parsing (CHAT).
• This is the logical part of the brain used to
  find connections between places, timings and all
  kind of logical and relational reasonings, such
  as answering questions about Japanese geography.
• (3) Use of generalization and analogy in dialog
  on many levels.
• Random and intentional linking of spoken
  language, sound effects and facial gestures.
• Use of Constructive Induction approach to help
  generalization, analogy reasoning and
  probabilistic generations in verbal and
  non-verbal dialog, like learning when to smile or
  turn the head off the partner.

80
Behavior, Dialog and Learning

• (4) Model of the robot, model of the user,
  scenario of the situation, history of the dialog,
  all used in the conversation.
• (5) Use of word spotting in speech recognition
  rather than single word or continuous speech
  recognition.
• (6) Continuous speech recognition (Microsoft)
• (7) Avoidance of I do not know, I do not
  understand answers from the robot.
• Our robot will have always something to say, in
  the worst case, over-generalized, with not valid
  analogies or even nonsensical and random.

81

• Constructive Induction

82
(No Transcript)
83
Example Age Recognition
Name (examples) Age (output) d Smile Height Hair Color Hair Color
Joan Kid (0) a(3) b(0) c(0) c(0)
Mike Teenager (1) a(2) b(1) c(1) c(1)
Peter Mid-age (2) a(1) b(2) c(2)  
Frank Old (3) a(0) b(3) c(3) c(3)
Examples of data for learning, four people, given
to the system
84
Example Age Recognition
Smile - a Very often often moderately rarely
Values 3 2 1 0
Height - b Very Tall Tall Middle Short
Values 3 2 1 0
Color - c Grey Black Brown Blonde
Values 3 2 1 0
Encoding of features, values of multiple-valued
variables
85
Multi-valued Map for Data
Groups show a simple induction from the Data
ab\ c 0 1 2 3
00 - - - -
01 - - - 3
02 - - - -
03 - - - -
10 - - - -
11 - - - -
12 - - 2 -
13 - - - -
20 - - - -
21 - 1 - -
22 - - - -
23 - - - -
30 0 - - -
31 - - - -
32 - - - -
33 - - - -
ab\ c 0 1 2 3
00 - - - -
01 - - - 3
02 - - - -
03 - - - -
10 - - - -
11 - - - -
12 - - 2 -
13 - - - -
20 - - - -
21 - 1 - -
22 - - - -
23 - - - -
30 0 - - -
31 - - - -
32 - - - -
33 - - - -
d F( a, b, c )
86
Old people smile rarely
Groups show a simple induction from the Data
blonde hair
Grey hair
ab\ c 0 1 2 3
00 - - - -
01 - - - 3
02 - - - -
03 - - - -
10 - - - -
11 - - - -
12 - - 2 -
13 - - - -
20 - - - -
21 - 1 - -
22 - - - -
23 - - - -
30 0 - - -
31 - - - -
32 - - - -
33 - - - -
Middle-age people smile moderately
Teenagers smile often
Children smile very often
87
Another example teaching movements
Input variables
Output variables
88
Generalization of the Ashenhurst-Curtis
decomposition model
89
This kind of tables known from Rough Sets,
Decision Trees, etc Data Mining
90
Original table
Second variant
First variant of decomposition
At every step many decompositions exist
Decomposition is hierarchical
Which decomposition is better?
91
Constructive Induction Technical Details

• U. Wong and M. Perkowski, A New Approach to
  Robots Imitation of Behaviors by Decomposition
  of Multiple-Valued Relations, Proc. 5th Intern.
  Workshop on Boolean Problems, Freiberg, Germany,
  Sept. 19-20, 2002, pp. 265-270.
• A. Mishchenko, B. Steinbach and M. Perkowski, An
  Algorithm for Bi-Decomposition of Logic
  Functions, Proc. DAC 2001, June 18-22, Las Vegas,
  pp. 103-108.
• A. Mishchenko, B. Steinbach and M. Perkowski,
  Bi-Decomposition of Multi-Valued Relations, Proc.
  10th IWLS, pp. 35-40, Granlibakken, CA, June
  12-15, 2001. IEEE Computer Society and ACM SIGDA.
  

92
Constructive Induction

• Decision Trees, Ashenhurst/Curtis hierarchical
  decomposition and Bi-Decomposition algorithms are
  used in our software
• These methods create our subset of MVSIS system
  developed under Prof. Robert Brayton at
  University of California at Berkeley 2.
• The entire MVSIS system can be also used.
• The system generates robots behaviors (C program
  codes) from examples given by the users.
• This method is used for embedded system design,
  but we use it specifically for robot interaction.

93
Ashenhurst Functional Decomposition
Evaluates the data function and attempts to
decompose into simpler functions.
F(X) H( G(B), A ), X A ? B
X
B - bound set
if A ? B ?, it is disjoint decomposition if A ?
B ? ?, it is non-disjoint decomposition
94
A Standard Map of function z
Explain the concept of generalized dont cares
Bound Set
a b \ c
Columns 0 and 1 and columns 0 and 2 are
compatible column compatibility 2
Free Set
z
95
NEW Decomposition of Multi-Valued Relations
F(X) H( G(B), A ), X A ? B
A
X
Relation
Relation
B
if A ? B ?, it is disjoint decomposition if A ?
B ? ?, it is non-disjoint decomposition
96
Forming a CCG from a K-Map
Columns 0 and 1 and columns 0 and 2 are
compatible column compatibility index 2
Column Compatibility Graph
z
97
Forming a CIG from a K-Map
Columns 1 and 2 are incompatible chromatic number
2
Column Incompatibility Graph
98
Constructive Induction

• A unified internal language is used to describe
  behaviors in which text generation and facial
  gestures are unified.
• This language is for learned behaviors.
• Expressions (programs) in this language are
  either created by humans or induced automatically
  from examples given by trainers.

99

• Braitenberg Vehicles and Quantum Automata Robots

100
Another Example Braitenberg Vehicles and Quantum
BV
101
Braitenberg Vehicles
102
Quantum Circuits
Toffoli gate Universal, uses controlled square
root of NOT
0? 1? x?
0? 1? Vx?
0? 1?
0? 1? x?
0? 1?
0? 1? x?
?

• Example 1 Simulation

103
Quantum Portland Faces
104
Conclusion. What did we learn

• (1) the more degrees of freedom the better the
  animation realism. Art and interesting behavior
  above certain threshold of complexity.
• (2) synchronization of spoken text and head
  (especially jaw) movements are important but
  difficult. Each robot is very different.
• (3) gestures and speech intonation of the head
  should be slightly exaggerated superrealism,
  not realism.

105
Conclusion. What did we learn(cont)

• (4) Noise of servos
• the sound should be laud to cover noises coming
  from motors and gears and for a better
  theatrical effect.
• noise of servos can be also reduced by
  appropriate animation and synchronization.
• (5) TTS should be enhanced with some new
  sound-generating system. What?
• (6) best available ATR and TTS packages should be
  applied.
• (7) OpenCV from Intel is excellent.
• (8) use puppet theatre experiences. We need
  artists. The weakness of technology can become
  the strength of the art in hands of an artist.

106
Conclusion. What did we learn(cont)

• (9) because of a too slow learning, improved
  parameterized learning methods should be
  developed, but also based on constructive
  induction.
• (10) open question funny versus beautiful.
• (11) either high quality voice recognition from
  headset or low quality in noisy room. YOU CANNOT
  HAVE BOTH WITH CURRENT ATR TOOLS.
• (12) low reliability of the latex skins and this
  entire technology is an issue.

107
We won an award in PDXBOT 2004. We showed our
robots to several audiences
Robot shows are exciting
Our Goal is to build toys for 21-st Century and
in this process, change the way how engineers are
educated.
International Intel Science Talent Competition
and PDXBOT 2004, 2005
108

• Commercial Value of Robot Toys and Theatres

109
Robot Toy Market - Robosapiens
toy, poses in front of
toy, poses in front of
toy, poses in front of
110
Globalization

• Globalization implies that images, technologies
  and messages are everywhere, but at the same time
  disconnected from a particular social structure
  or context. (Alain Touraine)
• The need of a constantly expanding market for its
  products chases the bourgoise over the whole
  surface of the globe. It must nestle everywhere,
  settle everywhere, establish connections
  everywhere. (Marx Engels, 1848)

111
India and China - whats different?

• They started at the same level of wealth and
  exports in 1980
• China today exports 184 Bn vs 34 Bn for India
  
• Chinas export industry employs today over 50
  million people (vs 2 m s/w in 2008, and 20 m in
  the entire organized sector in India today!)
• Chinas export industry consists of toys (gt 60
  of the world market), bicycles (10 m to the US
  alone last year), and textiles (a vision of
  having a share of gt 50 of the world market by
  2008)

112
Learning from Korea and Singapore

• The importance of Learning
• To manufacture efficiently
• To open the door to foreign technology and
  investment
• To have sufficient pride in ones own ability to
  open the door and go out and build ones own
  proprietary identity
• To invest in fundamentals like Education
• to have the right cultural prerequisites for
  catching up
• To have pragmatism rule, not ideology

113
Samsung

• 1979 Started making microwaves
• 1980 First export order (foreign brand)
• 1983 OEM contracts with General Electric
• 1985 All GE microwaves made by Samsung
• 1987 All GE microwaves designed by Samsung
• 1990 The worlds largest microwave
  manufacturer - without its own brand
• 1990 Launch own brand outside Korea
• Samsung microwaves 1 worldwide, twelve
  factories in twelve countries (including India,
  China and the US)
• 2003 the largest electronics company in the
  world

114
How did Samsung do it?

• By learning from GE and other buyers
• By working very hard - 70 hour weeks, 10 days
  holiday
• By being very productive - 9 microwaves per
  person per day vs 4 at GE
• By meeting every delivery on time, even if it
  meant working 7-day weeks for six months
• By developing new models so well that it got GE
  to stop developing their own

115

• Fundamental question for humanoid robot builders

116
Should we build humanoid robots?

• Mans design versus robots design
• The humanoid robot is versatile and adaptive, it
  takes its form from a human, a design
  well-verified by Nature.
• Complete isomorphism of a humanoid robot with a
  human is very difficult to achieve (walking) and
  not even not entirely desired.
• All what we need is to adapt the robot maximally
  to the needs of humans elderly, disabled,
  children, entertainment.
• Replicating human motor or sensor functionality
  are based on mechanistic methodologies,
• but adaptations and upgrades are possible for
  instance brain wave control or wheels
• Is it immoral?

117
Is it worthy to build humanoid robots?

• Can building a mechanistic digital synthetic
  version of man be anything less than a cheat when
  man is not mechanistic, digital nor synthetic? 
• If reference for the ultimate robot is man,
  then there is little confusion about ones aim to
  replace man with a machine.

118
Man Machine

• Main reason to build machines in our likeness is
  to facilitate their integration in our social
  space
• SOCIAL ROBOTICS
• Robot should do many things that we do, like
  climbing stairs, but not necessarily in the way
  we do it airplane and bird analogy.
• Humanoid robots/social robots should make our
  life easier.

119
The Social Robot

• developing a brain
• Cognitive abilities as developed from classical
  AI to modern cognitive ideas (neural networks,
  multi-agent systems, genetic algorithms)
• giving the brain a body
• Physical embodiment, as indicated by Brooks
  Bro86, Steels Ste94, etc.
• a world of bodies
• Social embodiment
• A Social Robot is
• A physical entity embodied in a complex, dynamic,
  and social environment sufficiently empowered to
  behave in a manner conducive to its own goals and
  those of its community.

120
Anthropomorphism

• Social interaction involves an adaptation on both
  sides to rationalise each others actions, and the
  interpretation of the others actions based on
  ones references
• Projective Intelligence the observer ascribes a
  degree of intelligence to the system through
  their rationalisation of its actions

121
Anthropomorphism The Social Robot

• Objectives
• Augment human-robot sociality
• Understand and rationalize robot behavior
• Embrace anthropomorphism
• BUT - How does the robot not become trapped by
  behavioral expectations?
• REQUIRED A balance between anthropomorphic
  features and behaviors leading to the robots own
  identity

122
Finding the Balance

• Movement
• Behavior (afraid of the light)
• Facial Action Coding System
• Form
• Physical construction
• Degrees of freedom
• Interaction
• Communication (robot-like vs. human voice)
• Social cues/timing
• Autonomy
• Function role
• machine vs. human capabilities

123
Humanoid Robots Experiments and Research Tasks

• Autonomous mobile robots
• Emotion through motion
• Projective emotion
• Anthropomorphism
• Social behaviors
• Qualitative and quantitative analysis to a wide
  audience through online web-based experiments

124
The perception learning tasks

• Robot Vision
• Where is a face? (Face detection)
• Who is this person (Face recognition, learning
  with supervisor, persons name is given in the
  process.
• Age and gender of the person.
• Hand gestures.
• Emotions expressed as facial gestures (smile, eye
  movements, etc)
• Objects hold by the person
• Lips reading for speech recognition.
• Body language.

125
The perception learning tasks

• Speech recognition
• Who is this person (voice based speaker
  recognition, learning with supervisor, persons
  name is given in the process.)
• Isolated words recognition for word spotting.
• Sentence recognition.
• Sensors.
• Temperature
• Touch
• movement

126
The behavior learning tasks

• Facial and upper body gestures
• Face/neck gesticulation for interactive dialog.
• Face/neck gesticulation for theatre plays.
• Face/neck gesticulation for singing/dancing.
• Hand gestures and manipulation.
• Hand gesticulation for interactive dialog.
• Hand gesticulation for theatre plays.
• Hand gesticulation for singing/dancing.

127
Learning the perception/behavior mappings

• Tracking the human.
• Full gesticulation as a response to human
  behavior in dialogs and dancing/singing.
• Modification of semi-autonomous behaviors such as
  breathing, eye blinking, mechanical hand
  withdrawals, speech acts as response to persons
  behaviors.
• Playing games with humans.
• Body contact with human such as safe
  gesticulation close to human and hand shaking.

128
What to emphasize in future work?

• We want to develop a general methodology for
  prototyping software/hardware systems for
  interactive robots that work in human
  environment.
• Image processing, voice recognition, speech
  synthesis, expressing emotions, recognizing human
  emotions.
• Machine Learning technologies.
• Safety, not hitting humans.

129
Can we build the first complete robot theatre in
the world?
Yes, if we will have more students who really
want to learn practical skills and not only to
take classes for grades.
Robotics I, Robotics II, individual projects,
RAS, high school students.
130
Where are we going?

• This is an adventure, we do not know where our
  research will lead us.
• This is truly interdisciplinary project. We need
  artists and psychologists.
• If this takes the social functions of a theatre,
  it is a theatre.
• Lessons from CAD and computer chess knowledge
  and search rather than super-intelligent logic
  mechanism.
• Initial complexity of knowledge.

131

• Lessons
• degeneration of robot soccer.
• OMSI project and security
• Laws about future robots, can he sue me?
• Our goal build a working environment for
• Education
• Entertainment
• Verification of theories (bacteria foraging,
  social dynamics, Freud, immunological robots)
• Verification of technologies (FPGA, clusters, net
  in chip technologies and AMBRIC).
• Many researchers will be able to base their own
  research on our environment. We provide the
  technical background for more advanced or
  artistic work.
• When there will be
• the first commercially successful robot theatre?
  
• the first humanoid social robot?

132
Humanoid robots

• 1. Teachers and helpers
• Language teachers
• Teaching children
• Teaching disabled children
• Helpers for disabled adults
• Helpers for old people
• Helpers and companions for mentally disabled

133
Humanoid robots

• 2. Toys
• Conversational toys for lonely girls and young
  woman
• Human-like robots as pets.
• Animal-like robots as pets.
• Interactive theatres of little robots sold
  separately and collected to families.

134
Humanoid robots

• 3. Robot Theatres
• Battle Bots (already commercial)
• Robot theatres for children, next generation of
  Chucky Cheese Pizza Theatres and Disney Worlds.
• Avangarda theatres for Adults (Umatilla, sex,
  violence, special effects like head separation,
  interaction, battle bots of new generation, and
  large size robot theatres in the prerries).
• Artistic robot theatres (none exist see
  Japanese Bunraku and Noh single robots, Kissmet,
  aquarium and new robots of Cynthia Breazeal from
  MIT).

135
Humanoid robots

• 4. Kiosks and receptionists
• Toy-like and simplified (commercial products).
• Realistic in view and size.
• Mobile museum robots (commercial).
• Wheeled humanoid robot of child-like size to be
  rented for exhibitions.

136
Humanoid robots

• 5. Top research robots
• Kissmet
• Honda
• Sony
• Fujitsu
• Hubo and KAIST
• Samsung
• Many Japanese
• 6. Commercial Robot kits.
• Mobile robots
• Walking robots
• Heads
• Humanoids small
• Humanoids childlike and expensive. Pino.

137
Existing technologies for robot theatre

• Mobile robots (battlebots, Los Angeles group,
  Carnegie Mellon Group)
• Walking animals
• Walking big humans with robotic featuresJapanese
  robots like trump playing Sony)
• Walking big humans with human-like features (head
  only - Albert Hubo, Small humans.
• Body on wheels.
• Head only
• Head with neck and shoulders.
• Upper body
• Head on wheels

138
New Robots

• 2005

139

• "Nothing serious. Just stunts. There are dogs,
  dolls, faces that contort and are supposed to
  express emotion on a robot," he said.
• Mr Engelberger, an American, founded the world's
  first company making industrial robots in 1961
  and became a specialist manufacturer of robots
  for hospitals.
• It was pointless, expensive and unnecessary for
  Japan, which today makes three-quarters of the
  world's robots, to tinker with trivial inventions
  like robotic house sitters that rang to say there
  was a burglary going on, he said.
• It made more sense to use the formidable amount
  of research that it had already done on personal
  robot technology to apply it to machines made for
  tasks that actually needed doing.
• Such as robots that could be told by elderly or
  infirm people to fetch a book from a shelf or
  find the television remote or get a beer from the
  fridge.

140

• "I've talked to visiting nurses who say that
  older people have to go to the bathroom more
  often and are embarrassed to say to somebody in
  the house, 'Please take me to the bathroom
  again'. But who cares how many times you ask a
  robot to take you to the bathroom?"
• The future market for robots installed in the
  homes of elderly people was bigger than the
  luxury car market, he said, predicting that they
  would be leased out for US500 (673) per month.
• Human care-givers cost 10 times that, Mr
  Engelberger said, and nursing homes were higher
  still.
• "I know that there are things that a robot can't
  do. It's not going to bathe you and it's not
  going to dress you but it can be made to find the
  milk in the fridge," he said.
• What the US8 billion robotics industry needs is
  for engineers to design practical robots for
  personal care. So why isn't more work being done?
  Mainly, Mr Engelberger thinks, it's because
  everyone is immersed in needless research and
  companies are distracted by the uneconomic quest
  for the humanoid, which he derides as toy making.
• "I say, stop it all go for the whole damn
  schmeer I've recently become an octogenarian
  and I'd ask you, please, hurry up."

141
Albert Hubo

• At an IT exhibition on the sidelines of the
  Asia-Pacific Economic Cooperation (APEC) summit
  in Busan, a participant shakes hands with a
  humanoid robot named Albert Hubo which has the
  face of Albert Einstein on Monday.
• The robot can walk and speak and expresses
  emotions by moving facial muscles

142
Albert Hubo meets President Bush
Help me robo-Einstein, youre my only hope
143
Fujitsus Enon is getting a job at the grocery
store

• Enon will be helping Aeon customers with
  everything from packing shopping bags and picking
  up groceries to find their way around the store.

144

• This is the new HAL-5, or more specifically the
  Hybrid Assistive Limb. Bionic Suit.
• Its developed as a walking aid for those who
  could use a bit of extra power, such as the
  disabled or apparently farmers who must add bags
  of sodium to their basement water softeners.

145
Walking Actors, Japan

• 1000 iXs Research Corp. robots at Tokyos
  International Robot Exhibition.

146
The Bandai BN-17 Swiffer bot

• Bandai BN-17 robot anthropomorphized robot for
  cleaning.
• It can also handle your email and act as a
  security system

147
SORA, a receptionist robot
The little bot sports a camera, microphone, and
speaker for one way video conferencing with
visitors, who can interface with an included
touchscreen for information, and even scan a
business card to show their identity to whoever
is subjecting them to this robotic greeter. Once
theyre all approved, the robot can wave its
arms at them and point out the directions to the
office being displayed on the screen.
148
The WowWee Robosapien v2

• The 230 second generation Robosapien v2 with
  remote from WowWee.
• The Robosapien v2 can see, hear, touch, and
  interact with you and his surroundings with a
  full range of fluid movement.
• Seething with attitude, his full-functioning arms
  with grippers allow him to pick-up and throw
  objects and then kung-fu your azz if you sass
  him.
• He features 100s of functions including a
  low-level gastro-intestinal condition resulting
  in the occasional air-biscuit or belch to your
  childrens (and yours, admit it) amusement.
• Hes also fully programmable which means youll
  find hacks-a-plenty in the open-source community
  allowing you to extend his functionality.

149
Questions to students

1. Learn about new robot toys and other toys that
   can be used in our theatre or converted to useful
   robots or their components.
2. Explain the concept of mapping architecture for a
   robot. Mapping being a combinational functions
   and mapping based on Finite State Machines.
3. Explain the concept of Probabilistic Finite State
   Machine and how it can be used to control
   movements of a robot.
4. How to use finite state machines and
   probabilistic machines for dialog and speech
   generation

150
Questions to students

1. Explain Morita Theory and think if it can be
   generalized.
2. Our robots have speech recognition and vision.
   Some have also sonar, infrared, touch and other
   sensors. What kind of sensors you would like to
   add and how you would like to program them for
   your applications.
3. What is your concept of interactive robot toy
   that would extend the ideas of our Theatre.
4. Write a script-scenario of conversation with
   robot that can be in 3 emotional states. The
   robot is a receptionist in Electrical Engineering
   Department at PSU.