Title: Behavior, Dialog and Learning
1Behavior, Dialog and Learning
- The dialog/behavior has the following components
- (1) Eliza-like natural language dialogs based on
pattern matching and limited parsing. - Commercial products like Memoni, Dog.Com, Heart,
Alice, and Doctor all use this technology, very
successfully for instance Alice program won the
2001 Turing competition. - This is a conversational part of the robot
brain, based on pattern-matching, parsing and
black-board principles. - It is also a kind of operating system of the
robot, which supervises other subroutines.
2Behavior, Dialog and Learning
- (2) Subroutines with logical data base and
natural language parsing (CHAT). - This is the logical part of the brain used to
find connections between places, timings and all
kind of logical and relational reasonings, such
as answering questions about Japanese geography. - (3) Use of generalization and analogy in dialog
on many levels. - Random and intentional linking of spoken
language, sound effects and facial gestures. - Use of Constructive Induction approach to help
generalization, analogy reasoning and
probabilistic generations in verbal and
non-verbal dialog, like learning when to smile or
turn the head off the partner.
3Behavior, Dialog and Learning
- (4) Model of the robot, model of the user,
scenario of the situation, history of the dialog,
all used in the conversation. - (5) Use of word spotting in speech recognition
rather than single word or continuous speech
recognition. - (6) Continuous speech recognition (Microsoft)
- (7) Avoidance of I do not know, I do not
understand answers from the robot. - Our robot will have always something to say, in
the worst case, over-generalized, with not valid
analogies or even nonsensical and random.
4 5(No Transcript)
6Example Age Recognition
Name (examples) Age (output) d Smile Height Hair Color Hair Color
Joan Kid (0) a(3) b(0) c(0) c(0)
Mike Teenager (1) a(2) b(1) c(1) c(1)
Peter Mid-age (2) a(1) b(2) c(2)
Frank Old (3) a(0) b(3) c(3) c(3)
Examples of data for learning, four people, given
to the system
7Example Age Recognition
Smile - a Very often often moderately rarely
Values 3 2 1 0
Height - b Very Tall Tall Middle Short
Values 3 2 1 0
Color - c Grey Black Brown Blonde
Values 3 2 1 0
Encoding of features, values of multiple-valued
variables
8Multi-valued Map for Data
Groups show a simple induction from the Data
ab\ c 0 1 2 3
00 - - - -
01 - - - 3
02 - - - -
03 - - - -
10 - - - -
11 - - - -
12 - - 2 -
13 - - - -
20 - - - -
21 - 1 - -
22 - - - -
23 - - - -
30 0 - - -
31 - - - -
32 - - - -
33 - - - -
ab\ c 0 1 2 3
00 - - - -
01 - - - 3
02 - - - -
03 - - - -
10 - - - -
11 - - - -
12 - - 2 -
13 - - - -
20 - - - -
21 - 1 - -
22 - - - -
23 - - - -
30 0 - - -
31 - - - -
32 - - - -
33 - - - -
d F( a, b, c )
9Old people smile rarely
Groups show a simple induction from the Data
blonde hair
Grey hair
ab\ c 0 1 2 3
00 - - - -
01 - - - 3
02 - - - -
03 - - - -
10 - - - -
11 - - - -
12 - - 2 -
13 - - - -
20 - - - -
21 - 1 - -
22 - - - -
23 - - - -
30 0 - - -
31 - - - -
32 - - - -
33 - - - -
Middle-age people smile moderately
Teenagers smile often
Children smile very often
10Another example teaching movements
Input variables
Output variables
11Generalization of the Ashenhurst-Curtis
decomposition model
12This kind of tables known from Rough Sets,
Decision Trees, etc Data Mining
13Original table
Second variant
First variant of decomposition
At every step many decompositions exist
Decomposition is hierarchical
Which decomposition is better?
14Constructive Induction Technical Details
- U. Wong and M. Perkowski, A New Approach to
Robots Imitation of Behaviors by Decomposition
of Multiple-Valued Relations, Proc. 5th Intern.
Workshop on Boolean Problems, Freiberg, Germany,
Sept. 19-20, 2002, pp. 265-270. - A. Mishchenko, B. Steinbach and M. Perkowski, An
Algorithm for Bi-Decomposition of Logic
Functions, Proc. DAC 2001, June 18-22, Las Vegas,
pp. 103-108. - A. Mishchenko, B. Steinbach and M. Perkowski,
Bi-Decomposition of Multi-Valued Relations, Proc.
10th IWLS, pp. 35-40, Granlibakken, CA, June
12-15, 2001. IEEE Computer Society and ACM SIGDA.
15Constructive Induction
- Decision Trees, Ashenhurst/Curtis hierarchical
decomposition and Bi-Decomposition algorithms are
used in our software - These methods create our subset of MVSIS system
developed under Prof. Robert Brayton at
University of California at Berkeley 2. - The entire MVSIS system can be also used.
- The system generates robots behaviors (C program
codes) from examples given by the users. - This method is used for embedded system design,
but we use it specifically for robot interaction.
16Ashenhurst Functional Decomposition
Evaluates the data function and attempts to
decompose into simpler functions.
F(X) H( G(B), A ), X A ? B
X
B - bound set
if A ? B ?, it is disjoint decomposition if A ?
B ? ?, it is non-disjoint decomposition
17A Standard Map of function z
Explain the concept of generalized dont cares
Bound Set
a b \ c
Columns 0 and 1 and columns 0 and 2 are
compatible column compatibility 2
Free Set
z
18NEW Decomposition of Multi-Valued Relations
F(X) H( G(B), A ), X A ? B
A
X
Relation
Relation
B
if A ? B ?, it is disjoint decomposition if A ?
B ? ?, it is non-disjoint decomposition
19Forming a CCG from a K-Map
Columns 0 and 1 and columns 0 and 2 are
compatible column compatibility index 2
Column Compatibility Graph
z
20Forming a CIG from a K-Map
Columns 1 and 2 are incompatible chromatic number
2
Column Incompatibility Graph
21Constructive Induction
- A unified internal language is used to describe
behaviors in which text generation and facial
gestures are unified. - This language is for learned behaviors.
- Expressions (programs) in this language are
either created by humans or induced automatically
from examples given by trainers.
22- Braitenberg Vehicles and Quantum Automata Robots
23Another Example Braitenberg Vehicles and Quantum
BV
24Braitenberg Vehicles
25Quantum Circuits
Toffoli gate Universal, uses controlled square
root of NOT
0? 1? x?
0? 1? Vx?
0? 1?
0? 1? x?
0? 1?
0? 1? x?
?
26Quantum Portland Faces
27Conclusion. What did we learn
- (1) the more degrees of freedom the better the
animation realism. Art and interesting behavior
above certain threshold of complexity. - (2) synchronization of spoken text and head
(especially jaw) movements are important but
difficult. Each robot is very different. - (3) gestures and speech intonation of the head
should be slightly exaggerated superrealism,
not realism.
28Conclusion. What did we learn(cont)
- (4) Noise of servos
- the sound should be laud to cover noises coming
from motors and gears and for a better
theatrical effect. - noise of servos can be also reduced by
appropriate animation and synchronization. - (5) TTS should be enhanced with some new
sound-generating system. What? - (6) best available ATR and TTS packages should be
applied. - (7) OpenCV from Intel is excellent.
- (8) use puppet theatre experiences. We need
artists. The weakness of technology can become
the strength of the art in hands of an artist.
29Conclusion. What did we learn(cont)
- (9) because of a too slow learning, improved
parameterized learning methods should be
developed, but also based on constructive
induction. - (10) open question funny versus beautiful.
- (11) either high quality voice recognition from
headset or low quality in noisy room. YOU CANNOT
HAVE BOTH WITH CURRENT ATR TOOLS. - (12) low reliability of the latex skins and this
entire technology is an issue.
30We won an award in PDXBOT 2004. We showed our
robots to several audiences
Robot shows are exciting
Our Goal is to build toys for 21-st Century and
in this process, change the way how engineers are
educated.
International Intel Science Talent Competition
and PDXBOT 2004, 2005
31- Commercial Value of Robot Toys and Theatres
32Robot Toy Market - Robosapiens
toy, poses in front of
toy, poses in front of
toy, poses in front of
33Globalization
- Globalization implies that images, technologies
and messages are everywhere, but at the same time
disconnected from a particular social structure
or context. (Alain Touraine) - The need of a constantly expanding market for its
products chases the bourgoise over the whole
surface of the globe. It must nestle everywhere,
settle everywhere, establish connections
everywhere. (Marx Engels, 1848)
34India and China - whats different?
- They started at the same level of wealth and
exports in 1980 - China today exports 184 Bn vs 34 Bn for India
- Chinas export industry employs today over 50
million people (vs 2 m s/w in 2008, and 20 m in
the entire organized sector in India today!) - Chinas export industry consists of toys (gt 60
of the world market), bicycles (10 m to the US
alone last year), and textiles (a vision of
having a share of gt 50 of the world market by
2008)
35Learning from Korea and Singapore
- The importance of Learning
- To manufacture efficiently
- To open the door to foreign technology and
investment - To have sufficient pride in ones own ability to
open the door and go out and build ones own
proprietary identity - To invest in fundamentals like Education
- to have the right cultural prerequisites for
catching up - To have pragmatism rule, not ideology
36Samsung
- 1979 Started making microwaves
- 1980 First export order (foreign brand)
- 1983 OEM contracts with General Electric
- 1985 All GE microwaves made by Samsung
- 1987 All GE microwaves designed by Samsung
- 1990 The worlds largest microwave
manufacturer - without its own brand - 1990 Launch own brand outside Korea
- Samsung microwaves 1 worldwide, twelve
factories in twelve countries (including India,
China and the US) - 2003 the largest electronics company in the
world
37How did Samsung do it?
- By learning from GE and other buyers
- By working very hard - 70 hour weeks, 10 days
holiday - By being very productive - 9 microwaves per
person per day vs 4 at GE - By meeting every delivery on time, even if it
meant working 7-day weeks for six months - By developing new models so well that it got GE
to stop developing their own
38- Fundamental question for humanoid robot builders
39Should we build humanoid robots?
- Mans design versus robots design
- The humanoid robot is versatile and adaptive, it
takes its form from a human, a design
well-verified by Nature. - Complete isomorphism of a humanoid robot with a
human is very difficult to achieve (walking) and
not even not entirely desired. - All what we need is to adapt the robot maximally
to the needs of humans elderly, disabled,
children, entertainment. - Replicating human motor or sensor functionality
are based on mechanistic methodologies, - but adaptations and upgrades are possible for
instance brain wave control or wheels - Is it immoral?
40Is it worthy to build humanoid robots?
- Can building a mechanistic digital synthetic
version of man be anything less than a cheat when
man is not mechanistic, digital nor synthetic? - If reference for the ultimate robot is man,
then there is little confusion about ones aim to
replace man with a machine.
41Man Machine
- Main reason to build machines in our likeness is
to facilitate their integration in our social
space - SOCIAL ROBOTICS
- Robot should do many things that we do, like
climbing stairs, but not necessarily in the way
we do it airplane and bird analogy. - Humanoid robots/social robots should make our
life easier.
42The Social Robot
- developing a brain
- Cognitive abilities as developed from classical
AI to modern cognitive ideas (neural networks,
multi-agent systems, genetic algorithms) - giving the brain a body
- Physical embodiment, as indicated by Brooks
Bro86, Steels Ste94, etc. - a world of bodies
- Social embodiment
- A Social Robot is
- A physical entity embodied in a complex, dynamic,
and social environment sufficiently empowered to
behave in a manner conducive to its own goals and
those of its community.
43Anthropomorphism
- Social interaction involves an adaptation on both
sides to rationalise each others actions, and the
interpretation of the others actions based on
ones references - Projective Intelligence the observer ascribes a
degree of intelligence to the system through
their rationalisation of its actions
44Anthropomorphism The Social Robot
- Objectives
- Augment human-robot sociality
- Understand and rationalize robot behavior
- Embrace anthropomorphism
- BUT - How does the robot not become trapped by
behavioral expectations? - REQUIRED A balance between anthropomorphic
features and behaviors leading to the robots own
identity
45Finding the Balance
- Movement
- Behavior (afraid of the light)
- Facial Action Coding System
- Form
- Physical construction
- Degrees of freedom
- Interaction
- Communication (robot-like vs. human voice)
- Social cues/timing
- Autonomy
- Function role
- machine vs. human capabilities
46Humanoid Robots Experiments and Research Tasks
- Autonomous mobile robots
- Emotion through motion
- Projective emotion
- Anthropomorphism
- Social behaviors
- Qualitative and quantitative analysis to a wide
audience through online web-based experiments
47The perception learning tasks
- Robot Vision
- Where is a face? (Face detection)
- Who is this person (Face recognition, learning
with supervisor, persons name is given in the
process. - Age and gender of the person.
- Hand gestures.
- Emotions expressed as facial gestures (smile, eye
movements, etc) - Objects hold by the person
- Lips reading for speech recognition.
- Body language.
48The perception learning tasks
- Speech recognition
- Who is this person (voice based speaker
recognition, learning with supervisor, persons
name is given in the process.) - Isolated words recognition for word spotting.
- Sentence recognition.
- Sensors.
- Temperature
- Touch
- movement
49The behavior learning tasks
- Facial and upper body gestures
- Face/neck gesticulation for interactive dialog.
- Face/neck gesticulation for theatre plays.
- Face/neck gesticulation for singing/dancing.
- Hand gestures and manipulation.
- Hand gesticulation for interactive dialog.
- Hand gesticulation for theatre plays.
- Hand gesticulation for singing/dancing.
50Learning the perception/behavior mappings
- Tracking the human.
- Full gesticulation as a response to human
behavior in dialogs and dancing/singing. - Modification of semi-autonomous behaviors such as
breathing, eye blinking, mechanical hand
withdrawals, speech acts as response to persons
behaviors. - Playing games with humans.
- Body contact with human such as safe
gesticulation close to human and hand shaking.
51What to emphasize in future work?
- We want to develop a general methodology for
prototyping software/hardware systems for
interactive robots that work in human
environment. - Image processing, voice recognition, speech
synthesis, expressing emotions, recognizing human
emotions. - Machine Learning technologies.
- Safety, not hitting humans.
52Can we build the first complete robot theatre in
the world?
Yes, if we will have more students who really
want to learn practical skills and not only to
take classes for grades.
Robotics I, Robotics II, individual projects,
RAS, high school students.
53Where are we going?
- This is an adventure, we do not know where our
research will lead us. - This is truly interdisciplinary project. We need
artists and psychologists. - If this takes the social functions of a theatre,
it is a theatre. - Lessons from CAD and computer chess knowledge
and search rather than super-intelligent logic
mechanism. - Initial complexity of knowledge.
54- Lessons
- degeneration of robot soccer.
- OMSI project and security
- Laws about future robots, can he sue me?
- Our goal build a working environment for
- Education
- Entertainment
- Verification of theories (bacteria foraging,
social dynamics, Freud, immunological robots) - Verification of technologies (FPGA, clusters, net
in chip technologies and AMBRIC). - Many researchers will be able to base their own
research on our environment. We provide the
technical background for more advanced or
artistic work. - When there will be
- the first commercially successful robot theatre?
- the first humanoid social robot?
55Humanoid robots
- 1. Teachers and helpers
- Language teachers
- Teaching children
- Teaching disabled children
- Helpers for disabled adults
- Helpers for old people
- Helpers and companions for mentally disabled
56Humanoid robots
- 2. Toys
- Conversational toys for lonely girls and young
woman - Human-like robots as pets.
- Animal-like robots as pets.
- Interactive theatres of little robots sold
separately and collected to families.
57Humanoid robots
- 3. Robot Theatres
- Battle Bots (already commercial)
- Robot theatres for children, next generation of
Chucky Cheese Pizza Theatres and Disney Worlds. - Avangarda theatres for Adults (Umatilla, sex,
violence, special effects like head separation,
interaction, battle bots of new generation, and
large size robot theatres in the prerries). - Artistic robot theatres (none exist see
Japanese Bunraku and Noh single robots, Kissmet,
aquarium and new robots of Cynthia Breazeal from
MIT).
58Humanoid robots
- 4. Kiosks and receptionists
- Toy-like and simplified (commercial products).
- Realistic in view and size.
- Mobile museum robots (commercial).
- Wheeled humanoid robot of child-like size to be
rented for exhibitions.
59Humanoid robots
- 5. Top research robots
- Kissmet
- Honda
- Sony
- Fujitsu
- Hubo and KAIST
- Samsung
- Many Japanese
- 6. Commercial Robot kits.
- Mobile robots
- Walking robots
- Heads
- Humanoids small
- Humanoids childlike and expensive. Pino.
60Existing technologies for robot theatre
- Mobile robots (battlebots, Los Angeles group,
Carnegie Mellon Group) - Walking animals
- Walking big humans with robotic featuresJapanese
robots like trump playing Sony) - Walking big humans with human-like features (head
only - Albert Hubo, Small humans. - Body on wheels.
- Head only
- Head with neck and shoulders.
- Upper body
- Head on wheels
61New Robots
62- "Nothing serious. Just stunts. There are dogs,
dolls, faces that contort and are supposed to
express emotion on a robot," he said. - Mr Engelberger, an American, founded the world's
first company making industrial robots in 1961
and became a specialist manufacturer of robots
for hospitals. - It was pointless, expensive and unnecessary for
Japan, which today makes three-quarters of the
world's robots, to tinker with trivial inventions
like robotic house sitters that rang to say there
was a burglary going on, he said. - It made more sense to use the formidable amount
of research that it had already done on personal
robot technology to apply it to machines made for
tasks that actually needed doing. - Such as robots that could be told by elderly or
infirm people to fetch a book from a shelf or
find the television remote or get a beer from the
fridge.
63- "I've talked to visiting nurses who say that
older people have to go to the bathroom more
often and are embarrassed to say to somebody in
the house, 'Please take me to the bathroom
again'. But who cares how many times you ask a
robot to take you to the bathroom?" - The future market for robots installed in the
homes of elderly people was bigger than the
luxury car market, he said, predicting that they
would be leased out for US500 (673) per month. - Human care-givers cost 10 times that, Mr
Engelberger said, and nursing homes were higher
still. - "I know that there are things that a robot can't
do. It's not going to bathe you and it's not
going to dress you but it can be made to find the
milk in the fridge," he said. - What the US8 billion robotics industry needs is
for engineers to design practical robots for
personal care. So why isn't more work being done?
Mainly, Mr Engelberger thinks, it's because
everyone is immersed in needless research and
companies are distracted by the uneconomic quest
for the humanoid, which he derides as toy making. - "I say, stop it all go for the whole damn
schmeer I've recently become an octogenarian
and I'd ask you, please, hurry up."
64Albert Hubo
- At an IT exhibition on the sidelines of the
Asia-Pacific Economic Cooperation (APEC) summit
in Busan, a participant shakes hands with a
humanoid robot named Albert Hubo which has the
face of Albert Einstein on Monday. - The robot can walk and speak and expresses
emotions by moving facial muscles
65Albert Hubo meets President Bush
Help me robo-Einstein, youre my only hope
66Fujitsus Enon is getting a job at the grocery
store
- Enon will be helping Aeon customers with
everything from packing shopping bags and picking
up groceries to find their way around the store.
67- This is the new HAL-5, or more specifically the
Hybrid Assistive Limb. Bionic Suit. - Its developed as a walking aid for those who
could use a bit of extra power, such as the
disabled or apparently farmers who must add bags
of sodium to their basement water softeners.
68Walking Actors, Japan
- 1000 iXs Research Corp. robots at Tokyos
International Robot Exhibition.
69The Bandai BN-17 Swiffer bot
- Bandai BN-17 robot anthropomorphized robot for
cleaning. - It can also handle your email and act as a
security system
70SORA, a receptionist robot
The little bot sports a camera, microphone, and
speaker for one way video conferencing with
visitors, who can interface with an included
touchscreen for information, and even scan a
business card to show their identity to whoever
is subjecting them to this robotic greeter. Once
theyre all approved, the robot can wave its
arms at them and point out the directions to the
office being displayed on the screen.
71The WowWee Robosapien v2
- The 230 second generation Robosapien v2 with
remote from WowWee. - The Robosapien v2 can see, hear, touch, and
interact with you and his surroundings with a
full range of fluid movement. - Seething with attitude, his full-functioning arms
with grippers allow him to pick-up and throw
objects and then kung-fu your azz if you sass
him. - He features 100s of functions including a
low-level gastro-intestinal condition resulting
in the occasional air-biscuit or belch to your
childrens (and yours, admit it) amusement. - Hes also fully programmable which means youll
find hacks-a-plenty in the open-source community
allowing you to extend his functionality.
72Questions to students
- Learn about new robot toys and other toys that
can be used in our theatre or converted to useful
robots or their components. - Explain the concept of mapping architecture for a
robot. Mapping being a combinational functions
and mapping based on Finite State Machines. - Explain the concept of Probabilistic Finite State
Machine and how it can be used to control
movements of a robot. - How to use finite state machines and
probabilistic machines for dialog and speech
generation
73Questions to students
- Explain Morita Theory and think if it can be
generalized. - Our robots have speech recognition and vision.
Some have also sonar, infrared, touch and other
sensors. What kind of sensors you would like to
add and how you would like to program them for
your applications. - What is your concept of interactive robot toy
that would extend the ideas of our Theatre. - Write a script-scenario of conversation with
robot that can be in 3 emotional states. The
robot is a receptionist in Electrical Engineering
Department at PSU.