From Teleoperation to Autonomy

1
From Teleoperation to Autonomy

• Define Intelligent Robot
• Be able to describe at least two differences
  between AI and engineering approaches to robotics
• Be able to describe the difference between
  telepresence and semi-autonomous control
• Have some feel for the history and societal
  impact of robotics

History -AI -Engineering Teleop -Motivation -Compo
nents -Problems -Alternatives Case
Studies Programming Summary Review
2
Intelligent Robot

• Mechanical creature which can function
  autonomously
• Mechanical built, constructed
• Creature think of it as an entity with its own
  motivation, decision making processes
• Function autonomously can sense, act, maybe even
  reason doesnt just do the same thing over and
  over like automation

History -AI -Engineering Teleop Case
Studies Programming Summary Review
3
What are Robots?

• Autonomous mechanical creatures
• Capek 1921 R.U.R.
• Intelligent because teleoperation doesnt work,
  doesnt scale
• Physically situated, but now software agents or
  softbots
• Principles from robotics influenced AI community,
  esp. planning
• Combines programming, networks, operating
  systems, algorithms, everything about CS into a
  system (the ultimate software engineering project)

History -AI -Engineering Teleop Case
Studies Programming Summary Review
www.fradulent.org/rur.htm
4
Robots Constantly in the Press
History -AI -Engineering Teleop Case
Studies Programming Summary Review
www.sony.com
courtesy of MIT AI Lab
www.irobot.com
courtesy of Honda
5
Less Famous Cousins at WTC
Inuktun microTracks
6
Why Robots? Dirty, Dangerous, Dull Tasks

• JV2010, TRADOC, JFCOM, all branches even down to
  the organic level
• Reconnaissance, MOUT, denial of area, consequence
  management, logistics, demining

History -AI -Engineering Teleop Case
Studies Programming Summary Review
www.friendlymachines.com
Replace Humans with Robots
7
Why Robots? Better Than Bio

• Robots at WTC
• voids smaller than person could enter
• voids on fire or oxygen depleted
• NBC Response
• Lose ½ cognitive attention with each level of
  protection
• Level A12.5 of normal ability

Void1x2.5x60
History -AI -Engineering Teleop Case
Studies Programming Summary Review
Void on fire
Do Things that Living Things Cant
8
Major Robot Modalities UAV, UGV, UUV

• Unmanned Aerial Vehicles
• drones since Vietnam Global Hawk, UCAV
• easy nothing to hit
• hard mission sensing, human-in-the-loop control
• Unmanned Ground Vehicles
• since 1967
• easy can always stop and think, a priori maps
• hard perceiving, e.g., light vegetation vs. wall
• Unmanned Underwater Vehicles
• ROVs since 1960s
• easy run tethers
• hard platform operation in unfriendly environment

History -AI -Engineering Teleop Case
Studies Programming Summary Review
Mobility (platform), Perception, Communications
HRI, Control (Intelligence), Power
9
A Brief History
10
Industrial Manipulators
History -AI -Engineering Teleop Case
Studies Programming Summary Review

• Tommy type of robots deaf, dumb, and blind
• High precision, fast repetition
• Usually no sensing of the environment
• Welding can be off by an inch

11
3 Ways of Controlling a Robot

• RC-ing
• you control the robot
• you can view the robot and its relationship to
  the environment
• ex. radio controlled cars, bomb robots
• operator isnt removed from scene, not very safe

History Teleop -Motivation -Components -Problems -
Alternatives Case Studies Programming Summary Revi
ew
12
3 Ways of Controlling a Robot

• teleoperation
• you control the robot
• you can only view the environment through the
  robots eyes
• dont have to figure out AI

History Teleop -Motivation -Components -Problems -
Alternatives Case Studies Programming Summary Revi
ew
13
3 Ways of Controlling a Robot

• semi- or full autonomy
• you might control the robot sometimes
• you can only view the environment through the
  robots eyes
• ex. Sojouner with different modes
• human doesnt have to do everything

History Teleop -Motivation -Components -Problems -
Alternatives Case Studies Programming Summary Revi
ew
14
Components of a Telesystem(after Uttal 89)

• Local
• display
• Local control device
• Communication
• Remote
• sensor
• mobility
• effector
• power

15
Example
History Teleop -Motivation -Components -Problems -
Alternatives Case Studies Programming Summary Revi
ew
Remote
Local
16
Typical Run
History Teleop -Motivation -Components -Problems -
Alternatives Case Studies Programming Summary Revi
ew
17
Problems That You Saw

• no feedback, couldnt really tell that the robot
  was stuck but finally got free
• robot doesnt have proprioception or internal
  sensing to tell you what the flippers were doing.
  No crunching noises, no pose widget to show the
  flippers
• no localization, mapping-gt no idea how far
  traveled
• partial solution better instrumentation (but
  cant do dead reckoning well)
• operator doesnt have an external viewpoint to
  show itself relative to the environment
• solution two robots, one to spot the other
• communications dropout, even though 3 meters
  away
• lighting conditions went from dark to very bright
• hard for computer vision or human to adjust

History Teleop -Motivation -Components -Problems -
Alternatives Case Studies Programming Summary Revi
ew
18
But good for unmodeled events
History Teleop -Motivation -Components -Problems -
Alternatives Case Studies Programming Summary Revi
ew
19
Communications is ImportantDarkStar7
secondsDarkSpot
History Teleop -Motivation -Components -Problems -
Alternatives Case Studies Programming Summary Revi
ew

• 7 second communications lag (satellite relay)
• interruption lag on part of operator

20
Predator71 human to robot ratio
History Teleop -Motivation -Components -Problems -
Alternatives Case Studies Programming Summary Revi
ew
Leos unofficial Predator page

• 4 people to control it (52-56 weeks of training)
• one for flying
• two for instruments
• one for landing/takeoff
• plus maintenance, sensor processing and routing
• lack of self-awareness in Kosovo, come along
  side in helicopter and shoot down

21
Summary of Teleop Problems

• cognitive fatigue
• communications dropout
• communications bandwidth
• communications lag
• too many people to run one robot (hidden cost)

History Teleop -Motivation -Components -Problems -
Alternatives Case Studies Programming Summary Revi
ew
22
Telesystems Best Suited For

• the tasks are unstructured and not repetitive
• the task workspace cannot be engineered to permit
  the use of industrial manipulators
• key portions of the task require dexterous
  manipulation, especially hand-eye coordination,
  but not continuously
• key portions of the task require object
  recognition or situational awareness
• the needs of the display technology do not exceed
  the limitations of the communication link
  (bandwidth, time delays)
• the availability of trained personnel is not an
  issue

History Teleop -Motivation -Components -Problems -
Alternatives Case Studies Programming Summary Revi
ew
23
Teleop Improvements Telepresence

• Telepresence
• improves human control, reduces simulator
  sickness and cognitive fatigue by providing
  sensory feedback to the point that teleoperator
  feels they are present in robots environment
• increases demands on bandwidth

History Teleop -Motivation -Components -Problems -
Alternatives Case Studies Programming Summary Revi
ew
24
Teleop ImprovementsSupervisory Control

• Semi-autonomous
• Supervisory Control
• human is involved, but routine or safe portions
  of the task are handled autonomously by the robot
• is really a type of mixed-initiative
• Shared Control/ Guarded Control
• human initiates action, interacts with remote by
  adding perceptual inputs or feedback, and
  interrupts execution as needed
• robot may protect itself by not bumping into
  things
• Traded Control
• human initiates action, does not interact

History Teleop -Motivation -Components -Problems -
Alternatives Case Studies Programming Summary Revi
ew
25
Teleop ImprovementsMixed-Initiative

• Levels of Initiative
• do only what told to do (teleoperation)
• recommend or augment (cognitive augmentation)
• act and report
• act on own and supervise itself (autonomy)

History Teleop -Motivation -Components -Problems -
Alternatives Case Studies Programming Summary Revi
ew
26
No Hands Across America

• 1994
• CMU NavLab
• Pittsburgh to San Diego
• 2897 miles total
• 2849 autonomously
• Autonomous or Mixed-Initiative?

History Teleop -Motivation -Components -Problems -
Alternatives Case Studies Programming Summary Revi
ew
27
Mixed-Initiative Matching Game

• Level of Initiative
• teleoperation
• cognitive augmentation
• act and report
• act on own

• Application
• janitorial robot
• medical robot (telemedicine)
• high flying surveillance drone
• combat aerial vehicle
• resupply system for bringing water to fire
  fighters
• guard dog robot
• nurse robot

History Teleop -Motivation -Components -Problems -
Alternatives Case Studies Programming Summary Revi
ew
28
Collaborative Teleoperation
1
3
mpg June 2, 2000 SRDR Miami Beach view from
Inuktun as it falls
mpg June 2, 2000 SRDR Miami Beach view from
Inuktun from hoisted position
2

• Urban is stuck, Inuktun cant help from current
  perspective
• Driven off 3rd floor
• Hoisted to 2nd floor by tether
• Has better view, changing configuration rocking
  extend view

still June 2, 2000 SRDR Miami Beach
29
2000 AAAI Mobile Robot
History Teleop Case Studies Programming Summary Re
view

• 2 robots helping each other reduced collision
  errors, sped up time navigating confined space,
  righting

30
ExampleMixed-Initiative Collab. Teleop

• 9/2000 DARPA Tactical Mobile Robots demonstration
• Robot used an intelligent assistant agent to look
  for signs of snipers hiding in urban rubble
• motion
• skin color
• difference in color
• thermal (IR camera)
• Human navigated mother robot using viewpoint of
  2nd robot (not in picture)
• Once deposited the human moved the daughter
  robot, and either saw a sniper or was alerted by
  the agent

History Teleop Case Studies Programming Summary Re
view
31
AI provides the other stuff

• knowledge representation
• understanding natural langugage
• learning
• planning and problem solving
• inference
• search
• vision

History Teleop Case Studies Programming Summary Re
view
32
Example User Expectation of AI

• Proposed Goal 11 soldierany robot, where 1
  soldier is responsible for 1 or more active
  robots but does not have to pay continuous
  attention to them.

MAV-UGV cooperative monitoring
UAVs as theater assets
MAVs as organic assets
Field recon- figurable UUVs
Flocks of MAVs
33
More Reasonable Expections
Consolidation
agents with tactical autonomy, toolkits
Vehicle success is still based on human, but
robot is in front
History Teleop Case Studies Programming Summary Re
view
Dedicated Autonomy Systems
Human intermittent attention as team coordinator,
not with individuals
Mass-produced dedicated agents
Cooperating pack or herd agents
Reconfigurable AutonomySystems
Human primary responsibility as a tool builder,
expert advisor. Peer-level communication
Field- reconfigurable agents
34
Programming Notes

• You always need telesystem or human intervention
  as a backup
• at some point a human will need to take control
• embed in your design
• Roboticists automate what is easy and leave the
  rest to the human- Don Norman
• The user interface is absolutely critical
• User interface make up 60 of commercial code
• Useful is the program purpose useful?
• usually given to designer via specifications and
  requirements
• Usable can a human use it efficiently?
• designer must conduct usability studies
• avoid if I can use it, some one else will

History Teleop Case Studies Programming Summary Re
view
35
Example of How an Internal Display Can Hurt
History Teleop Case Studies Programming Summary Re
view
iRobot PackBot video, FLIR, 2 way audio

• gamer joystick plus laptop with video audio
• robot state battery, comms, orientation, camera,
  encoders
• was not used on rubble pile at WTC because it
  scared off rescuers too complicated, too long to
  boot, too toy
• now integrated with Land Warrior used in
  Afghanistan

36
Summary

• Teleoperation arose a partial solution to
  autonomy
• cognitive fatigue, high comms bandwidth, long
  delays, and manyone human to robot ratios
• Telepresence tries to reduce cognitive fatigue
  through enhanced immersive environments
• Semi-autonomy tries to reduce fatigue, bandwidth
  by delegating portions of the task to robot
• mixed-initiative
• Teleop isnt simple and improvements arent just
  better user interfaces

History Teleop Case Studies Programming Summary Re
view
37
Review Questions

• What is an intelligent robot?
• What is the difference between engineering and AI
  robotics?
• What are 3 types of control?
• What are the parts of a telesystem?
• What are problems with teleoperation?
• Whats the difference between telepresence and
  semi-autonomous control?
• What are the levels of initiative
  (mixed-initiative)?
• What are alternatives to traditional
  teleoperation?

History Teleop Case Studies Programming Summary Re
view