HumanRobot Interaction through a Dynamic Virtual Environement

1
Human-Robot Interaction through a Dynamic Virtual
Environement

• Andrew Fagg, Shichao Ou,
• Reed Hedges, Matt Brewer, Michael Piantedosi

2
Outline

• Introduction and goal
• Virtual Reality and Augmented Reality
• System components
• System architecture
• Network Visualization Client Implementation
• Low Level Robot Controllers
• Demo
• Next Steps
• Questions

3
Introduction and goal

• Develop generalized robot-teams control interface
• Enable user to coordinate data-gathering
  activities
• Intuitively access large amounts of data gathered
  by the robots

4
Essential Elements

• Distributed (work under low bandwidth)
• Mobile
• Handle large teams of robots
• Intuitive and unobtrusive
• Allows to train control policies

5
Virtual Reality and Augmented Reality

• Spatial perception.
• Remote operation by multiple people.
• Display of abstract information (logical
  relationships, sensor data, sensor- and
  algorithm- debugging information.
• Wearable computers and tracking devices offer
  mobility, immersion
• More modes of input, and presence in both real
  and virtual spaces.

6
System Components

• Network Visualization Client CS and VOS
• Voice command IBM Viavoice API
• Player/Stage platform
• Robot controllers path-planners
• Egosphere viewer
• Hardware wearables, RWI robots

7
System Architecture
User
User .
.. User
Voice command
3D world, virtual avatar, egoviewer
Visualization Client
Controllers...
Player
Stage
Real Robots
Simulated Robots
8
Network Visualization Client Implementation

• VOS (Virtual Object System) allows programs to
  intercommunicate through a shared hierarchy of
  distributed objects objects can exist on any
  network host, and support standard, well defined
  interfaces (types).
• Each operator accesses the environment through a
  3D-display client. A robot interface client
  represents the robot in the environment,
  interpreting commands, updating the robot's 3D
  "avatar" object and even creating new 3D objects.
• Features
• Operators and robots use the same "talk"
  interface for communicating with other operators
  and robots.
• Robot client can use the operator's properties
  (e.g. 3D position) as context for executing
  commands (e.g. "set goal here")
• Multi-user environment facilitates operators'
  collaboration.

9
Network Visualization Client Implementation

• Reeds VOS architecture diagram

10
Low Level Robot Controllers Player/Stage Servers

• Player
• designed as driver system for pioneer robots
  (note not RWI)
• a "client" runs control algorythms locally
  controls player server running on the robot
• server has a set of devices - position,sonar ...
• Stage
• replaces Player's real devices with simulator
• advantages
• seamless simulator/robot conversion - invisible
  to clients
• control algorythms can run anywhere on network
  (or in the world)
• rapid client development - simple interfacing
  through proxies
• light interface, little overhead, good for
  realtime applications

11
Low Level Robot Controllers Our Clients

• Localization
• Monte Carlo implementation of a Kalman Filter
  (Sebastion Thrun)
• Advantages
• efficient
• error tolerant
• Problems
• sonar noisy sensor
• PathPlanning
• Global Distance Propogation generates "carrot"
  for Local Harmonic function
• Advantages
• exploits advantages of both
• Harmonic Function - minimum hitting probability
• Distance Propagation - no 71 problems, fast
  calculation

12
Low Level Robot Controllers A little bit
Bigger picture

• build a library of services
• interface between through player com device
• data shared through buffer lying on server -
  neat, easy (maps, goals ...)

13
Demo

• User-Interface introduction
• Search Investigate Scenario
• 2 operators, 1 real robot, 1 virtual robot (?)
• Send robot to investigate dangerous corner
• Reactive robot avoidance
• Robot teleoperation with reactive safety feature
• (what can we get the other operator simulated
  robot to do other than demonstrate the
  distributed visualization feature)

14
Next Steps

• Mapping
• Dynamic mosaics implicitly driving the goal
  position of a camera (fixed position or a robot)
  based on how the user is exploring an egosphere
  or the 3D environment
• Additional controllers search or wait for some
  feature (e.g. motion)
• patrol, etc. In many of these cases, the robots
  will generate alerts and requests from a user.
• Training robot control policies.
• Team robots hierarchical command feedback
  structure

15
Next Steps

• Look more closely at how to handle vast sensory
  data feedback with large teams of robots
• Queuing, delivery based on priority
• Learning, adapting to the users preference
• Hierarchical chain of commands
• Multiple operator team coordination
• User Evaluation
• Easy-to-use-ness
• Effectiveness
• Operate under familiar environment
• Operate under unknown environment

16
Questions

• ?
• ?
• ?
• ?
• ?
• ?