Faculty: Manuela Veloso, Anthony Stentz, Alex Rudnicky - PowerPoint PPT Presentation

About This Presentation
Title:

Faculty: Manuela Veloso, Anthony Stentz, Alex Rudnicky

Description:

Sphinx Phoenix Theta Rosetta X AIBOs X X X Gators X X X Pioneers X X X Segways X X X Humans Leader (L) Deliverer (D) Handler (H) Seeker (S) Mapper (M) ... – PowerPoint PPT presentation

Number of Views:97
Avg rating:3.0/5.0
Slides: 25
Provided by: curm
Learn more at: http://www.cs.cmu.edu
Category:

less

Transcript and Presenter's Notes

Title: Faculty: Manuela Veloso, Anthony Stentz, Alex Rudnicky


1
Dynamic Human-Robot Teams Engaged in Complex
Adversarial Tasks Using Language-Based
Communication
  • Faculty Manuela Veloso, Anthony Stentz, Alex
    Rudnicky
  • Brett Browning, M. Bernardine Dias
  • Students Thomas Harris, Brenna Argall, Gil
    Jones
  • Satanjeev Banerjee

Carnegie Mellon School of Computer
Science Boeing Human-Robot Teams Project
2
Project Goals
  • Treasure hunt with two or more teams of humans
    and robots competing to locate target objects as
    they explore an unknown space
  • Research Goals/Questions
  • Specify team members roles and capabilities to
    perform tasks
  • Rapidly form ad-hoc heterogeneous teams of robots
    and humans
  • Robots and humans executing synchronized action
    as a team, while communicating via speech
  • Improve team performance with experience

3
Alignment with Boeings Objectives
  • Our project develops component technologies
    relevant to
  • FCS Force multiplication for human-robot teams
  • NASA Code T Emphasis on robots assisting and
    augmenting humans in complex tasks
  • Space activities (Other than Code T)
  • Automating launch sites
  • Automating escape systems
  • Automating maintenance inspections/repair
  • Ice inspections
  • Aircraft maintenance
  • Automating maintenance inspections/repair
  • Ice inspections
  • Automated baggage handlers

In collaboration with Phillip Koons, Boeing
4
STP Overview
  • Skills for low-level control, Tactics for
    single robot, Plays for teamwork

APPLICABLE offense DONE aborted !offense ROLE 1
pass 3 mark best_opponent ROLE 2 block ROLE
3 pos_for_pass R B 1000 0 receive_pass
shoot A ROLE 4 defend_lane
5
STP Implementation
  • Play selection from playbook
  • Dynamic role assignment
  • Active tactic determines transition
  • Execution monitoring
  • Reward, and adaptation

Selection
Monitoring, Adaptation
Execution
Robot 1 Tactic
Robot 1 Tactic
Robot N Tactic
6
Relevant Research Interests
  1. Methods for pickup teams
  2. Extend play-based coordination to distributed
    execution
  3. Effective human-robot teamwork with pickup teams

7
TraderBots Overview
  • Robots are organized as an economy
  • Team mission is to maximize production and
    minimize costs
  • Robots exchange virtual money for tasks to
    maximize individual profit
  • System is designed to align local and global
    profit maximization






8
TraderBots Implementation
9
Relevant Research Interests
  1. Human-multi-robot interaction
  2. Role assignment for highly heterogeneous teams
  3. Improving robustness and adaptivity

10
Speech Overview
  • Communication in mobile environments
  • Natural spoken language-based interaction
  • Supporting high semantic content
  • Communication in multi-participant domains
  • Human(s) and robots(s) interacting on the same
    channel

11
Speech Implementation
  • Sphinx ASR engine
  • Speaker-independent recognition
  • Speaker-adaptive capability
  • Phoenix semantic parser
  • Concept extraction
  • RavenClaw dialogue engine
  • Task-based representations
  • Full mixed-initiative dialogue
  • Festival/Theta synthesis engine
  • Rosetta generation engine
  • Portable/Wearable platforms

Comms Channel
. . .
12
Relevant Research Interests
  1. Developing conversational algorithms for
    human-robot polylogue
  2. Communicating navigational information between
    humans and robots in unstructured environments
  3. Cooperative grounding of objects, locations, and
    tasks in novel environments

13
Current Status
  • Wireless network-based (UDP) communication
    protocol for human-robot interaction designed and
    implemented
  • Closed loop integration between speech-based
    command system, a Segway robot, and a Pioneer
    robot accomplished
  • 2-D kinematic simulation tool for testing
    interface to Pioneer robots
  • Video demonstration of speech-based command of a
    Segway robot and a Pioneer robot
  • Plan for year 1 scenario and relevant
    technology development and integration to
    accomplish this scenario

14
Treasure Hunt Scenario
  • First Year Objectives

15
Basic Scenario
  • 2 Segways, 2 Pioneers, and 1 human
  • Discover and return to base as much treasure as
    possible within a 20 minute period
  • Treasure will be identified by landmarks
    currently used by the Segway soccer team

Mapper (M) Seeker (S) Handler (H) Deliverer (D) Leader (L)
Humans X X X
Segways X X X
Pioneers X X X
Gators X X X
AIBOs X
16
Detailed Scenario
  • Goal
  • To combine TraderBots for negotiation and role
    assignment with plays for synchronization
  • Low-level software remains independent
  • Challenges
  • How do we decide which Play to adopt?
  • How do we handle sub-teams?
  • How do we generate leaders?

17
Step 1 Form Sub-Teams
Lets form a sub-team
Fixed sub-teams for May 2005
18
Step 2 Command to Search
Team A, search area 1
  1. Build map

19
Step 3 Explore and Search
  1. Follow
  1. Go to search area
  2. Build map

20
Step 4 Search
  1. Follow and search for treasure
  1. Execute search pattern
  2. Build map

21
Step 5 Found!
We found it!
We are at ltx,ygt
22
Step 6 Recovery
Human aided loading
23
Step 7 Return
  1. Follow
  2. Unload
  1. Return home
  2. Unload

Human aided unloading
24
Post Year 1 Goals
  • Failure detection and recovery
  • Outdoor environments
  • Adversaries
  • Larger more diverse teams
  • Adaptation and learning
Write a Comment
User Comments (0)
About PowerShow.com