Multi-Robot Teams Separating Teamwork from Taskwork

1
Multi-Robot TeamsSeparating Teamwork from
Taskwork
Gal A. KaminkaThe MAVERICK GroupComputer
Science DepartmentBar Ilan University, Israel
2
Teams of Robots and Agents

• Rescue, disaster management
• Coordinated search
• Mapping
• Exploration
• Space and science applications
• Satellite formations
• Cleaning
• Vacuum cleaning

• Surveillance, Security
• Defense, civilian
• Hazardous materials
• Sensor, ad-hoc networks
• Agriculture
• Plowing
• Medical applications
• Hospital deliveries

10-15 years ago Application-specific solutions
3
Teams of Robots and Agents
Entertainment...
4
Then, 10-15 Years ago
Team Teamwork Taskwork

• Philosophy, Computer Science Theory
• Joint intentions (Cohen Levesque 1990, 1991)
• SharedPlans (Sidner Grosz 1986, Grosz Kraus
  1996)
• Systems needing automated collaboration
• Robot foraging, coordinated movement (Parker
  1993, 1998)
• Simulated helicopter teams (Tambe 1996)

5
Teamwork and Taskwork

• Soccer Taskwork
• Kicking to a target
• Turning, tackling
• Tracking the ball, goal
• Soccer Teamwork
• Allocating players to roles
• Synchronizing tactics
• Sharing relevant information
• .

6
(No Transcript)
7
Teamwork Research

• Task-Dependent
• Teamwork

General Teamwork
8
MAVERICK Research in teamwork(See elsewhere for
plan recognition and social cognition)

• Task-Specific Teamwork
• Foraging (AIJ 07, ICRA 10)
• Coverage (ICRA 05,06,)
• Passing (FSS 01)
• Scheduling (SSS 06)
• Coordinated movement (AAAI 05,ICRA 06,JOPHA 09)
• Patrolling (ICRA 07,AAMAS 08,IJCAI 09)
• Multi-robot HRI (ICRA 06)
• .

• General Teamwork
• Monitoring (JAIR 00,02, JAAMAS, AAMAS 07,ICRA 10)
• Fault diagnosis (AAAI 98, IJCAI 03, AAAI 05, AAAI
  06)
• Synchronization (AAAI 05, ICRA 07)
• Allocation (AAMAS 03, AAAI 05, AAMAS 07)
• Learning, adaptation (DARS 06)
• Maintenance (AAMAS 07)
• .

9
MAVERICK Research in teamwork(See elsewhere for
plan recognition and social cognition)

• Task-Specific Teamwork
• Foraging (AIJ 07, ICRA 10)
• Coverage (ICRA 05,06,)
• Passing (FSS 01)
• Scheduling (SSS 06)
• Coordinated movement (AAAI 05,ICRA 06,JOPHA 09)
• Patrolling (ICRA 07,AAMAS 08,IJCAI 09)
• Multi-robot HRI (ICRA 06)
• .

• General Teamwork
• Monitoring (JAIR 00,02, JAAMAS, AAMAS 07,ICRA 10)
• Fault diagnosis (AAAI 98, IJCAI 03, AAAI 05, AAAI
  06)
• Synchronization (AAAI 05, ICRA 07)
• Allocation (AAMAS 03, AAAI 05, AAMAS 07)
• Learning, adaptation (DARS 06)
• Maintenance (AAMAS 07)
• .

10
Individual Movement in Formation
11
BDI robot controlA quick review

• Each node is a behavior (a controller)
• Nodes arranged hierarchically (to decompose
  tasks)
• Also sequentially
• Every behavior
• Preconditions
• End conditions

• A behavior graph (a)

12
Teamwork?

• Each robot executing own copy of recipe
• No agreement
• Coordination, but no collaboration

13
Principle 1Synchronizing Plan Execution

• Agents must achieve mutual belief with teammates
• Sidner Grosz Kraus 1986,1996 Cohen Levesque
  1990
• Team behaviors Start together, end together
• Jennings 1995, Tambe 96

14
Principle 2Allocate tasks to team-members

• Teammates assist each other, take over when
  failure
• Allocate tasks to members (Task Allocation)
• e.g., Shehory Kraus 1995, Dias et al. 2000.
• Anticipate needs of teammates
• Proactive Communications (Yen et al. 2001)
• Dynamically re-adjust team to address failures
• Parker 1998, Tambe 1996, 1997, Kumar et al. 2001

15
Principle 2Allocate tasks to team-members

• Teammates assist each other, take over when
  failure
• Allocate tasks to members (Task Allocation)
• e.g., Shehory Kraus 1995, Dias et al. 2000.
• Anticipate needs of teammates
• Proactive Communications (Yen et al. 2001)
• Dynamically re-adjust team to address failures
• Parker 1998, Tambe 1996, 1997, Kumar et al. 2001

16
Organization hierarchy, allocated tasks

• Behavior Graph (a)

• Team Hierarchy (b) (Tambe et al. 2000)

• Agents can find out with whom to communicate
• Agents also know who responsible for a behavior

17
Opportunities for interaction
18
BITE Bar Ilan Teamwork Engine (IAS 04, AAAI 05,
ICRA 07, AAMAS 07)

• Interaction protocols (behaviors) first-class
  objects
• Allow using different protocols at different
  times
• Insead of e.g., always vote, always ask leader,
• Context-dependent coordination policies

19
BITE Components

• Behavior Graph (a)

• Team Hierarchy (b)

• Interaction
• Behaviors (c)

20
BITE Control Algorithm

• Each robot has a copy of the three structures
• Each robot executes the following control loop
• push initial behavior onto execution stack
• repeat
• e ? current top behavior on stack
• if (e can be decomposed)
• push ALLOCATE(children(e))
• else break
• Execute stack until a behavior b terminates
• SYNC_TERM(b, children(b))
• e ? SYNC_SELECT(b, followers(b)), push e
• GOTO 2

21
BITE in Action
22
Lesson Flexibility matters (I)
80000
3500
70251
70000
3000
2900
65487
65059
60000
58183
2500
54005
53588
52730
2257
50000
2000
1831
Task Time
40000
Protocol Time
1500
30000
1284
1000
20000
836
688
Social Time
500
10000
Task Time
0
0
0
1
2
3
4
5
6
7
Configuration Same behaviors, different
protocols
23
Lesson Flexibility matters (II)
Allocation and synchronization needs for each
config.
2900
2800
2400
2000
1896
1600
Social Time
1284
1287
1200
836
800
688
544
400
361
0
0
0
0
0
0
0
1
2
3
4
5
6
7
Allocation
Groups
Synchronization
24
Patrolling
Adversarial (w/ Kraus, Agmon)
Frequency-Based (w/ Elmaliach)
ICRA 07, AMAI 10
AAMAS 08
AAMAS 08, AAMAS 09
IAS 08, IJCAI 09
Perimeter
Fence (Border)
Area
25
Patrolling
Adversarial (w/ Kraus, Agmon)
Frequency-Based (w/ Elmaliach)
ICRA 07, AMAI 10
AAMAS 08
AAMAS 08, AAMAS 09
IAS 08, IJCAI 09
Perimeter
Fence (Border)
Area
26
Frequency-Based Patrolling

• Maximize lowest frequency (F or more)
• max min F(position)
• Maximize average frequency (best average F)
• max avg F(position)
• Maximal uniformity frequency (same F)
• min stddev F(position)

27
Multi-robot, single zoneIndividual
(unsynchronized) patrol
e
b
a
d
c
28
Multi-robot, single zone Synchronized patrol
e
b
a
d
c
Theorem Synchronized patrol is better. (response
time is at most half-segment.)
29
Multi robots, multi zones (overlap)

• Internal points visited by multiple robots
• Improved uniformity everywhere except for edges
• Low frequency in edges
• Important for system design, placement of cameras

30
Responding to break-ins, faults

• Detected by the robots or operator
• Affected robot is committed
• Sends a message to others
• Other re-allocate segments
• Maintain frequency (now reduced)
• Many issues in practice
• Intermittent minor stops vs. events
• Noise in the actual velocity of the robots
• Turning points, communication ranges

31
Multi-robot, single zone(Fence Break)
32
Frequency function Realistic
model

• Let
• o-overlap factor (o1 means no overlap)
• l fence length
• r number of robots (and number of segments)
• t time it take a robot to turn
• p point (as fraction)
• v robot velocity
• si- number of robots that visit a specific
  segment I

33
Frequency function(we know math!)

• Timep)l,r,p,v,o,s,n,t,d)

34
Principle 3Organizational Maintenance

• Teammates assist each other, take over when
  failure
• Allocate tasks to members (Task Allocation)
• e.g., Shehory Kraus 1995, Dias et al. 2000.
• Dynamically re-adjust team to address failures
• Parker 1998, Tambe 1996, 1997, Kumar et al. 2001
• Anticipate needs of teammates
• Proactive Communications (Yen et al. 2001)

35
Task Formation Maintenance

• Separation-Bearing Formation Control (SBC)
• One robot assigned to lead the group (leader)
• Others maintain distance and angle to peers
  (targets)
• Each robot monitors single target
• Monitoring rules imply a control graph
• Leader node Out degree of 0
• Other nodes Out degree of 1
• Connected, path from all to leader
• Proven stable (e.g., Fierro et al. 2001)

36
Diamond FormationPossible Control Graphs
No
37
Generating an Organizational StructureControl
Graph Generation (AAAI 05, ICRA 06)

• Key idea
• Generate a multi-graph containing all possible
  control-graphs
• Then, induce a single control graph

• Multi-graph
• Vertices are robots in formation positions
• Directed edge (r1,r2) implies r1 can sense r2.
• Multiple edges between a pair of vertices allowed
• Correspond to multiple sensing options
• According to sensing morphology of robot

38
Sensing Costs Setting edge weights

• A robot can have multiple sensors (edges)
• Each sensor has range, field of view, pan angle
• Cost associated with use to monitor target
• Edge weights correspond to sensor usage costs
• Induce minimum-cost organization
• Min weight path from each vertex to leader
• Single target shortest paths (Dijkstra)

39
Re-organization Dynamic Switching of Control
Graphs

• Called when robot (or more) loses its target
• Frequent failures update costs
• Recovery algorithm updates control graph

Glick and Kaminka, ICRA 2006
40
Formation Results
No Reorganization
Reorganization
41
Time spent in formation
Better
Reorganization
No Reorganization
Significantly higher percent of time (two-tailed
t-test, p lt 2.40109e-15)
42
Time to complete task
No Reorganization
Reorganization
Better
Significantly shorter duration in all
cases (two-tailed t-test, p lt 5e-13)
43
BITE Revisited

• Common lessons
• Synchronized execution is better for task
• Balancing load among homogeneous robots
• Improved robustness, bug-free development
• 60 reduction in software engineering effort!
• Is this it?

44
MAVERICK Research

• Task-Specific Teamwork
• Foraging (ECAI 04)
• Coverage (ICRA 05,06)
• Passing (FSS 01)
• Scheduling (SSS 06)
• Coordinated movement (AAAI 05, ICRA 06)
• Patrolling (ICRA 07, 08)
• .

• General Teamwork
• Monitoring (JAIR 00,02, JAAMAS, AAMAS 07)
• Fault diagnosis (AAAI 98, IJCAI 03, AAAI 05, AAAI
  06)
• Synchronization (AAAI 05, ICRA 07)
• Allocation (AAMAS 03, AAAI 05, AAMAS 07)
• Learning, adaptation (DARS 06)
• Maintenance (AAMAS 07)
• .

45
Multi-Robot CoverageCan we separate teamwork
from taskwork?

• Goal Visit each point in an area
• Applications Floor cleaning, mowing, de-mining,
  painting .
• Many approaches, distinguishing properties
• Off-line/on-line, heuristic/complete,
  approximate/exact
• Existing literature
• Typical goal Increase efficiency
• Often, no analytically proven robustness to
  catastrophic failures
• Catastrophic Complete robot failure
• Some systems show robustness empirically

46
MSTC Multirobot Spanning-Tree Coverage(ICRA
2005, ICRA 2006, ICRA 2006, RAS 08, AMAI 09)

• Family of multi-robot coverage algorithms
• Off-line, on-line
• Backtracking, non-backtracking
• Allow for heterogeneous robots, fuel/speed
  constraints
• Common theme Guaranteed Robustness
• Will complete as long as one robot is alive

47
Spanning-Tree Coverage

• Single robot Gabriely and Rimon, 2001
• Robots with coverage tool of size D
• Grid terrains, each cell of size D
• Switch to coarse grid, each cell of size 4D
• Draw Spanning Tree (ST) on the coarse grid
• Robot moves along ST path
• A Hamiltonian cycle!

48
Non-backtracking MSTC

• Initialization phase Build STC, distribute to
  robots
• Distributed execution Each robot follows its
  section
• Low risk of collisions

Robot B is done!
Robot A is done!
Robot C is done!
49
Guaranteed robustness

• Coverage completed as long as one robot is alive
• Teamwork requirement
• Need to know task completion status of others

50
Analysis Non-backtracking MSTC

• Running time max i k step(i)
• Best case
• Worst case n k
• Unfortunately, common case
• General non-backtracking worst case n - 2(k-1) -
  1

51
Backtracking MSTC

• Similar initialization phase
• Robots backtrack to assist others
• No point is covered more than twice

52
Backtracking MSTC analysis

• Best case The same
• Worst case k2
• kgt2

53
Generating a good spanning tree(believed to be
NP-Hard)
A
A












B
B
C
C
A B 28 cells B C 4 cells C A 4
cells
A B 12 cells B C 12 cells C A 12
cells
54
A heuristic solution

• Build k subtrees on coarse grid
• Start building subtrees from initial locations
• Add cells to each subtree gradually
• Spread away from other robots (based on Manhattan
  dist)
• Connect subtrees
• Randomly pick connections between subtrees
• Calculate x in resulting tree
• Repeat ka times (a is a parameter)
• Report tree yielding minimal x

55
Illustration Stage 1






Min3,4 3
Min1,2 1
Min2,3 2
56
Example






X
13
16
17
16
57
Experimental results
58
New (and in development)Exploration for SaR,
Clearing

• Team of 3-10 robots, 1-2 operators
• Fast exploration of single floor, indoor
• Allow surveillance by human
• Provide approximate positions
• Technology-transfer goal An advanced prototype
• Buy COTS when possible
• Aim for platform-independence, standards-based
• Multiple deliverables (maximize impacts)
• Early involvement of customer, suppliers (ODF,
  Cogniteam)

59
Simulated EyeDrive
60
Challenges in Exploration

• Single human is THE monitor
• No autonomous sensor readings
• Means difficult to save exploration time
• Real-time on small fast robots
• No quad-core laptops on top.
• Ideal Vision-only

61
Adaptive Coordination(Managing
interdependence)ECAI 06, AIJ 07, ICRA 10
(Rosenfeld, Erusalimchik)

• Foraging
• Collect pucks, bring them home
• In nature
• Bees, ants
• Humans
• Many coordination methods
• Arkin Balch 1998
• Rybski et al. 1998
• Vaughan et al. 2002
• .

62
Adaptive Coordination

• Track cost of coordination
• Costs spent before, during, after collision
• e.g., time spent avoiding, time spent moving away
• Switch to cheaper methods (triggered by
  thresholds)
• Rosenfeld et al. Use offline of thresholds
• Erusalimchik et al. Online reinforcement
  learning
• Converges to optimal!
• Long economics-based discussion of why (it was a
  surprise!)

63
Adaptive Coordination Results
64
Aggression
Adaptive
65
Lesson Teamwork is about robustness

• Imagine two cars driving one behind another
• If the cars are in a convoy (a team), then
• If one of them stops, other should stop or
• Convoy leader uses signals, drives slowly, looks
  in mirror
• .
• As long as all goes well, no need for teamwork
• Simple spatial coordination is sufficient

66
LessonReuse through architectures

• Reuse in multiple applications
• New commercial tools (www.cogniteam.com)

67
An Historical Perspective on TeamworkFrom a
Single Agent to Multiple Agents
Subjective
Neatness
Scruffiness
Time
The MAS Line
'90
'94
'96
05
68
Challenges

• From coverage, patrolling, exploration
• Temporal synchronization is an invariant.
• Spatial task allocation An Invariant?
• Run-time (can we compile into behavioral rules?)
• From more complex tasks (RoboCup, exploration)
• Looser teamwork Joining, retiring, asynchronous
• Proactive teamwork, anticipation, planning
• Role of plan-recognition within teamwork model

69
Other challenges

• From working with industry (Cogniteam)
• Validation and verification, at least
  probabilistically
• Commercial significance of re-use, explicit
  teamwork
• From HART Where to publish, what to read?
• Cannot believe I missed Stefans work!
• ARMS (Autonomous Robots Workshop) _at_ AAMAS
• HART mailing list?

70
Repeating Themes

• During planning time, execution (in response to
  events)
• Allocate tasks to agents
• Synchronize task execution
• Put a human in the loop
• Robustness

71
Thank you (galk_at_cs.biu.ac.il)
Acknowledgments (subset)
Inna Frenkel Asaf Shiloni Thuc Vu Ruti
Glick Noam Hazon Noa Agmon-Segal Avi
Rosenfeld Ari Yakir Yehuda Elmaliach
Milind Tambe Sarit Kraus Manuela Veloso Meir
Kalech Dan Erusalimchik Tom Shpigelman Meirav
Hadad Gilad Armon-Kest Jared Go