MultiAgent Exploration in Unknown Environments

1
Multi-Agent Exploration in Unknown Environments

• Changchang Wu
• Nov 2, 2006

2
Outline

• Why multiple robots
• Design issues
• Basic approaches
• Distributed
• Centralized
• Market-based

3
Why Multiple Robots

• Some tasks require a robot team
• Have potential to finish tasks faster
• Increase robustness w/ redundancy
• Compensate sensor uncertainty by merging
  overlapping information
• Multiple robots allow for more varied and
  creative solutions

4
A Good Multi-Robot System Is

• Robust no single point of failure
• Optimized, even under dynamic conditions
• Quick to respond to changes
• Able to deal with imperfect communication
• Able to avoid robot interference
• Able to allocate limited resources
• Heterogeneous and able to make use of different
  robot skills

5
Basic Approaches

• Distributed
• Every robot goes for itself
• Centralized
• Globally coordinate all robots
• Market-based
• Analogy To Real Economy

6
Distributed Methods

• Planning responsibility spread over team
• Each robot basically act independently
• Robots use locally observable information to
  coordinate and make their plans

7
Example Frontier-Based Exploration Using
Multiple Robots (Yamauchi 1998)

• A highly distributed approach
• Simple idea To gain the most new information
  about the world, move to the boundary between
  open space and uncertainty territory
• Frontiers are the boundaries between open space
  and unexplored space

8
Occupancy Grid

• World is represented as grid
• Each cell in the grid is assigned with a
  probability of being already occupied/observed
• The initial probability is all set to .5
• Cell status can be Open (lt0.5), Unknown (0.5) or
  Occupied (gt0.5)
• Bayesian rule is used to update cells by merging
  information from each sensor reading (sonar)

9
Frontier Detection

• Frontier Boundary between open and unexplored
  space.
• Any open cell adjacent to unknown cell is
  frontier edge cell.
• Frontier cells grouped into frontier regions
  based on adjacency.
• Accessible frontier Robot can pass through
  opening.
• Inaccessible frontier Robot cannot pass through
  opening.

10
Multi-Robot Navigation

• Simple algorithm Each robot goes along the
  shortest obstacle free path to a frontier region
• Robots share a common map All information
  obtained by any robot is available to all robots
• Robots are planning path independently
• Use reactive strategy to avoid collisions
• Robots may waste time for the same frontiers

11
An Exploration Sequence
12
Distributed Methods Pros Cons

• Pros
• Very robust. No single point failure
• Fast response to dynamic conditions
• Little or no communication is required
• Easy.Little computation required
• Cons
• Plans only based on local information
• Solutions are often sub-optimal

13
Centralized Methods

• Robot team treated as a single system with many
  degrees of freedom
• A single robot or computer is the leader
• Leader plans optimal tasks for groups
• Group members send information to leader and
  carry out actions

14
Example Arena (Jia 2004)

• Robots share a common map and only communicate
  with a leader
• Robots compete for resources by their efficiency
• leader greedily assigns the most efficient tasks
• Leader coordinate robots to handle interference

15
Background

• World representation
• Occupancy grid
• Cost unit
• Moving forward one step Turning 45 degrees
• Cost overflow
• Similar to minimum cost spanning tree
• Easy to compute the shortest path
• Easy to handle obstacle

16
Cost Overflow
Direction priority
Cost of 45 turning Cost of one cells step
17
Goal Candidates Detection

• A goal point P should satisfy
• P is passable (Mark the cells in warning range or
  obstacles/Wall/Unknown cells as impassable)
• Some unexplored cells lie in the circle with P as
  the center and (R K) as the radium, where R is
  the warning radius and K is usually 1

18
Goal Resource

• Reserved goal candidates
• Robots obtained by competition
• Recessive goal candidates
• The goal points in a given range to a reserved
  goal point
• This distance can be adjusted

Goal candidates
Recessive goals candidates
19
Path Resource

• Path resource is a time-space term
• For a given time, the cells close to any robot
  are marked off for safety
• Looks just like a widened path
• Basically a reactive strategy

goal
path
resource
20
Revenue and Utility

• Revenue
• The expected gain of information that robots
  observe at a goal point
• Utility used by many other approaches
• Utility revenue cost
• Utility in this paper
• Utility Revenue / Cost
• Better connected to purpose of smallest cost
• No need to care about unit conversion

21
Greedy Goal Selection

• Try to maximize the global utility
• Coordination robots obtain goal and path
  resources exclusively
• Competition repetitively select the pair of free
  agent and goal with highest utility
• Sub-optimal

22
Simple Algorithm

• Repeat until map is complete
• Repeat free robots times
• Cost computation (Also make sure no interference
  with the busy robots)
• Select the highest utility task (Compete)
• Mark off the associated robot and goal points,
  and nearby goal points

23
1st Competition
Interval 3
Competitor
24
1st Competition Result
6
6
5
5
4
4
Interval 3
Competitor
25
2nd Competition
4
4
4
4
4
4
4
4
4
3
3
3
2
2
2
1
1
Interval 3
Competitor
Satisfied
26
2nd Competition Result
4
4
4
4
4
4
4
4
4
3
3
3
2
2
2
1
1
Interval 3
Competitor
Satisfied
27
3rd Competition
6
6
6
6
6
6
4
4
4
6
6
6
4
4
4
5
5
5
4
4
4
2
4
4
4
3
3
3
2
3
3
3
2
2
2
2
2
2
1
1
1
Competitor
Satisfied
Interval 3
28
3rd Competition Result
6
1
13
6
6
6
2
9
10
11
12
4
4
4
3
8
4
3
4
7
3
2
2
5
6
1
1
Competitor
Satisfied
Interval 3
29
Planning Issues

• Do not transfer a reserved goal point to another
  free agent (unless necessary). Frequent change of
  tasks can cause localization error.
• Quit an assigned task when the goal point is
  unexpectedly observed by other robots
• Schedule at most one task for each agent

30
Possible Variations

• Still keep busy agents in competition. Remove the
  goal resources they win from competition.
• This prevents those goal resources being assigned
  to other agents
• It is too early to burden a new task on a robot
  who has not achieved it current task
• No need to schedule them.
• New resources probably will be found when they
  reach the goals

31
Handling Failure of Planning

• It may fail to plan safe paths
• When some robot get to a place where
• it is almost too close to other robot
• it has no good space to detour
• And it choose to just wait there for other robots
  to move away, which is not known by other robots
• Avoidance of unexpected obstacle
• Robots have simple reactive mechanism
• Release resources and try to gain new task

32
Fail to plan safe paths
collision
Competitor
Satisfied
Interval 3
33
Reactive Mechanism
Competitor
Satisfied
Interval 3
34
Exchange Tasks
Competitor
Satisfied
Interval 3
35
Some Statistics
36
Demo
37
Centralized Methods Pros

• Leader can take all relevant information into
  account for planning
• Optimal s islution possible!
• One can try different approximate solutions to
  this problem

38
Centralized Methods Cons

• Optimal solution is computationally hard
• Intractable for more than a few robots
• Makes unrealistic assumptions
• All relevant info can be transmitted to leader
• This info doesnt change during plan construction
• Vulnerable to malfunction of leader
• Heavy communication load for the leader

39
Market-Based Methods

• Based on market architecture
• Each robot seeks to maximize individual profit
• Robots can negotiate and bid for tasks
• Individual profit helps the common good
• Decisions are made locally but effects approach
  optimality
• Preserves advantages of distributed approach

40
Why Is This Good?

• Robust to changing conditions
• Not hierarchical
• If a robot breaks, tasks can be re-bid to others
• Distributed nature allows for quick response
• Only local communication necessary
• Efficient resource utilization and role adoption
• Advantages of distributed system with optimality
  approaching centralized system

41
Architecture

• World is represented as a grid
• Squares are unknown (0), occupied (), or empty
  (-)
• Goals are squares in the grid for a robot to
  explore
• Goal points to visit are the main commodity
  exchanged in market
• For any goal square in the grid
• Cost based on distance traveled to reach goal
• Revenue based on information gained by reaching
  goal
• R ( of unknown cells near goal) x (weighting
  factor)
• Team profit sum of individual profits
• When individual robots maximize profit, the whole
  team gains

42
Example World
43
Goal Selection Strategies

• Possible strategies
• Randomly select points, discard if already
  visited
• Greedy exploration
• Choose goal point in closest unexplored region
• Space division by quadtree

44
Exploration Algorithm

• Algorithm for each robot
• Generate goals (based on goal selection strategy)
• If OpExec (human operator) is reachable, check
  with OpExec to make sure goals are new to colony
• Rank goals greedily based on expected profit
• Try to auction off /bid goals to each reachable
  robot
• If a bid is worth more than you would profit from
  reaching the goal yourself (plus a markup), sell
  it

45
Exploration Algorithm

• Once all auctions are closed, explore
  highest-profit goal
• Upon reaching goal, generate new goal points
• Maximum of goal points is limited
• Repeat this algorithm until map is complete

46
Bidding Example

• R1 auctions goal to R2

47
Expected vs. Real

• Robots make decisions based on expected profit
• Expected cost and revenue based on current map
• Actual profit may be different
• Unforeseen obstacles may increase cost
• Once real costs exceed expected costs by some
  margin, abandon goal
• Dont get stuck trying for unreachable goals

48
Information Sharing

• If an auctioneer tries to auction a goal point
  already covered by a bidder
• Bidder tells auctioneer to update map
• Removes goal point
• Robots can sell map information to each other
• Price negotiated based on information gained
• Reduces overlapping exploration
• When needed, OpExec sends a map request to all
  reachable robots
• Robots respond by sending current maps
• OpExec combines the maps by adding up cell values

49
Advantages of Communication

• Low-bandwidth mechanisms for communicating
  aggregate information
• Unlike other systems, map info doesnt need to be
  communicated repeatedly for coordination

50
What Is a Robot Doing

• Goal generation and exploration
• Sharing Information with other robots
• Report information to OpExec at some frequency

51
Experimental Setup

• 4 or 5 robots
• Equipped with fiber optic gyroscopes
• 16 ultrasonic sensors

52
Experimental Setup

• Three test environments
• Large room cluttered with obstacles
• Outdoor patio, with open areas as well as walls
  and tables
• Large conference room with tables and 100 people
  wandering around
• Took between 5 and 10 minutes to map areas

53
Experimental Results
54
Experimental Results
55
Experimental Results

• Successfully mapped regions
• Performance metric (exploration efficiency)
• Area covered / distance traveled m2 / m
• Market architecture improved efficiency over no
  communication by a factor of 3.4

56
Conclusion

• Market-based approach for multi-robot
  coordination is promising
• Robustness and quickness of distributed system
• Approaches optimality of centralized system
• Low communication requirements
• Probably not perfect
• Cost heuristics can be inaccurate
• Much of this approach is still speculative
• Some pieces, such as leaders, may be too hard to
  do

57
In Sum

• Distributed vs. centralized mapping
• Distributed vs. centralized planning
• Revenue/Cost vs. Revenue Cost
• Often sub-optimal solutions
• No common evaluation system for comparisons

58
References

• Yamauchi, B., "Frontier-Based Exploration Using
  Multiple Robots," In Proc. of the Second
  International Conference on Autonomous Agents
  (Agents98), Minneapolis, MN., 1998.
• Menglei Jia , Guangming Zhou ,Zonghai Chen,
  "Arenaan Architecture for Multi-Robot
  Exploration Combining Task Allocation and Path
  Planning, 2004
• Zlot, R., Stentz, A., Dias, M. B., and Thayer, S.
  Multi-Robot Exploration Controlled By A Market
  Economy. Proceedings of the IEEE International
  Conference on Robotics and Automation, 2002.
• http//voronoi.sbp.ri.cmu.edu/presentations/motion
  planning2001Fall/FrontierExploration.ppt
• http//www.ai.mit.edu/courses/16.412J/lectures/adv
  anced20lecture_11.6.ppt
• http//mail.ustc.edu.cn/jml/jml.files/Arena.ppt