Distributed Control in Multiagent Systems: Design and Analysis

1
Distributed Control in Multi-agent Systems
Design and Analysis

• Kristina Lerman
• Aram Galstyan
• Information Sciences Institute
• University of Southern California

2
Design of Multi-Agent Systems

• Multi-agent systems must function in
• Dynamic environments
• Unreliable communication channels
• Large systems
• Solution
• Simple agents
• No reasoning, planning, negotiation
• Distributed control
• No central authority

3
Advantages of Distributed Control

• Robust
• tolerant of agent error and failure
• Reliable
• good performance in dynamic environments with
  unreliable communication channels
• Scalable
• performance does not depend on the number of
  agents or task size
• Analyzable
• amenable to quantitative analysis

4
Analysis of Multi-Agent Systems

• Tools to study behavior of multi-agent systems
• Experiments
• Costly, time consuming to set up and run
• Grounded simulations e.g., sensor-based
  simulations of robots
• Time consuming for large systems
• Numerical approaches
• Microscopic models, numeric simulations
• Analytical approaches
• Macroscopic mathematical models
• Predict dynamics and long term behavior
• Get insight into system design
• Parameters to optimize system performance
• Prevent instability, etc.

5
DC Two Approaches and Analyses

• Biologically-inspired approach
• Local interactions among many simple agents leads
  to desirable collective behavior
• Mathematical models describe collective dynamics
  of the system
• Markov-based systems
• Application collaboration, foraging in robots
• Market-based approach
• Adaptation via iterative games
• Numeric simulations
• Application dynamic resource allocation

6
Biologically-Inspired Control
7
Analysis of Collective Behavior

• Bio control modeled on social insects
• complex collective behavior arises in simple,
  locally interacting agents
• Individual agent behavior is unpredictable
• external forces may not be anticipated
• noise fluctuations and random events
• other agents with complex trajectories
• probabilistic controllers e.g. avoidance
• Collective behavior described probabilistically

8
Some Terms Defined

• State - labels a set of agent behaviors
• e.g., for robots Search State Wander, Detect
  Objects, Avoid Obstacles
• finite number of states
• each agent is in exactly one of the states
• Probability distribution
• probability system is in
  configuration n at time t
• where Ni is number of
  agents in the i th of L states

9
Markov Systems

• Markov property configuration at time tDt
  depends only on configuration at time t
• also,
• change in probability density

10
Stochastic Master Equation

• In the continuum limit,
• with transition rates

11
Rate Equation

• Derive the Rate Equation from the Master Eqn
• describes how the average number of agents in
  state k changes in time
• Macroscopic dynamical model

12
Collaboration in Robots
13
Stick-Pulling Experiments (Ijspeert, Martinoli
Billard, 2001)
A. Ijspeert et al.

• Collaboration in a group of reactive robots
• Task completed only through collaboration
• Experiments with 2 6 Khepera robots
• Minimalist robot controller

14
Experimental Results

• Key observations
• Different dynamics for different ratio of robots
  to sticks
• Optimal gripping time parameter

15
(No Transcript)
16
Model Variables

• Macroscopic dynamic variables
• Ns(t) number of robots in search state at time
  t
• Ng(t) number of robots gripping state at time t
• M(t) number of uncollected sticks at time t
• Parameters
• connect the model to the real system
• a rate of encountering a stick
• aRG rate of encountering a gripping robot
• t gripping time

17
Mathematical Model of Collaboration
18
Dimensional Analysis

• Rewrite equations in dimensionless form by making
  the following transformations
• only the parameters b and t appear in the eqns
  and determine the behavior of solutions
• Collaboration rate
• rate at which robots pull sticks out

19
Searching Robots vs Time
t5 b0.5
20
Searching Robots vs t
b1.5
b1.0
b0.5
21
Collaboration Rate vs t

• Key observations
• critical b
• optimal gripping time parameter

22
Comparison to Experimental Results
Ijspeert et al.
23
Summary of Results

• Analyzed the system mathematically
• importance of b
• analytic expression for bc and topt
• superlinear performance
• Agreement with experimental data and simulations

24
Foraging in Robots
25
Robot Foraging

• Collect objects scattered in the arena and
  assemble them at a home location
• Single vs group of robots
• no collaboration
• benefits of a group
• robust to individual failure
• group can speed up collection
• But, increased interference

Goldberg Mataric
26
Interference Collision Avoidance

• Collision avoidance
• Interference effects
• robot working alone is more efficient
• larger groups experience more interference
• optimal group size beyond some group size,
  interference outweighs the benefits of the
  groups increased robustness and parallelism

27
State Diagram
start
look for pucks
object detected?
obstacle?
avoid obstacle
grab puck
go home
28
Model Variables

• Macroscopic dynamic variables
• Ns(t) number of robots in search state at time
  t
• Nh(t) number of robots in homing state at time
  t
• Nsav(t), Nhav(t) number of avoiding robots at
  time t
• M(t) number of undelivered pucks at time t
• Parameters
• ar rate of encountering a robot
• ap rate of encountering a puck
• t avoiding time
• th0 homing time in the absence of interference

29
Mathematical Model of Foraging
Initial conditions
30
Searching Robots and Pucks vs Time
robots
pucks
31
Pucks vs Time
N05
N015
N020
32
Group Efficiency vs Group Size
t1
t5
33
Sensor-Based Simulations

• Player/Stage simulator
• number of robots 1 - 10
• number of pucks 20
• arena radius 3 m
• home radius 0.75 m
• robot radius 0.2 m
• robot speed 30 cm/s
• puck radius 0.05 m
• rev. hom. time 10 s

34
Simulations Results
35
Simulations Results
36
Summary

• Biologically inspired mechanisms are feasible for
  distributed control in multi-agent systems
• Methodology for creating mathematical models of
  collective behavior of MAS
• Rate equations
• Model and analysis of robotic systems
• Collaboration, foraging
• Future directions
• Generalized Markov systems integrating
  learning, memory, decision making

37
Market-Based Control
38
Distributed Resource Allocation

• N agents use a set of M common resources with
  limited, time dependent capacity LM(t)
• At each time step the agents decide whether to
  use the resource m or not
• Objective is to minimize the waste
• where Am(t) is the number of agents utilizing
  resource m

39
Minority Games

• N agents repeatedly choose between two
  alternatives (labeled 0 and 1), and those in the
  minority group are rewarded
• Each agent has a set of S strategies that
  prescribe a certain action given the last m
  outcomes of the game (memory)

strategy with m3
input
action

• Reinforce strategies that predicted the winning
  group
• Play the strategy that has predicted the winning
  side most often

40
MG as a Complex System

• Let be the size of the group that chooses
  1 at time t
• The waste of the resource is measured by the
  standard deviation
• 
  - average over time
• In the default Random Choice Game (agents take
  either action with probability ½) , the standard
  deviation is

41
Variations of MG

• MG with local information
• Instead of global history agents may use local
  interactions (e.g., cellular automata)
• MG with arbitrary capacities
• The winning choice is 1 if
  where is the capacity, is
  the number of agents that chose 1

To what degree agents (and the system as a whole)
can coordinate in externally changing environment?
42
MG on Kauffman Networks

• Set of N Boolean agents

• Each agent has
• A set of K neighbors
• A set of S randomly chosen Boolean functions of K
  variables

• Dynamics is given by

• The winning choice is 1 if
  where

43
Simulation Results
K2 networks show a tendency towards
self-organization into a coordinated phase
characterized by small fluctuations and effective
resource utilization
44
Results (continued)
Coordination occurs even in the presence of
vastly different time scales in the environmental
dynamics
45
Scalability
For K2 the variance per agent is almost
independent on the group size,

In the absence of coordination
46
Phase Transitions in Kauffman Nets
Kauffman Nets phase transition at K2 separating
ordered (Klt2) and chaotic (Kgt2) phases
For Kgt2 one can arrive at the phase transition by
tuning the homogeneity parameter P (the fraction
of 0s or 1s in the output of the Boolean
functions)
The coordinated phase might be related to the
phase transition in Kauffman Nets.
47
Summary of Results

• Generalized Minority Games on K2 Kauffman Nets
  are highly adaptive and can serve as a mechanism
  for distributed resource allocation
• In the coordinated phase the system is highly
  scalable
• The adaptation occurs even in the presence of
  different time scales, and without the agents
  explicitly coordinating or knowing the resource
  capacity
• For Kgt2 similar coordination emerges in the
  vicinity of the ordered/chaotic phase transitions
  in the corresponding Kauffman Nets

48
Conclusion

• Biologically-inspired and market-based mechanisms
  are feasible models for distributed control in
  multi-agent systems
• Collaboration and foraging in robots
• Resource allocation in a dynamic environment
• Studied both mechanisms quantitatively
• Analytical model of collective dynamics
• Numeric simulations of adaptive behavior