Feedback Control and MultiAgent Control

1
Feedback Control and Multi-Agent Control
2
Richard Murray, Caltech
3
Types Of Control

• Passive Control
• Make structural modifications to change the plant
  dynamics
• Use this technique whenever it is a viable
  option cheap, robust
• Open Loop Control
• Exploit knowledge of system dynamics to compute
  appropriate inputs
• Requires very accurate model of plant dynamics in
  order to work well
• Active (Feedback) Control
• Use sensors and actuators connected by a computer
  to modify dynamics
• Allow uncertainty and noise to be taken into
  account

Richard Murray, Caltech
4
Richard Murray, Caltech
5
Richard Murray, Caltech
6
Richard Murray, Caltech
7
Richard Murray, Caltech
8
Richard Murray, Caltech
9
Richard Murray, Caltech
10
Richard Murray, Caltech
11
Richard Murray, Caltech
12
Richard Murray, Caltech
13
Feedback Control - Basic Concept
14
Richard Murray, Caltech
15
Feedback Control - Aircraft Landing
16
Why Stop Here?

• Controls have evolved to the digital and computer
  ages
• Communications are now broadband and ubiquitous
• So, why not combine the two and come up with
  schemes to control a multi-body system?

17
Why Multi-Agent Systems?
UAV roadmap, section 4.4, United States
Department of Defense, 2002
18
Example 1

• How to get rich in no time, with little effort,
  and totally legal?

19
Basic Agents Rules

• Initialization
• Take a 1 bill out of your wallet
• If youre not sitting next to anyone move to a
  seat next to someone
• Rules
• If you have a bill in your hand
• If you have a neighbor to your right pass the
  bill to your right
• If you have no neighbor to your right but have
  one in front of you pass the bill forward
• If you have no neighbor to your right or in front
  of you, pass the bill to the person sitting
  closest to you in front of you and to your right
• If you dont have a bill in your hand wait until
  you have one
• If you want to become rich be the person in the
  forward right corner of the class!

20
What is an Agent?
http//upload.wikimedia.org/wikipedia/en/2/2e/Ch1-
Nwanna.gif
21
What is a Multi-Agent System?

• Several agents
• Homogeneous
• Heterogeneous
• Communicating
• Cooperating
• Collaborating on a task
• Allocating resources

22
What is Swarm Intelligence?

• More common term is emergent behavior
• Swarm Intelligence is a property of systems of
  non-intelligent robots exhibiting collectively
  intelligent behavior.
• Characteristics of a swarm
• distributed, no central control or data source
• no (explicit) model of the environment
• perception of environment, i.e. sensing
• ability to manipulate the environment.

Tony Whiteemail tony_at_sce.carleton.ca
23
What is Swarm Intelligence (cont.)?

• Swarm systems are examples of behavior-based
  systems exhibiting
• multiple lower level competences
• situated in environment
• limited time to act
• autonomous with no explicit control provided
• problem solving is emergent behavior
• strong emphasis on reaction and adaptation

Tony Whiteemail tony_at_sce.carleton.ca
24
Motivations Why Use This?

• Robust nature of animal problem-solving
• simple creatures exhibit complex behavior
• behavior modified by dynamic environment.
• Emergent behavior observed in
• bacteria
• ants
• bees
• ...

Tony Whiteemail tony_at_sce.carleton.ca
25
Emergent Problem Solving

• For Lasius Niger ants, Franks, 89 observed
• regulation of 1 degree celcius range
• forming bridges
• raiding specific areas for food
• building and protecting nest
• sorting brood and food items
• cooperating in carrying large items
• emigration of a colony
• finding shortest route from nest to food source
• preferentially exploiting the richest food source
  available.

Tony Whiteemail tony_at_sce.carleton.ca
26
Swarms as Complex Systems

• A system is complex when
• It consists of a large number of elements
• Significant interactions exist between elements
• System exhibits emergent behavior cannot predict
  system behavior from analysis of individual
  elements

Traditional reductionist approaches cannot cope
with complex systems
Paolo Gaudiano, Icosystem Corporation, Cambridge,
MA
27
The Bad News

• Cannot predict emergent behavior from individual
  rules, even for a simple complex system
• Small changes in rules lead to dramatically
  different emergent behaviors

Paolo Gaudiano, Icosystem Corporation, Cambridge,
MA
28
The Good News

• It is possible to manipulate the behavior of a
  complex system by changing the rules that control
  individual elements
• We have developed a methodology to predict
  emergent behavior in complex systems using
  bottom-up simulation

Paolo Gaudiano, Icosystem Corporation, Cambridge,
MA
29
Controlling Emergent Behavior

• How can we control emergence?
• How do we define individual behaviors and
  interactions to produce desired emergent patterns?

Here is where we think the problem is...
Paolo Gaudiano, Icosystem Corporation, Cambridge,
MA
30
Bottom-up modeling

• Shift viewpoint from system (centralized) to
  individual elements (de-centralized)
• Each agent follows local rules
• Behavior depends on interactions with other
  agents
• Overall system behavior emerges from local
  interactions

Paolo Gaudiano, Icosystem Corporation, Cambridge,
MA
31
Example 2

• Ants Foraging

32
Stigmergy

• Indirect communication via interaction with
  environment Gass
• Sematonic Wilson, 75 stigmergy
• action of agent directly related to problem
  solving and affects behavior of other agents.
• Sign-based stigmergy
• action of agent affects environment not directly
  related to problem solving activity.

Tony Whiteemail tony_at_sce.carleton.ca
33
Pheromone Trails

• Species lay pheromone trails travelling from
  nest, to nest or possibly in both directions.
• pheromones evaporate.
• pheromones accumulate with multiple ants using
  path.

Tony Whiteemail tony_at_sce.carleton.ca
34
Ants Foraging
http//www.mcrit.com/Complexity/applets/boid.html
35
Applications of Ant Foraging

• It can be proven that ants foraging behavior
  solves the classical traveling salesman problem
• An NP-hard problem (in terms of algorithmic
  research)
• The ants based solution is relatively cheap and
  resources efficient
• Ants foraging behavior is used in various
  applications
• Routing in large scale networks, e.g. the
  internet
• Robots

36
Example 3

• Flocks, Herds, and Schools

37
Flocks of Birds, and Schools of Fish
dylan.winter_at_virgin.net http//www.youtube.com/wat
ch?vXH-groCeKbEfeaturerelated
38
Flocking Algorithm

• Separation steer to avoid local crowding
  flock-mates
• Alignment steer towards the average heading of
  local flock-mates
• Cohesion steer to move toward the average
  position of local flock-mates

Craig Reynolds (1987). http//www.red3d.com/cwr/bo
ids/index.html
Craig Reynolds, Flock, herds and schools a
behavioral model http//www.red3d.com/cwr/boids/i
ndex.html
39
Flocking Example in Computer Animation
40
Example 4 Maze Exploration
41
Maze Problem Definition
Start Point
Start at the Start Point End at the End Point Do
it as fast as possible Get everyone out
End Point
Original maze program was written by Rodney
Meyer, rodney_meyer_at_yahoo.com Downloaded from
http//webscripts.softpedia.com/script/Games/Matla
b-Maze-33561.html
42
The Agent We Have In Mind
Agent description taken from Professor Kumar
Slides
43
Why Multi-Agent?

• Each maze is different no a-priori knowledge
• You cant pre-program an object
• You need agents that react to the situation
• A single agent has limited and local information
  on its surroundings inside the maze
• It can be vision based sensing
• It can be tactile sensing
• It makes sense that more agents will be able to
  explore the maze faster

44
Agent Rules Exploration

• Use vision communication cues to find possible
  directions
• Rules
• Choose direction less traveled
• If more than one possible choose arbitrarily
• If all were traveled, choose arbitrarily one not
  traveled by you
• If you hit a dead-end
• Back until the nearest junction
• Mark as dead-end
• Log your location (history)
• Communication
• Notify the rest your location, possible
  directions (incl. own selection)
• When at the end, notify

45
Agent Rules Getting Out

• First one to find the exit communicates
• Its entire history
• Each of the other agent makes a plan
• Look for common history location with the first
• Find the shortest way to the last common location
  (LCL)
• Find the shortest way from the LCL to the exit
  based on the firsts history
• Then, in each step each agent follows the plan

46
Example
47
Aeronautical Applications
48
Many Types of UAV
Paolo Gaudiano, Icosystem Corporation, Cambridge,
MA
49
Possible UAV swarm applications

• Military
• Aerial surveillance, intel gathering
• SEAD
• Search Rescue
• Law Enforcement
• Urban surveillance
• Crowd control
• Civilian
• Traffic monitoring
• Powerline, bridge inspection
• Fish marine mammal tracking
• Tourism (e.g., virtual tours)
• Aerial surveys

Paolo Gaudiano, Icosystem Corporation, Cambridge,
MA
50
Controlling Multiple UAVs

• Problem Statement
• Current UAVs require at least one operator per
  UAV
• Technological advances make multi-UAV missions a
  near-term reality
• Need control strategies that allow one operator
  to monitor/control multiple UAVs

Paolo Gaudiano, Icosystem Corporation, Cambridge,
MA
51
Swarm Control of UAVsSupported by Air Force
Research Labs SBIR

• Create Agent-Based Model of UAV swarm
• Test various swarm control strategies for two
  mission types
• Search (area coverage)
• Search, track and hit targets (SEAD)
• Measure performance systematically under various
  scenarios and conditions

Paolo Gaudiano, Icosystem Corporation, Cambridge,
MA
52
Scenario
53
Simulation Modules
Environment
PAVs

Communication Input
Payload
Vehicles

Enemy

Offender list fusion
Status
Behavior
Auction
Behavior
Movement
Mission
Movement
Location
Camera Operation
Location
Movement
Location
Communication Output
Video Clips

Offender list
Abort / Continue
Quality Assessment
54
PAV Block Diagram
55
Task Assignment Behavior
56
A Flock With A Leader
57
Conclusions

• Some things become evident in our world
• Communications bandwidth is ever growing
• Computers are ubiquitous
• Problems become more complex
• Systems become more complex
• Multi-agent systems
• Are the cutting edge of this world
• Allow the user to manage this complexity and
  benefit from it
• Rely on a solid basis nature and evolution of
  species
• Why should you care?
• The next big thing (Gaudiano)

58
Richard Murray, Caltech