Distributed Control of Multiagent Systems: From Engineering to Economics

1
Distributed Control of Multiagent Systems From
Engineering to Economics
Prof. William Dunbar Autonomous Systems
Group Computer Engineering
2
What are Systems? ANYTHING in Engineering,
usually with Dynamics.

• Some familiar examples
• How do we describe (or predict) systems?
• with math!

(Images courtesy of http//www.cds.caltech.edu/mu
rray/cdspanel/ unless stated otherwise)
3
Math Describing Diverse Engineering Systems in a
Common Way
4
Control Systems are Hidden Engineering Systems

• A Control System is a device in which a sensed
  quantity is used to modify the behavior of a
  system through computation and actuation.

5
Abstraction Multiagent Systems

• The Internet
• Air traffic control
• Supply Chain

• Control Problems with
• Subsystem dynamics
• Shared resources (constraints)
• Communications topology
• Shared objectives

(SC image courtest of www.vipgroup.us)
6
Multiagent Systems Distributed and (presumed)
Cooperative

• Multiagent System
• autonomous agents
• communication network

Distributed local decisions based on local
information.
Cooperative agents agree on roles dynamically
coordinate.
7
A Relevant Decision Method Model Predictive
Control (MPC)
MPC uses optimization to find feasible/optimal
plans for near future.
To mitigate uncertainty, plan is revised after a
short time.
computed
8
Mathematics of MPC is Finite Horizon Optimal
Control
9
Convergence of MPC Requires Appropriate Planning
Horizon
Theoretical conditions sufficient in absence of
explicit uncertainty.
Mayne et al., 2000
10
MPC Compared to Other Techniques

• Gives planning feedback with built-in
  contingency plans.
• Only technique that handles state and control
  constraints explicitly.
• Tradeoff computationally intensive.

11
MPC Successful in Applications Process to
Flight Control
Caltech flight control experiment Tracking ramp
input of 16 meters in horizontal, step input of
1m in altitude. MPC updates at 10 Hz,
trajectories generated by NTG software package.
Movie
12
MPC Admits Cooperation
Get 1 to pump, 2 follow 1 3 follow 2.
Decoupled dynamics
Avoid collision
13
MPC of Multiagent Systems Whats Missing?
Enables autonomy of single agent.
Amenable to cooperation for multiple agents.
14
My Contribution A Distributed Implementation of
MPC
Distributed local decisions based on local
information.
Decoupled subsystem dynamics/constraints, Coupled
cost L
15
Solution of Sub-problems requires Assumed Plan
for Neighbors
Agent 3 ?
16
Compatibility of Actual and Assumed Plans via
Constraint
Compatibility constraint
Assumed plan
17
Distributed Implementation Requires Synchrony
Common Horizon T
18
Conditions for Theory are General
19
Convergence Conditions Centralized plus Bound on
Update Period
Dunbar Murray, Accepted to Automatica, June,
2004
20
Venue Multi-Vehicle Fingertip Formation
21
Simulation Parameters
4
2
22
Centralized MPCBenchmark for Comparison
23
Centralized MPC Simulation
24
Distributed MPC is Comparable to Centralized MPC
25
Distributed MPC Simulation
26
Naive Approach Produces Less Desirable Performance
27
Naïve Approach Bad Overshoot
28
Summary of Contribution

• Distributed implementation of MPC is provable
  convergent, performs well, and is applicable to a
  class of Multiagent Systems
• Distributed cooperative structure
• Local decisions based on local information
• Decomposition and incorporation of compatibility
  constraint
• Coordination via sharing feasible plans
• Applicable for
• Heterogeneous nonlinear dynamics
• Generic objective function (need not be
  quadratic)
• Coupling constraints and coupled dynamics

29
Supply Chain Management (SCM) is an Attractive
Venue for DMPC

• Dynamics (Linear/Nonlinear) s.t. constraints and
  moving set points.
• Forecasts of measurable inputs often available,
  which MPC can easily incorporate.
• Dynamic time scales and inter-stage communication
  BW are not limiting factors.
• Active research area. Why? Companies dont
  compete - their supply chains do. Thus, SCM will
  make or break companies. Examples Dell, Walmart.

Challenge distributed (asynchronous)
coordination in the presence of time delays.
30
Overview

1. Define three stage SCM problem from supply chain
   literature
2. Distributed Problem gt Distributed MPC
   Implementation
3. Nominal decentralized feedback policy from supply
   chain literature
4. Numerical Experiments for Comparison
5. Conclusions and Extensions

31
SCM Information Flows Upstream (orders) and
Material Flows Downstream (goods)
Three Stages Supplier S, Manufacturer M,
Retailer R
UP stream
DOWN stream
32
Bi-drectional Coupling in the Dynamics
For each stage Dynamics
Constraints Coupling x depends on downstream
order rate upstream backlog Objective Keep
stock and unfulfilled order at desired levels
33
DMPC Parallel Updates Assuming Remainder of
Previous Response for Neighbors
Q-cost with move suppression
34
Experiments Show Comparable Performance with
Nominal Policy Single Stage Case
Nominal Devised to match observed responses
Response to initial stock offset Standard
MPC (not DMPC)
35
Step in Demand Rate Comparable Performance
Advantage of Anticipation
Add anticipation
36
Three Stage with Pulse in Customer Demand
Comparable then Better with Anticipation
Nominal
DMPC
37
Conclusions and Extensions

• Realistic SCM problem (classic MIT Beer Game)
• DMPC comparable to validated nominal feedback
  policy.
• Clear advantage when customer demand can be
  reliably forecasted (anticipation).
• A detailed relative degree, controllability and
  stabilizability analysis to come. Unfulfilled
  order in stages M and R exhibited nonzero
  steady-state error.
• Next leap multi-echelon chains - at least two
  (and possibly many) players operate within each
  stage, e.g., the S stage in Dell's
  build-to-order" supply chain management
  strategy might contain several chip suppliers
  such as Samsung, Intel and Micron.
• Extend theory asynchronous time conditions.