Distributed Control The Importance of Signals and Boundaries

1
Distributed ControlThe Importance of Signals and
Boundaries??
Nothing is less real than realism. Details are
confusing. It is only by selection, by
elimination, by emphases that we get at the real
meaning of things. --Georgia
OKeefe
2
Outline

• Introduction to Distributed Control. (pp. 3-14)
• Reaction Networks. (pp. 15-25)
• Urn? models of uncontrolled reactions. (pp.
  26-31)
• Hierarchical?? control. (pp. 32-42)
• Markov models of hierarchical control. (pp.
  43-48)

3

Introduction to Distributed Control (1)
In traditional control theory An external
controller directs a system (plant) using
signals generated by the plant and external
feedback.Distributed Control occurs when the
control originates internally through interaction
of components of the system Networks of
interaction (Reaction Networks) have a central
role in the study of Distributed Control.
4

Introduction to Distributed Control (2) Adaptive
Control contrasted to Distributed Control
Adaptive Control
Fitness 1/operating cost
Distributed Control
Different agents have different
controls. Connections between agents are
frequently made and broken.
5

Introduction to Distributed Control (3)
Adaptive control in many complex systems, such as
flight controllers, is often implemented by a
program -- a linked set of IF/THEN rules. As we
will see, reaction nets can also be represented
by a linked IF/THEN rules, and they are
distributed. Reaction nets are a natural way to
study distributed control.
6
Introduction to Distributed Control (4)
Reactions in biological cells are primarily
controlled through enzymes? (catalysts???)
and a hierarchy of enclosures by semi-permeable
membranes (selective filters). This hierarchy
yields adaptive control orders of magnitude
better than we can obtain with artificial
systems. The details of the hierarchy are
immensely complex. Even comparative
physiology?????, comparing very simple cells to
much more complex organisms, gives only vague
ideas about the hierarchy.
?-???
7
Introduction to Distributed Control (5)
Spontaneous??? Emergence of Levels
Spore??
water
differentiation of cells ????
single cell
no water
aggregation of cells
Slime mold (Discoideum) ??? When the
environment is unfavorable, individual cells
aggregate, form boundaries??, and specialize.
8
Introduction to Distributed Control (6)
Genetically-specified signal/boundary
interactions in plants mitochondria???(ener
gy conversion), chloroplasts
(photosynthesis????) mycorrhizal fungi??
(nutrient and water uptake) debris-decomposing
bacteria (recycling) pollinators??? (e.g.,
bees), seed-transporting organisms (e.g.,
fruit eaters) predators ??? (e.g., plant
eaters) Signals and boundaries control the
network of interactions at each level of the
hierarchy. It is difficult to analyze these
interactions using traditional mathematics.
?
9
Introduction to Distributed Control (6)
Genetically-specified signal/boundary
interactions in plants mitochondria???(ener
gy conversion), chloroplasts
(photosynthesis????) mycorrhizal fungi??
(nutrient and water uptake) debris-decomposing
bacteria (recycling) pollinators??? (e.g.,
bees), seed-transporting organisms (e.g.,
fruit eaters) predators ??? (e.g., plant
eaters) Signals and boundaries control the
network of interactions at each level of the
hierarchy. It is difficult to analyze these
interactions using traditional mathematics.
?
10
Introduction to Distributed Control (6)
Genetically-specified signal/boundary
interactions in plants mitochondria???(ener
gy conversion) chloroplasts (photosynthesis???
?) mycorrhizal fungi?? (nutrient and water
uptake) debris-decomposing bacteria
(recycling) pollinators??? (e.g., bees),
seed-transporting organisms (e.g., fruit eaters)
predators ??? (e.g., plant eaters) Signals
and boundaries control the network of
interactions at each level of the
hierarchy. It is difficult to analyze these
interactions using traditional mathematics.
?
11
Introduction to Distributed Control (6)
Genetically-specified signal/boundary
interactions in plants mitochondria???(ener
gy conversion) chloroplasts
(photosynthesis????) mycorrhizal fungi??
(nutrient and water uptake) debris-decomposing
bacteria (recycling) pollinators??? (e.g.,
bees), seed-transporting organisms (e.g.,
fruit eaters) predators ??? (e.g., plant
eaters) Signals and boundaries control the
network of interactions at each level of the
hierarchy. It is difficult to analyze these
interactions using traditional mathematics.
?
12
Introduction to Distributed Control (6)
Genetically-specified signal/boundary
interactions in plants mitochondria???(ener
gy conversion) chloroplasts
(photosynthesis????) mycorrhizal fungi??
(nutrient and water uptake) debris-decomposing
bacteria (recycling) pollinators??? (e.g.,
bees) seed-transporting organisms (e.g.,
fruit eaters) predators ??? (e.g., plant
eaters) Signals and boundaries control the
network of interactions at each level of the
hierarchy. It is difficult to analyze these
interactions using traditional mathematics.
?
13
Introduction to Distributed Control (6)
Genetically-specified signal/boundary
interactions in plants mitochondria???(ener
gy conversion) chloroplasts (photosynthesis???
?) mycorrhizal fungi?? (nutrient and water
uptake) debris-decomposing bacteria
(recycling) pollinators??? (e.g., bees),
seed-transporting organisms (e.g., fruit eaters)
predators ??? (e.g., plant eaters) Signals
and boundaries control the network of
interactions at each level of the
hierarchy. It is difficult to analyze these
interactions using traditional mathematics.
?
14
Introduction to Distributed Control (6)
Genetically-specified signal/boundary
interactions in plants mitochondria???(ener
gy conversion) chloroplasts (photosynthesis???
?) mycorrhizal fungi?? (nutrient and water
uptake) debris-decomposing bacteria
(recycling) pollinators??? (e.g., bees),
seed-transporting organisms (e.g., fruit eaters)
predators ??? (e.g., plant eaters) Signals
and boundaries control the network of
interactions at each level of the
hierarchy. It is difficult to analyze these
interactions using traditional mathematics.
?
15
Introduction to Distributed Control (7)
Even beyond biology, it is important to
understand hierarchical distributed control
because it occurs in many different complex
adaptive systems (cas).
Systems where signal/boundary interactions are
critical


Biological cell (chromosome/protein
communication organelles)



Ecosystem



Geopolitics ????



Reaction network (with phases and/or membranes?)



Central Nervous System regions



Language



Markets (tags)



Psychology (induction and discovery)

16
Introduction to Distributed Control
(8)Spontaneous Boundary Formation
Before
????????????,????????????????
A boundary is the interface between two distinct
sets of reactants. (E.g., the boundary between
Mandarin??? and Cantonese???.)
After
Later, boundaries will be represented by tagged
urns.
17
Introduction to Distributed Control (9)
As is usual in theoretical physics, we try to use
simplified exploratory models to obtain ideas
about where to look in real systems. Then we
formulate critical experiments to test our
hypotheses. As we will see, tagged urn models
??????? are helpful in studying distributed
control.
18
Introduction to Distributed Control (10)

• Tagged urn models of hierarchical control offer
  the following
• advantages
• 1) Complicated hierarchies of enclosures and
  distributed control are
• easily presented.
• There is a relevant mathematics, Markov
  processes, and a relevant
• search technique, the Monte Carlo
  algorithm.
• 3) Using Monte Carlo algorithms, it is easy to
  find communities of
• reactants??? (niches???) --
  communities that persist
• because of recirculation??? and
  autocatalysis???.
• 4) Because all reactants and membranes are
  presented as strings, the
• adaptive co-evolution of tagged urn
  models is easily simulated
• using a genetic algorithm.

19
Introduction to Distributed Control (11)
In a complex adaptive system (cas) , where there
are multiple interacting agents that learn ,
control is necessarily distributed. Boundaries
separate agents into communities (modularity),
allowing specialization and higher
efficiency. Signals allow one community to
partially control others. Signals and Boundaries
are both modified by evolution.
20
Introduction to Distributed Control (12)
Distributed Control of Reaction Nets

• All cas can be presented as networks of reactions
  (rule based agents) and
• resource flows.
• Reaction networks give us a single formalism for
  studying both cas and distributed control.
• After reviewing reaction networks we will see
  how boundaries and signals make possible
  distributed control of networks.

21
Overview of Reaction Networks (1)

• Reaction networks result from spatially
  distributed sets of interactions
• between molecules, signals, and/or resources.
• E.g., a collection of reactions of the kind AB
  F CD distributed
• over a 2-dimensional geometry.

22
Overview of Reaction Networks (2)

• The reactants ??? (e.g. molecules) have tags
  ?? (active sites) that
• determine the reactions that take place.
• Reactants come into contact through random
  elastic collisions.
• Billiard-ball ?? mechanics.
• Reactants in contact react with a probability
  determined by the reaction rate.
• An urn model ??? of the state of the
  reaction network allows a Markov matrix
  analysis.

23
Overview of Reaction Networks (3) Objectives

• 1) Observe the robustness and organization of
  the evolving reaction networks.
• 2) Observe the kinds of agents (coordinated
  sets of reactions), if any, that form through the
  evolution of tags.
• Develop a concept of niche ??? (local resource
  enrichment) suitable for dynamic, perpetually
  ??? changing networks.
• These objectives have a major role in
  understanding any cas.

24
Overview of Reaction Networks (4)Emergent
Phenomena Expected

• 1) Boundaries arise through constraints provided
  by tags on reactants.
• Boundaries allow the formation of agents
  with individual histories.
• 2) Agents become building blocks for still more
  complex agents.
• A layered use of tags evolves (similar to the
  membrane, organelle, cell, organ, hierarchy of
  biological cells).
• 3) Cycles in the reaction net provide locally
  increased concentrations ?? of
• reactants.
• The resulting niches ??? offer possibilities
  for agent exploitation.
• 4) Increasing diversity of the rules, signals,
  and agents arises as the reaction
• network evolves (under a genetic algorithm).

25
A Quick Review of Reaction Networks (1)
Reactants

• Reactants are classified according to active
  sites (tags).
• Reactants diffuse and collide at random (a
  billiard ball mechanics), undergoing mass
• action chemistry depending upon their active
  sites.

26
A Quick Review of Reaction Networks(2) A Binary
Reaction
R1 R2 F R3 R4
27
A Quick Review of Reaction Networks (3) A
Binary Reaction Based on Tags
???
??
prefix ??
28
A Quick Review of Reaction Networks (4) A
Sequence of Binary Reactions
29
A Quick Review of Reaction Networks (5) The
Effect of Recycling (Material Feedback)
30
A Quick Review of Reaction Networks (9)
Summarizing Reactions, Conditions, Tags

• A reaction requires a given combination of tags
  (active sites) for each input reactant any
  reactant with that combination can serve as that
  input.
• The combination of tags required for a reaction
  can be specified by using using dont care ()
  symbols for the non-tag locations (as in a
  classifier rule).

31
A Quick Review of Reaction Networks (7)
Embedding a Simple Ecosystem in a Reaction Network
Substrate ??? a, b, c,
multiple copies of each element Plant a
a a b b c gt aababc
aa gt aa Herbivore
???? aa b b gt bbc a a
bb gt
bb Carnivore ???? bb c c gt cc b
b
cc gt cc Bacterium ?? c gt
elements in string c gt
c The two reactions in the
bacterium compete
32
Uncontrolled Reactions
In uncontrolled reactions, the reactants are
uniformly mixed and reactions occur when the
reactants undergo elastic collisions. Ui
reactant type i, i 1, , M. ni(t) number
of copies of reactant type i at time t. N Si
n i total number of copies of reactants, a
constant. pi(t) ni(t)/N the proportion of
reactant type i at time t. rij the
proportion of collisions resulting in a reaction
between reactants i and j the forward
reaction rate.
33
An Urn Model for Uncontrolled Reactions (1)
In this urn model, the number of balls in urn Uj
is proportional to pj . Assume that the
reaction between reactants indexed by Uh and Ui
produces the products Uj and Uk. The
probability that the products will be produced is
given by the reaction equation pj pk
rhiphpi
34
An Urn Model for Uncontrolled Reactions (2)
The state of the system at time t is S(t) (
n1(t), n2(t), , nM(t)), the number of balls
in each urn In the urn model, to go from S(t)
to S(t1), pick two balls at random from the urns
and produce the products with probability rhi.
If the products are produced S(t1) (
, nh(t)-1, , ni(t)-1, , nj(t)1, nk(t)1,
), else S(t1) S(t).
35
An Urn Model for Uncontrolled Reactions (3)
This simple urn model can be presented as a
Markov process with one dimension for each
possible distribution of balls in the
urns. Notation Let X,Y be the number of
different ways of choosing Y objects from a
total of X objects. There are b NM-1,
M-1 distinct distributions of N balls in M urns.
b is easily derived by considering the number
of binary numbers of length NM-1 having
exactly M-1 ones, (For example, with N3 and
M2, 0100 gt 1 ball in the first
urn 2 balls in the second urn.)
36
An Urn Model for Uncontrolled Reactions (4)
R is a bxb matrix having Ruv as its component at
row h and column i. Row u corresponds to one
possible distribution of balls in urns, and
column v represents a possible product
distribution Let S(t), the current state,
correspond to row u of the matrix. Let S(t1)
( , nh(t)-1, , ni(t)-1, , nj(t)1, nk(t)1,
), a possible result of a random draw
from the distribution S(t), correspond to
column v. Then Ruv rhiphpi. Note that the
distribution corresponding to row u fully
specifies phand pi . Note that several
different draws can yield the result S(t1).
37
An Urn Model for Uncontrolled Reactions (5)
As is usual with a Markov representation S(tT
) S(t)RT and the equilibrium distribution of
reactants is given by the eigenvector S
corresponding to the positive eigenvalue e of the
matrix R S(t1) eS(t).
38
A Spatial Urn Model
Each site (square in the array) contains a set of
urns representing the reactant types present in
that area. The number of balls in each urn
gives the local concentration of that
reactant type. Each reactant species (same
active sites) is assigned a distinct
color. Diffusion takes place by moving balls at
random between the urns.
39
Urn Models of Tag-based Reactions
40
Control viaSemi-permeable Membranes

• A semi-permeable membrane is a filter. It allows
  only reactants with specific tags
• to diffuse from one side of the membrane to the
  other.

Outside
Inside
Outside High diversity of reactants with low
individual concentrations gt Low reaction
rates Semi-permeable boundary filters
flow gt Increased reaction rates for selected
reactants gt Locally increased concentration
of selected reactants Inside Increased
concentrations gt Catalysts become effective as
"switches" determining reaction sequences
(much like a computer program)
41
Urn Model of a Semi-permeable Membrane

• Each urn is assigned entry and exit conditions.
• For a ball to enter (exit) an urn under
  diffusion, its tags must match the
• corresponding condition.

42
Using Reactants to Define Urn Entry/Exit
Conditions
A new urn is formed each time the system forms a
different kind of reactant with an urn tag.
43
Reaction Inside a Semi-permeable Membrane
44
Distributed control results from the control of
diffusion provided by boundaries, combined with
the control of reactions provided by tags
(signals).
45
Urn Model ofCoupled Membrane-enclosed Reactions
x 111111 y 111000
46
Hierarchical Membrane-enclosed Reactions
By controlling concentrations, this hierarchy
selects and amplifies a particular sequence of
reactions.
47
Corresponding Hierarchical Urn Model
The suffix on the each urn entry tag specifies
the urn(s) from which incoming balls may be drawn.
48
Review of the Urn Model of Controlled Diffusion
Steps in executing the model (1) A ball is
chosen at random from one of the urns. (2) The
match between the balls tags and the exit tags
of an urn determines its probability of
leaving the urn. (3) The match between the
balls tags and the entry tags of the other
urns determines its probability of entering
another urn. These steps are repeated to obtain
the effect of simultaneous diffusion.
49
Markov Process Corresponding tothe Urn Model of
Controlled Diffusion (1)
h index of the ball type chosen. x index of
the urn containing the ball y index of the
target urn. When a ball h moves from urn x to
urn y then S(t1) ( , nhx(t)-1, ,nky(t)1,
), where nhx(t) is the number of balls of type
h in urn x at time t.
50
Markov Process Corresponding tothe Urn Model of
Controlled Diffusion (2)
qhxy the probability that ball h will move
from urn x to urn y, as determined by the match
scores between the tags. As with the
uncontrolled reactions, the state S(t) at time t
is given by the distribution of the balls in the
urns. A Markov matrix D can be used to define
this process, where Duv qhxyph. With b
diffusion steps followed by c reaction steps, the
result is S(tbc) S(t)DbRc.
51
Markov Process Corresponding tothe Urn Model of
Controlled Diffusion (3)
Because the Markov matrix D is sparse -- most
Duv qhxyph 0 -- S(tbc) S(t)DbRc can be
quickly calculated. By using states with high
occupation probabilities as starting points,
Monte Carlo simulations can quickly determine
communities of states niches with large exit
times.
52
Simple examples of interactions in reaction
networks with boundaries and signals

• mass action and unbalanced flows
• counter-current flows

Some themes

• persistent patterns in flows
• niche formation and specialization
• diversity and increasing complexity
• modules, motifs, and building blocks

53
Evolution Generates Feedback and Controlsin
Complex Adaptive Systems
Three examples from natural systems Niche
formation The devils garden in
Peru. Octopus joints and convergent
evolution. Octopus mimicry of snakes and fish.

Evolution can increase interactions, boundaries,
and signals in reaction networks.
54
Recapitulation ????

• Hypotheses
• Local concentrations of resources, induced by
  feedback and recycling, provide opportunities for
  the formation and adaptive radiation of agents.
• This process of agent formation leads to
  increasing diversity of agents and progressively
  larger amounts of resources tied up in agents.
• Under tranquil??? conditions, increasing
  agent specialization should be observed.
• If any of these hypotheses can be established we
  will have substantially
• increased our understanding of cas.
• Outline

55
(No Transcript)
56
Details
57
A Quick Review of Reaction Networks (6) A
Rule-based Version of a Reaction Network
t1... t2... gt t3...
58
A Quick Review of Reaction Networks (8)
Formalisms for Reaction Nets
Difference Equations x(t1) x(t) rx(t)y(t)
ru(t)v(t), where x, y. are
concentrations and r, r, are reaction
rates. Urn Model
Billiard Ball ?? Mechanics (Markov
Process) Rule-based Signal Processing a1
a 2 a 3 a k b1b2 bk T a 1 a k b 1 b k
59
Urn Models of Tag-based Reactions (1)