Overview

1
Biochemical Network E. Coli Metabolism
Regulatory Interactions
Complexity ? Robustness
Supplies Materials Energy
Supplies Robustness
From Adam Arkin
from EcoCYC by Peter Karp
2
Robustness
the environment is uncertain
3
3
10
2
10
Frequency of outages gt N
1
10
US Power outages 1984-1997
0
10
4
5
6
7
10
10
10
10
N of customers affected by outage
4
Background

• Much attention has been given to complex
  adaptive systems in the last decade.
• Popularization of information, entropy, phase
  transitions, criticality, fractals,
  self-similarity, power laws, chaos, emergence,
  self-organization, etc.
• Physicists emphasize emergent complexity via
  self-organization of a homogeneous substrate near
  a critical or bifurcation point (SOC/EOC)

5
Criticality and power laws

• Tuning 1-2 parameters ? critical point
• In certain model systems (percolation, Ising, )
  power laws and universality iff at criticality.
• Physics power laws are suggestive of criticality
• Engineers/mathematicians have opposite
  interpretation
• Power laws arise from tuning and optimization.
• Criticality is a very rare and extreme special
  case.
• What if many parameters are optimized?
• Are evolution and engineering design different?
  How?
• Which perspective has greater explanatory power
  for power laws in natural and man-made systems?

6
Highly Optimized Tolerance (HOT)

• Complex systems in biology, ecology, technology,
  sociology, economics,
• are driven by design or evolution to
  high-performance states which are also tolerant
  to uncertainty in the environment and components.
• This leads to specialized, modular, hierarchical
  structures, often with enormous hidden
  complexity,
• with new sensitivities to unknown or neglected
  perturbations and design flaws.
• Robust, yet fragile!

7
Robustness of HOT systems
Fragile
Fragile (to unknown or rare perturbations)
Robust (to known and designed-for uncertainties)
Uncertainties
Robust
8
Robustness of shear flows
Fragile
Viscosity
Everything else
Robust
9
Robustness of HOT systems
Fragile
Humans
Archaea
Chess
Meteors
Machines
Robust
10
Robustness
Complexity
Interconnection
Aim simplest possible story
11
The simplest possible spatial model of HOT.
Square site percolation or simplified forest
fire model.
Carlson and Doyle, PRE, Aug. 1999
12
Assume one spark hits the lattice at a single
site.
A spark that hits an empty site does nothing.
13
A spark that hits a cluster causes loss of that
cluster.
14
Think of (toy) forest fires.
15
Think of (toy) forest fires.
16
Yield the density after one spark
17
1
0.9
critical point
0.8
Y (avg.) yield
0.7
0.6
0.5
N100
0.4
0.3
0.2
0.1
0
0
0.2
0.4
0.6
0.8
1
? density
18
1
0.9
critical point
Y (avg.) yield
0.8
limit N ? ?
0.7
0.6
0.5
0.4
0.3
0.2
?c .5927
0.1
0
0
0.2
0.4
0.6
0.8
1
? density
19
Yield the density after one spark
20
Y
Fires dont matter.
Cold
?
21
Everything burns.
Y
Burned
?
22
Critical point
Y
?
23
Power laws
Criticality
cumulative frequency
cluster size
24
Edge-of-chaos, criticality, self-organized
criticality (EOC/SOC)

• Essential claims
• Nature is adequately described by generic
  configurations (with generic sensitivity).

yield

• Interesting phenomena is at criticality (or
  near a bifurcation).

25
18 Sep 1998
Forest Fires An Example of Self-Organized
Critical Behavior Bruce D. Malamud, Gleb Morein,
Donald L. Turcotte
4 data sets
26
2
10
-1/2
1
10
0
10
-2
-1
0
1
2
3
4
10
10
10
10
10
10
10
Exponents are way off
27

• Rare, nongeneric, measure zero.
• Structured, stylized configurations.
• Essentially ignored in stat. physics.
• Ubiquitous in
• engineering
• biology
• geophysical phenomena?

What about high yield configurations?
28
Highly Optimized Tolerance (HOT)
critical
Cold
Burned
29
Why power laws?
Optimize Yield
Almost any distribution of sparks
Power law distribution of events
30
Probability distribution (tail of normal)
2.9529e-016
0.1902
5
10
15
20
25
30
5
10
15
20
25
30
2.8655e-011
4.4486e-026
31
David Reynolds
Increasing Design Degrees of Freedom
Coarse grain underlying lattice into an M x M
design lattice
1
DDOF1
Optimize yield as a function of ? for a
percolation forest fire model
32
Increasing Design Degrees of Freedom
DDOF4
1
4 tunable densities Each region is
characterized by the ensemble of random
configurations at density ?i
33
Increasing Design Degrees of Freedom
DDOF16
1
16 tunable densities
34
Increasing Design Degrees of Freedom
Coarse grain underlying lattice into an M x M
design lattice
1
DDOF1
Optimize yield As a function of ? for a
percolation forest fire model
35
Design Degrees of Freedom Tunable Parameters
SOC1 DDOF
The density for percolation
Generic fractal patterns
36
Increasing Design Degrees of Freedom
Tunable Parameters
37
HOT many mechanisms
grid
evolved
DDOF
All produce

• High densities
• Modular structures reflecting external
  disturbance patterns
• Efficient barriers, limiting losses in cascading
  failure
• Power laws

38
Small events likely
Optimized grid
density 0.8496 yield 0.7752
1
0.9
High yields.
0.8
0.7
0.6
grid
0.5
0.4
random
0.3
0.2
0.1
0
0
0.2
0.4
0.6
0.8
1
39
Increasing DDOF increases densities and
resolution of P(i,j) increases yields, decreases
losses, increases sensitivity
Robust, yet fragile
40
Extreme robustness and extreme hypersensitivity.
Small flaws
Robust, yet fragile?
41
If probability of sparks changes.
disaster
42
0
10
-1
10
Cum. Prob.
evolved
-2
10
All produce Power laws
-3
10
grid
-4
10
0
1
2
3
10
10
10
10
size
43

• HOT yields compact events of nontrivial size.
• SOC has infinitesimal, fractal events.

HOT
SOC
large
infinitesimal
size
44
6
Data compression (Huffman)
WWW files Mbytes (Crovella)
5
4
Cumulative
3
Frequency
Forest fires 1000 km2 (Malamud)
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
Decimated data Log (base 10)
Size of events
45
6
Web files
5
Codewords
4
Cumulative
3
Frequency
Fires
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
Size of events
Log (base 10)
46
Data Model/Theory
6
DC
5
WWW
4
3
2
1
Forest fire
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
47
Critical percolation and SOC forest fire models

• SOC HOT have completely different
  characteristics.
• SOC vs HOT story is consistent across different
  models.
• Focus on generalized coding abstraction for
  HOT

48
A HOT forest fire abstraction
Fire suppression mechanisms must stop a 1-d front.
Optimal strategies must tradeoff resources with
risk.
49
Why power laws?
Optimize Yield
Almost any distribution of sparks
Power law distribution of events
50
Web/internet traffic
web traffic
Is streamed out on the net.
Web client
Creating internet traffic
Web servers
51
web traffic
Lets look at some web traffic
Is streamed out on the net.
Web client
Creating internet traffic
Web servers
52
6
Data compression (Huffman)
WWW files Mbytes (Crovella)
5
Cumulative
4
3
Frequency
Forest fires 1000 km2 (Malamud)
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
Decimated data Log (base 10)
Size of events
(codewords, files, fires)
53
Universal network behavior?
Congestion induced phase transition.
throughput

• Similar for
• Power grid?
• Freeway traffic?
• Gene regulation?
• Ecosystems?
• Finance?

demand
54
Web/Internet?
55
Networks

• Making a random network
• Remove protocols
• No IP routing
• No TCP congestion control
• Broadcast everything
• ? Many orders of magnitude slower

log(thru-put)
log(demand)
56
Networks
HOT
log(thru-put)
log(demand)
57
Issues in web layout design

• Logical and aesthetic structure determines rough
  graph topology
• Navigability, manageability, and download times
  drive geometry of links and files
• Navigability and manageability
• Low diameter (number of clicks)
• Low out-degree (number of choices)
• Download time small files
• These objectives are in conflict

58
A toy website model( 1-d grid HOT design)
document
59
Optimize 0-dimensional cuts in a 1-dimensional
document
links files
60
Source coding for data compression
(simplest optimal design theory in engineering)
61
Generalized coding problems
Data compression
Optimizing d-1 dimensional cuts in d dimensional
spaces.
Web
62
PLR optimization
Minimize expected loss
63
d-dimensional
li volume enclosed ri barrier density
pi Probability of event
Resource/loss relationship
64
PLR optimization
? 0 data compression ? 1 web layout ?
2 forest fires
? dimension
65
Minimize average cost using standard Lagrange
multipliers
Leads to optimal solutions for resource
allocations and the relationship between the
event probabilities and sizes.
With optimal cost
66
To compare with data.
67
To compare with data.
68
Power Laws
a1/ß
69
6
Data compression (Huffman)
WWW files Mbytes (Crovella)
5
4
Cumulative
3
Frequency
Forest fires 1000 km2 (Malamud)
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
Decimated data Log (base 10)
Size of events
70
6
Web files
5
Codewords
4
Cumulative
3
Frequency
Fires
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
Size of events
Log (base 10)
71
Data Model
6
DC
5
WWW
4
3
FF
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
72
Data Model
6
DC
5
WWW
4
3
FF
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
73
from data
from data
74
What can we learn from this simple model?

• P uncertain events with probabilities pi
• R limited resources ri to minimize
• L loss li due to event i

• Be cautious about simple theories that ignore
  design.
• Power laws arise easily in designed systems due
  to resource vs. loss tradeoffs.
• Exploiting assumptions, makes you sensitive to
  them.
• More robustness leads to sensitivities elsewhere.
• Robust, yet fragile.

75
California geographyfurther irresponsible
speculation

• Rugged terrain, mountains, deserts
• Fractal dimension ? ? 1?
• Dry Santa Ana winds drive large (? 1-d) fires

76
Data HOT Model/Theory
6
5
California brushfires
4
3
FF (national) d 2
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
77
Data HOTSOC
6
5
4
3
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
78
Critical/SOC exponents are way off
Data ? gt .5
SOC ? lt .15
79
HOT
SOC and HOT have very different power laws.
d1
SOC
d1

• HOT ? decreases with dimension.
• SOC?? increases with dimension.

80

• HOT yields compact events of nontrivial size.
• SOC has infinitesimal, fractal events.

HOT
SOC
large
infinitesimal
size
81
SOC and HOT are extremely different.
HOT
SOC
82
SOC and HOT are extremely different.
HOT
SOC
83
Summary

• Power laws are ubiquitous, but not surprising
• HOT may be a unifying perspective for many
• Criticality SOC is an interesting and extreme
  special case
• but very rare in the lab, and even much rarer
  still outside it.
• Viewing a system as HOT is just the beginning.

84
The real work is

• New Internet protocol design
• Forest fire suppression, ecosystem management
• Analysis of biological regulatory networks
• Convergent networking protocols
• etc

85
Forest fires dynamics
Intensity Frequency Extent
86
Los Padres National Forest
Max Moritz
87
Red human ignitions (near roads)
Yellow lightning (at high altitudes in ponderosa
pines)
Brown chaperal Pink Pinon Juniper
Ignition and vegetation patterns in Los Padres
National Forest
88
California brushfires
FF ? 2
89
Santa Monica Mountains
Max Moritz and Marco Morais
90
SAMO Fire History
91
Fires are compact regions of nontrivial area.
Fires 1930-1990
Fires 1991-1995
92
We are developing realistic fire spread models
GIS data for Landscape images
93
Models for Fuel Succession
94
1996 Calabasas Fire
Historical fire spread
Simulated fire spread
Suppression?
95
Preliminary Results from the HFIRES simulations
(no Extreme Weather conditions included)
96
Increasing the number of extreme weather events
extends the tail
97
Varying suppression leads to a steeper power laws
across the broad range of event sizes
98
Excellent agreement with data for realistic
values of the parameters
99
Agreement with the PLR HOT theory based on
optimal allocation of resources
a1/2
a1
100
What is the optimization problem?
(we have not answered this question for fires
today)
Plausibility Argument

• Fire is a dominant disturbance which shapes
  terrestrial ecosystems
• Vegetation adapts to the local fire regime
• Natural and human suppression plays an important
  role
• Over time, ecosystems evolve resilience to
  common variations
• But may be vulnerable to rare events
• Regardless of whether the initial trigger for
  the event is large or small

HFIREs Simulations

• We assume forests have evolved this resiliency
  (GIS topography
• and fuel models)
• For the disturbance patterns in California
  (ignitions, weather models)
• And study the more recent effect of human
  suppression
• Find consistency with HOT theory
• But it remains to be seen whether a model which
  is optimized or
• evolves on geological times scales will
  produce similar results

101
The shape of trees by Karl Niklas
Simulations of selective pressure shaping
early plants

• L Light from the sun (no overlapping branches)
• R Reproductive success (tall to spread seeds)
• M Mechanical stability (few horizontal
  branches)
• L,R,M All three look like real trees

Our hypothesis is that robustness in an uncertain
environment is the dominant force shaping
complexity in most biological, ecological,
and technological systems
102
(No Transcript)