Defining complexity

1
Defining complexity

• Aim simple but universal taxonomy
• Widely divergent starting points from math,
  biology, technology, physics, etc,
• Can be organized into a coherent and consistent
  picture
• Even very different notions of complexity
• Organized complexity
• Chaocritical complexity
• Irreducible complexity

2
The essence of simplicity

• Simple questions
• Elegant experiments, flexible platforms
• Small, simple models
• explain data
• testable predictions
• computable
• Elegant unifying theorems

• Simple answers
• Simple outcomes, data robust, repeatable
• Robust, predictable
• explain data
• testable predictions
• both computable
• Short proofs

3
The essence of simplicity

• Simple questions
• Elegant experiments, flexible platforms
• Small, simple models
• explain data
• testable predictions
• computable
• Elegant unifying theorems

• Simple answers
• Simple outcomes, data robust, repeatable
• Robust, predictable
• explain data
• testable predictions
• both computable
• Short proofs

4
The essence of simplicity

• Simple questions
• Elegant experiments
• Small models
• Elegant theorems

• Simple answers
• Simple outcomes
• Robust, predictable
• Short proofs

Integrated infrastructure experimental,
software, computational, theory To iterate
between these elements
5
A minimal notion of simplicity

• Simple questions
• Elegant experiments
• Small models
• Elegant theorems

• Simple answers
• Simple outcomes
• Robust, predictable
• Short proofs

6
A minimal notion of simplicity

• Simple questions
• Small models

• Simple answers
• Robust, predictable

• Reductionist science Reduce the apparent
  complexity of the world directly to an underlying
  simplicity.
• Physics has always epitomized this approach
• Molecular biology has successfully mimicked
  physics
• Engineering at the circuit level
• Bio and tech networks need more
• How to describe the space of complexity?

7
Two dimensions of complexity

• Small vs large descriptions, models, theorems
• Robust vs fragile features in response to
  perturbations in descriptions, components, or the
  environment.

8
Where were going

• Aim simple but universal taxonomy
• Widely divergent starting points from math,
  biology, technology, physics, etc,
• Can be organized into a coherent and consistent
  picture
• Even very different notions of complexity
• Organized complexity
• chaocritical complexity
• Irreducible complexity

9

• Simple questions
• Elegant experiments
• Small models
• Elegant theorems

• Simple answers
• Simple outcomes
• Robust, predictable
• Short proofs

10
(No Transcript)
11

• Simple questions
• Elegant experiments
• Small models
• Elegant theorems

• Simple answers
• Simple outcomes
• Robust, predictable
• Short proofs

• Godel Incompleteness, Turing Undecidability
• Even simple questions can be complex and
  fragile
• Profoundly effected mathematics and computation
• Modest impact on science, primarily through
  emphasis on chaocritical complexity

12
chaocritical complexity

• Simple question
• Undecidable

• No short proof
• Chaos
• Fractals

Mandelbrot
13

• The most fragile details of complex systems will
  likely always be experimentally and
  computationally intractable.

• Fortunately, we care more about understanding,
  avoiding, and managing fragility (than about all
  its details)

14
Organized complexity
15
Organized complexity
16

• Simple questions
• Elegant experiments
• Small models
• Elegant theorems

• Simple answers
• Simple outcomes
• Robust, predictable
• Short proofs

• The triumph (and horror) of organization
• Complex, uncertain, hostile environments
• Unreliable, uncertain, changing components
• Limited testing and experimentation
• Yet predictable, robust, adaptable, evolvable
  systems

17

• Requires highly organized interactions, by design
  or evolution
• Completely different theory and technology from
  chaocritical

• Simple questions
• Elegant experiments
• Small models
• Elegant theorems

• Simple answers
• Simple outcomes
• Robust, predictable
• Short proofs

• The triumph (and horror) of organization
• Complex, uncertain, hostile environments
• Unreliable, uncertain, changing components
• Limited testing and experimentation
• Yet predictable, robust, adaptable, evolvable
  systems

18
Organized
19
Organized
chaocritical
20
Robust yet fragile
Organization
Organization

• Even systems most designed/evolved for extreme
  robustness, also have fragilities and this is not
  accidental
• The most designed/evolved systems have
  systematic, universal spirals of
  robustness/complexity/fragility
• Understanding/controlling this spiral is
  arguably the central challenge in organized
  complexity
• There are hard constraints on robustness/fragility
  
• Thus robustness is never free and is paid for
  with fragility somewhere

21
Issues

• chaocritical and Organization are opposites, but
  can be viewed in this unified framework
• chaocritical celebrates fragility
• Organization seeks to manage robustness/fragility
  
• Much confusion is caused by failure to get
  this
• The most fundamental challenge of organizing
  complexity for robust and evolvable systems is
  inherent and unavoidable robustness/fragility
  tradeoffs

22
Robust
Human complexity
Yet Fragile

• Efficient, flexible metabolism
• Complex development and
• Immune systems
• Regeneration renewal
• Complex societies
• Advanced technologies

• Obesity and diabetes
• Rich parasite ecosystem
• Inflammation, Auto-Im.
• Cancer
• Epidemics, war,
• Catastrophic failures

• Evolved mechanisms for robustness allow for, even
  facilitate, novel, severe fragilities elsewhere
  (often involving hijacking/exploiting the same
  mechanism)
• Universal challenge Understand/manage/overcome
  this robustness/complexity/fragility spiral

23
Nightmare?
Biology We might accumulate more complete parts
lists but never understand how it all
works. Technology We might build increasingly
complex and incomprehensible systems which will
eventually fail completely yet cryptically.
Nothing in the orthodox views of complexity says
this wont happen (apparently).
24
HOPE?
Interesting complex systems are robust yet
fragile. Focus robustness on environment/componen
t uncertainty. Identify the fragility, evaluate
and protect it.
Nothing in the orthodox views of complexity says
this cant happen (apparently).
25
The full picture
Irreducibility
?

• Some fragilities are inevitable in robust
  complex systems.
• But are there circumstances in which large
  descriptions, long proofs, and high fragility are
  desirable?

• Yes, all are important features of cryptography
  and security, including host-pathogen
  interactions.

26
Recap
? Nightmare ?

• Nightmare that technology, biology, and
  medicine (and social sciences) get stuck with
  only spiraling complexity, large
  models/descriptions, no coherent understanding,
  and uncontrollable fragilities.
• Hope that more rigorous methods can provide
  systematic tools for managing complexity and
  robustness/fragility.
• There is both tremendous progress and
  substantial confusion, and robustness/fragility
  is at the heart of both.

27
Errors and confusion

• Irreducible complexity, intelligent design, and
  creationism
• Overly mystical (like chaocritical) and
  underestimates the organized complexity of
  evolved organisms
• If biology were irreducibly complex, it would
• require a (rather incompetent) creator, since it
  would be too fragile to evolve
• also be so fragile as to require constant
  intervention of supernatural control mechanisms

28
Irreducibility and intelligent design
Rube Goldberg
29
The essential ID argument
If biology is like this, then it could not have
evolved.

• This is actually true, and in fact
• If biology is like this,
• then it would be too fragile to persist,
• and would need the constant intervention of
  supernatural forces

30
The flaw
This is a cartoon.

• It is too fragile to actually build.
• Neither biology nor (most of) technology is
  anything like this.
• Who said otherwise? Lots of real scientists!
• Oops!

31
Organized

• chaocritical and Irreducible have much in
  common.
• Popular among nonexperts, politically powerful.
• Both within science (chaocritical ) and between
  science and society (Irreducible ). Oops!!!

chaocritical
Irreducible
Too fragile to evolve or work in the real world.
32
Key concepts
This is a parsimonious and coherent ontology for
organized complexity. There are many success
stories but they are still fragmented.

• Robust yet fragile
• Architecture
• Hard limits
• Small models
• Short proofs

33
Key concepts

• Robust yet fragile
• Architecture
• Hard limits
• Small models
• Short proofs

What creates extremes of robustness (and thus
fragility)?
34
Key concepts

• Robust yet fragile
• Architecture
• Hard limits
• Small models
• Short proofs

What creates extremes of robustness (and thus
fragility)?
35
System-level
Constraints
Aim a universal taxonomy of complex systems and
theories
Emergent
Architecture
Component

• Describe systems/components in terms of
  constraints on what is possible
• Decompose constraints into component,
  system-level, architecture, and emergent
• Not necessarily unique, but hopefully
  illuminating nonetheless

36
Systems requirements functional,
efficient, robust, evolvable
Hard constraints Thermo (Carnot) Info
(Shannon) Control (Bode) Compute (Turing)
Architecture Constraints That Deconstrain
Constraints
Components and materials Energy, moieties
37
Environment Robust power generation
Architecture Carnot cycle (Combined cycle)
System Hard limits Entropy, 2nd law
Components Energy conserved
38
Environment Robust power generation
Architecture Carnot cycle (Combined cycle)
System Hard limits Entropy, 2nd law
Chance/choice Or Necessity?
Components Energy conserved
39
Electricity generation and consumption
From chance to necessity?
http//phe.rockefeller.edu/Daedalus/Elektron/
40
Environment Robust power generation
Chance/choice Or Necessity?
Similar architectures
System Hard limits Entropy

• Similar Efficiencies
• Overall (30-60)
• Mechanical ? Electrical (100)

Components Energy conserved
41
De Duve, Wachtershauser

• PMF
• DNA
• Proteins
• Lipids
• RNA
• ATP
• NTP and (pyro-)phospho-transfer
• Choice of ions? (Ca2, Na, K, Mg2)
• Choice of metals? (Fe, etc.)
• Thioesters
• Group transfer
• Electron transfer
• Catalysis
• Carbon, Nitrogen, Hydrogen,
• Energy, matter, small moieties

Chance/ Choice? Necessity
Necessity Physico-chemical
42
Necessity (Environment) Robustness of system
Chance/ choice Or Necessity?
Necessity (Theory) Hard limits on Robustness
Choice? Robust Architecture
Complexity?
Robustness
Necessity Physico-chemical
43
System-level Constraints
Emergent Constraints Hard limits
Architecture Constraints Protocols and rules
Reverse engineering

• Given a system or domain (e.g. biology)
• Figure out what are these four sources of
  constraints

Component constraints
44
System-level Constraints

• System-level constraints are on the system as a
  whole and usually depend on environment and
  history (which in many cases may be only
  partially known)
• Component constraints are often the easiest to
  determine, and have been the focus of
  reductionist science. Can often be determined to
  some extent from their study in isolation.
• There may be some choice as to whether any given
  constraints goes into systems vs components

Component constraints
45

• Architectural constraints do not necessarily
  follow from systems and components.
• These constraints are usually described in terms
  of rules or protocols that specify some allowable
  subset of all possible interconnection of
  components.
• In reverse engineering, the aim is to figure out
  what rules are being followed that allow for the
  system to be efficient, robust, evolvable, etc.
• In forward engineer, the aim is to specify
  protocols that insure such system behavior

Architecture Constraints Protocols and rules
46

• Emergent constraints are additional hard limits
  on system characteristics that are implied by the
  intersection of component, other system, and
  architectural constraints.
• They are most interesting when they do not follow
  trivially from the other constraints.
• Examples of emergent constraints include
• Entropy in thermodynamics
• Channel capacity theorems in information theory
• Bode integral and related limits in control
  theory
• Undecidability, NP-hardness, etc in computational
  complexity theory

Emergent Constraints Hard limits
47
System-level Constraints
Forward engineering
Architecture Constraints Protocols
Design architectures and components to insure
system-level constraints
Emergent Constraints Hard limits
Component constraints
48
System-level
Constraints and architectures
Aim a universal taxonomy of complex systems and
theories
Emergent
Architecture
Component

• Existing theories
• Thermodynamics (Carnot)
• Communications (Shannon)
• Control (Bode)
• Computation (Turing/Gödel)

• Assume different constraints and architectures a
  priori
• Fragmented and incompatible
• Cannot be used as a basis for comparing
  architectures
• New unifications are encouraging

49
Hard constraints Thermo (Carnot) Info
(Shannon) Control (Bode) Compute (Turing)

• Assume architectures a priori
• Fragmented and incompatible
• Cannot be used as a basis for comparing
  architectures
• New unifications are encouraging

Emergent Constraints
50
Nuno C Martins and Munther A Dahleh, Feedback
Control in the Presence of Noisy Channels
Bode-Like Fundamental Limitations of
Performance. Nuno C. Martins, Munther A. Dahleh
and John C. Doyle Fundamental Limitations of
Disturbance Attenuation in the Presence of Side
Information (Both in IEEE Transactions on
Automatic Control)
http//www.glue.umd.edu/nmartins/
51
Fragile
Disturbance
-
ed-u
d
Remote Sensor
Control Channel
Plant
Sensor Channel
Control
Encode
remote control
benefits
stabilize
remote sensing
feedback
costs

• Good designs transform/manipulate robustness
• Subject to hard limits
• Unifies theorems of Shannon and Bode (1940s)
• Claim This is the most crucial (known) limit
  against which network complexity must cope

52
Bodes integral formula
Yet fragile
?
Robust
benefits
costs
53
d
ed-u
Disturbance
-
u
Plant
Cost of stabilization
Control
Cost of control
?
benefits
costs
54
-
ed-u
Cost of remote control
Plant
Control Channel
Control
benefits
costs
55
Disturbance
-
ed-u
d
Plant
Control Channel
Control
remote control
benefits
stabilize
feedback
costs
56
Disturbance
-
ed-u
d
Remote Sensor
Plant
Control Channel
Sensor Channel
Control
Encode
remote control
benefits
stabilize
remote sensing
feedback
costs
57
Disturbance
-
ed-u
d
Remote Sensor
Plant
Control Channel
Sensor Channel
Control
Encode
Benefit of remote sensing
benefits
costs
58
Disturbance
-
ed-u
d
Remote Sensor
Plant
Control Channel
Sensor Channel
Control
Encode
remote control
benefits
stabilize
remote sensing
feedback
costs
59
Disturbance
-
ed-u
d
Remote Sensor
Plant
Control Channel
Sensor Channel
Control
Encode
Bode/Shannon is likely a better p-to-p comms
theory to serve as a foundation for networks than
either Bode or Shannon alone.
60
Electric power network
Variety of producers
Variety of consumers

• Good designs transform/manipulate energy
• Subject (and close) to hard limits

61
Fragile
Disturbance
Control
-
ed-u
d
Remote Sensor
Control Channel
Plant
Sensor Channel
Control Channel
Control
Encode

• Robust designs transform/manipulate robustness
• Subject (and close) to hard limits
• Fragile designs are far away from hard limits and
  waste robustness.

62
Hard constraints Thermo (Carnot) Info
(Shannon) Control (Bode) Compute (Turing)

• Assume architectures a priori
• Fragmented and incompatible
• Cannot be used as a basis for comparing
  architectures
• New unifications are encouraging
• Robust/fragile is unifying concept

Constraints