Science in the 20th century

1
Science in the 20th century

• Systems
• Robustness (Bode, Zames,)
• Computational complexity (Turing, Godel, )
• Information (Shannon, Kolmogorov)
• Chaos and dynamical systems (Poincare, Lorenz,)
• Optimal control (Pontryagin, Bellman,)

• Materials and devices
• Relativity
• Quantum mechanics
• Chemical bond
• Molecular basis of life

2
Current dominant challenges

• Materials and devices
• Unified field theory
• Dynamics of chemical reactions
• Dynamics of biological macromolecules

• Systems
• Robustness of complex interconnected dynamical
  systems and networks
• Unified field theory of control,
  communications, computing

3
Current dominant challenges

• Robustness of complex interconnected dynamical
  systems and networks

• Role of control theory
• Robustness
• Interconnection
• Rigor

We need an expanded view of all of these.
4
Robust
Humans have exceptionally robust systems for
vision and speech.
5
Yet fragile
but were not so good at surviving, say, large
meteor impacts.
6
Yet fragile
but were not so good at surviving, say, large
meteor impacts.
7
Robustness and uncertainty
Sensitive
Error, sensitivity
Robust
Types of uncertainty
8
Robustness and uncertainty
Sensitive
Error, sensitivity
Robust
Meteor impact
speech/ vision
Types of uncertainty
9
Robustness and uncertainty
yet fragile
Sensitive
Error, sensitivity
Robust
Meteor impact
speech/ vision
Types of uncertainty
10
Complex systems
yet fragile
Sensitive
Error, sensitivity
Robust
Robust
Types of uncertainty
11
Robust, yet fragile

• Robust to uncertainties
• that are common,
• the system was designed for, or
• has evolved to handle,
• yet fragile otherwise
• This is the most important feature of complex
  systems (the essence of HOT).

12
Example Auto airbags

• Reduces risk in high-speed collisions
• Increases risk otherwise
• Increases risk to small occupants
• Mitigated by new designs with greater complexity

• Could just get a heavier vehicle
• Reduces risk without the increase!
• But shifts it elsewhere occupants of other
  vehicles, pollution

13
Biology (and engineering)

• Grow, persist, reproduce, and function despite
  large uncertainties in environments and
  components.
• Yet tiny perturbations can be fatal
• a single specie or gene
• minute quantities of toxins

• Complex, highly evolved organisms and ecosystems
  have high throughput,
• But are the most vulnerable in large extinctions.
• Complex engineering systems have similar
  characteristics

14
Automobile air bags
Error, sensitivity
Types of uncertainty
15
Is robustness a conserved quantity?
Information/ Computation
Robustness/ Uncertainty
constrained
Materials
Energy
Entropy
16
Uncertainty and Robustness
Complexity
Interconnection/ Feedback Dynamics Hierarchical/ M
ultiscale Heterogeneous Nonlinearity
17
Uncertainty and Robustness
Complexity
Interconnection/ Feedback Dynamics Hierarchical/ M
ultiscale Heterogeneous Nonlinearity
18
Prediction the most basic scientific question.
19
x(k) uncertain sequence
-
e(k)
u(k-1)
u(k)
delay
predictor
u(k) prediction of x(k1) e(k) error
e(k) x(k) - u(k-1)
20
Prediction is a special case of feedforward. For
known stable plant, these are the same
21
For simplicity, assume x, u, and e are finite
sequences.
x(k)
u(k)
k
e(k)
k
Then the discrete Fourier transform X, U, and E
are polynomials in the transform variable z.
If we set z ei? , ? ? 0,?? then X(w) measures
the frequency content of x at frequency w.
22
x
x(k)
-
e
u(k-1)
u(k)
u(k)
delay
C
e(k) x(k) - u(k-1)
e(k)
How do we measure performance of our predictor C
in terms of x, e, X, and E?
Typically want ratios of norms
or
to be small.
23
Good performance (prediction) means
or
Equivalently,
or
For example,
Plancheral Theorem
24
Interesting alternative
Or to make it closer to existing norms
Not a norm, but a very useful measure of signal
size, as well see. (The b in ?b is in honor
of Bode.)
25
A useful measure of performance is in terms of
the sensitivity function S(z) defined by Bode as
If we set z ei? , ? ? 0,?? then S(w)
measures how well C does at each frequency. (If C
is linear then S is independent of x, but in
general S depends on x.) It is convenient to
study log S(w) and then
u ? 0 ( u(k)0 ? k) ? S ? 1, and logS ? 0.
log S(w) lt 0 ? C attenuates x at frequency
w.
log S(w) gt 0 ? C amplifies x at frequency w.
26
Note as long as we assume that for any possible
sequence x(k) it is equally likely that -x(k)
will occur, then guessing ahead can never help.
Assume u is a causal function of x.
x(k)
u(k-1)
k
0
This will be used later.
27
-
e(k)
x(k)
u(k-1)
u(k)
delay
C
e(k) x(k) - u(k-1)
For any C, an unconstrained worst-case x(k)
is x(k) -u(k-1), which gives e(k) x(k) -
u(k-1) - 2u(k-1) 2x(k)
Thus, if nothing is known about x(k), the
safest choice is u ? 0. Any other choice of u
does worse in any norm.
If x is white noise, then u ? 0 is also the best
choice for optimizing average behavior in almost
any norm.
28
Summary so far

• Some assumptions must be valid about x in order
  that it be at all predictable.
• Intuitively, there appear to be fundamental
  limitations on how well x can be predicted.
• Can we give a precise mathematical description
  of these limitations that depends only on
  causality and require no further assumptions?

-
e(k)
x(k)
u(k-1)
u(k)
delay
C
e(k) x(k) - u(k-1)
29

• Recall that S(z) E(z)/X(z) and S(?) 1.
• Denote by ek and xk the complex zeros for z
  gt 1 of E(z) and X(z), respectively. Then

Proof Follows directly from Jensens formula, a
standard result in complex analysis (advanced
undergraduate level).
If x is chosen so that X(z) has no zeros in z gt
1 (this is an open set), then
30

• Recall that S(z) E(z)/X(z) and S(?) 1.
• Denote by ek and xk the complex zeros for z
  gt 1 of E(z) and X(z), respectively.

If the predictor is linear and time-invariant,
then
Under some circumstances, a time-varying
predictor can exploit signal precursors that
create known xk
31
logS gt 0 amplified logS lt 0 attenuated
?
?he amplification must at least balance the
attenuation.
logS
32
yet fragile
?
Robust
33

• Originally due to Bode (1945).
• Well known in control theory as a property of
  linear systems.
• But its a property of causality, not linearity.
• Many generalizations in the control literature,
  particularly in the last decade or so.
• Because it only depends on causality, it is in
  some sense the most fundamental known
  conservation principle.
• This conservation of robustness and related
  concepts are as important to complex systems as
  more familiar notions of matter, energy, entropy,
  and information.

34
Recall
is equivalent to
35
Uncertainty and Robustness
Complexity
Interconnection/ Feedback Dynamics Hierarchical/ M
ultiscale Heterogeneous Nonlinearity
36
What about feedback?
37
Simple case of feedback.
e error d disturbance c control
e d c d F (e)
(1-F )e d
38
F gt 0 ln(S) gt 0
ln(S)
amplification
F
F lt 0 ln(S) lt 0
attenuation
39
F ? 1 ln(S) ? ?
ln(S)
extreme sensitivity
F
extreme robustness
F ? ?? ln(S) ? ??
40
Uncertainty and Robustness
Complexity
Interconnection/ Feedback Dynamics Hierarchical/ M
ultiscale Heterogeneous Nonlinearity
41
If these model physical processes, then d and e
are signals and F is an operator. We can still
define S(?? E(?? /D(?? where E and D are
the Fourier transforms of e and d. ( If F is
linear, then S is independent of D.)
Under assumptions that are consistent with F and
d modeling physical systems (in particular,
causality), it is possible to prove that
42
logS gt 0 amplified logS lt 0 attenuated
?
?he amplification must at least balance the
attenuation.
logS
Positive and negative feedback are balanced.
43
Negative feedback
?
lnS
logS
Positive feedback
F
44
yet fragile
Negative feedback
?
lnS
logS
Positive feedback
Robust
F
45
Feedback is very powerful, but there are
limitations.
It gives us remarkable robustness, as well as
recursion and looping.
Formula 1 The ultimate high technology sport
But can lead to instability, chaos, and
undecidability.
46

• In development
• drive-by-wire
• steering/traction control
• collision avoidance

47

• Electronic fuel injection
• Computers
• Sensors
• Telemetry/Communications
• Power steering

Formula 1 allows
sensors
actuators
driver
computers
telemetry
48
Control Theory
Information Theory
Computational
Theory of Complex systems?
Complexity
Statistical Physics
Dynamical Systems
49
uncertain sequence
d
error
-

e
c
delay
predictor
F

• Kolmogorov complexity
• Undecidability
• Chaos
• Probability, entropy
• Information
• Bifurcations, phase transitions

This is a natural departure point for
introduction of chaos and undecidability.