Phase Transitions and Cellular Automata

1
Phase Transitions and Cellular Automata

• Blake Johnson
• Paper Group A1CS 297 Complex Systems

2
Introduction

• Computation, Dynamics and the Phase Transition,
  by J.Avnet, Santa Fe Institute. 2000.
• An introduction to the relationship between
  Cellular Automata, computation, and dynamical
  systems, that reviews and explicates previous
  work, primarily by Crutchfield, Wolfram, and
  Langton.

3
Cellular Automata

• Cellular Automata are "Systems of Finite
  Automata," i.e. Deterministic Finite Automata
  (DFAs) arranged in a lattice structure. The input
  tp each DFA is the collective state of itself and
  some group of nearby cells considered its
  neighborhood, N.
• Each cell contains the same DFA as the others and
  the same neighborhood template.
• "Uniform" CAs are most commonly studied, where
  the state transition functions are the same for
  all automatons. (But non-Uniform CAs have been
  shown to be able to solve some problems that
  Uniform CAs cannot.)
• The authors believe CAs to have properties
  relevant to the studies of computation and
  dynamical systems.

4
The Transition Function

• A DFA consists of a set of states Q and rules
  that transition between them based on an input.
• The authors are interested in what properties of
  the ruleset correspond to CAs which are most
  likely to be capable of performing useful
  computations.
• The number of possible rule sets for a DFA is
  outputsinputs. (where inputs is the number
  of possible inputs times the number of states)
• For example, for a DFA with 2 states and 3
  possible input values, there are 2(32) 64
  possible rule sets. Here are the first few
• s0(0)s0, s0(1)s0, s0(2)s0, s1(0)s0, s1(1)s0,
  s1(2)s0
• s0(0)s0, s0(1)s0, s0(2)s0, s1(0)s0, s1(1)s0,
  s1(2)s1
• s0(0)s0, s0(1)s0, s0(2)s0, s1(0)s0, s1(1)s1,
  s1(2)s0
• s0(0)s0, s0(1)s0, s0(2)s0, s1(0)s0, s1(1)s1,
  s1(2)s1
• s0(0)s0, s0(1)s0, s0(2)s0, s1(0)s1, s1(1)s0,
  s1(2)s0

5
The Transition Function

• In a Cellular Automaton, the input is the state
  of a neighborhood of N cells. The number of
  possible inputs is therefore QN.
• Thus, the rule space the number of possible
  rule sets, in a CA is outputsinputs
  QQN.
• Thats a lot of possible rule sets. For the game
  of live, we have 229 approximately 10153.

6
How Can We Analyze 10153 Possible Rulesets?

• We would like to partition, or organize, the
  space of rulesets in some structured way, so that
  we can calculate some parameter, or attribute, of
  each rule set which might correspond, in some
  rough way, with those rulesets that produce CAs
  capable of performing computation.
• Such a parameter should be designed so that
  rulesets with similar parameter values have
  similar properties.

7
Introducting ? (lambda)

• Christopher Langton proposed the parameter ? as
  an important property of rulesets.
• Pick one state in your CA to be considered the
  quiescent, or inactive state, sq.
• Let there be n randomly selected transitions to
  sq in the ruleset. Let all other transitions be
  selected uniformly and randomly to states other
  than sq.
• Define ? as (QN-n)/QN. Essentially, this
  means that ? is the proportion of all transitions
  that lead to the quiescent state sq.

8
Introducting ? (lambda)

• Some interesting values of ?
• ? 0 means that all transitions are to sq. This
  is the most homogeneous scenario.
• ? 1 means that no transitions are to sq.
• ? 1.0 1/Q means that all states are equally
  represented in the rule set. (Ill elaborate on
  the next page). This is the most heterogeneous
  scenario. For the Game of Life, where each cell
  has two states, ? 1.0 1/Q 0.5

9
Derivation of ? 1.0 1/Q

• There are Q states, so if we want each state to
  be equally represented, 1/Q of all transition
  rules should result in each state.
• There are QN rules in each ruleset, so n, the
  number of rules resulting in sq, must be n
  1/Q QN QN-1
• So, ? (QN-n)/QN (QN -
  QN-1)/QN 1 QN-1/QN 1
  1/Q
• The other transitions are equally distributed
  because we required they be chosen randomly and
  uniformly.

10
Searching with ?

• ? blurs the space of rulesets, washing out
  small differences, and is used as a
  low-resolution survey to identify interesting
  areas of the rule space.
• The authors search the rulespace by stepping
  through the range of ? in discrete steps and
  examining the behavior of the CA system to find a
  relationship between ? and behavior.

11
Additional Restrictions

• Before beginning the search, however, the authors
  add a few more restrictions on rulesets in the
  hope of obtaining more meaningful results
• Quiescence When all cells in the neighborhood
  are in the quiescent state sq, the rule should
  map to sq.
• Strong queiescence When all cells in
  neighborhood are in a given state si, the rule
  should map to si. (Note that many systems like
  the Game of Life do not follow this restriction)
• Spatial Isotropy All planar rotations of a
  neighborhood will map to the same state.
• I know we all want to get on with it and see what
  the search found. But first, a bit of a
  digression to try and explain what we are
  actually searching for

12
Brief Detour CAs and Dynamical Systems

• Dynamical Systems are complex systems with
  variables that change over time and for which we
  cannot find formal mathematical solutions.
• Rather than attempting to find exact solutions or
  statistical approximations, Dynamical Systems
  researchers try to analyze, categorize, and
  describe the geometric and topological structure
  of solutions.

13
Phase Space

• The study of a dynamical system frequently
  involves the analysis of the systems phase
  space, the space of all possible values that the
  systems variables can take on.
• The state points taken over time form a
  trajectory of the system.
• It is this trajectory which is analyzed for its
  geometric and topologic properties.

14
Behavior of Dynamical Systems

• There have been three common behaviors identified
  in many dynamical systems
• (1) Fixed Point Attractive the systems
  behavior stabilizes to a single point in state
  space. The system is said to tend towards a
  fixed attractor.
• (2) Periodic Attractive the systems behavior
  stabilizes to a closed, repeating path through
  state space. It is said to tend towards a
  periodic attractor.

15
Behavior of Dynamical Systems

• (3) Chaotic Attractive Behavior never really
  stabilizes, but it does seem to track to a
  bounded manifold (a multidimensional surface in
  phase space) with a complex structure. Similar
  start states do NOT produce similar long term
  trajectories. Said to tend towards a strange
  attractor.

16
Dynamical Systems and CAs

• How can we relate this back to Cellular Auomata?
• Wolfram (1984) identified four main types of CAs
• Class I From almost any initial state, the CA
  degrades to a homogeneous state in a finite
  amount of time.
• Class II The CA evolves into simple periodic
  structures. (the crystalline growth the Prof.
  Simha mentioned).
• Class III The CA tends towards aperiodic
  patterns. After many stems, the systems become
  statistically indistinguishable (maximal
  disorder).
• Class IV The CA produces stable, periodic and
  propagating structures which can last for a long
  time. Final states with any cycle length can be
  obtained with the right initial structure. A
  great deal of local order. Such CAs exhibit very
  long transient lengths, having no direct analogue
  in the field of dynamical systems.

17
Wolframs Classes of CAs
18
Transient Behavior

• We can examine a dynamical systems behavior by
  putting it in a non-typical state and watching
  the resulting behavior as it moves towards its
  attractor.
• The time period between the initial state and the
  settled state is known as the transient
  behavior.
• The authors are interested in how long the
  transient period is and what its relationship is
  to the size of the system (for a CA, the number
  of cells).

19
Transient Behavior

• For class I, II, and III systems, there is little
  relationship between transient length and system
  size.
• For class IV systems (and for dynamical systems
  which are in-between periodic and chaotic), the
  transient length seems to show a strong
  dependence on the size of the system. Sometimes
  the transient period can become more-or-less
  infinite.

20
Brief Detour 2 CAs and Computation

• Christopher Langton Cellular Automata can be
  viewed as (1) computers themselves or as (2)
  logical universes within which computers may be
  embedded.
• In case (1), the starting configuration of the CA
  is the data to be processed and the state
  transition function represents the algorithm
  being computed on that data.
• In case (2), the starting configuration is a
  computer, and input data, and algorithm, and the
  transition function is the physics under which
  the computer operates.
• It has been shown that some CAs (such as the game
  of Life) are capable of performing any algorithm,
  i.e. they are universal general-purpose
  computers.

21
CAs and Computation

• It is believed that a CA must have certain
  properties to be capable of universal general
  computation
• It must support the storage of information
  local regions of state information which can be
  preserved for long periods of time.
• It must support the transmission of information
  the ability for small regions of state
  information to propagate over long distances.
• The stored/transmitted information must be able
  to interact with/modify each other.
• There seems to be a relationship between
  Wolframs Class IV CAs and computational
  capability.

22
Finally Searching the ? Space

• Langton and Avnet each performed a search using
  circular one dimensional CAs with 128 cells, with
  a neighborhood size of 5 (the cell being updated
  and its two neighbors on either side). The CAs
  were randomly chosen for specific ? values.
• It really was an experiment, similar to
  experiments in biology or chemistry, where
  initial parameters are altered and the outcome is
  observed.

23
Results

• ? 0.0 After the first step, the entire array
  is in sq
• ? 0.15 Decay to sq takes 4 to 5 steps
• ? 0.2 Permanent periodic structures can be
  produced, transient time is 7-10 steps
• ? 0.25 Cells can get stuck in a single
  (non-sq) state
• ? 0.4 Periodic structures exist with periods
  up to 40 steps, transient time is up to 60 steps
  before system collapses to isolated areas of
  periodic activity
• ? 0.45 Transient length nearly 1000 steps
  near balance between collapsing and expanding
  activity propagating structures with up to
  15000 steps are possible
• ? 0.5 Transient Time 12000 steps Long
  transients leading to sudden break-down
• ? 0.55 Transient activity is now the
  long-term behavior, system now tends towards
  chaotic
• ? 0.65 System becomes chaotic within 10 steps
• ? 0.75 Transient time is decreasing. System
  becomes chaotic in about 1 step (quick decay to a
  strange attractor).

24
Sample Results
25
Analysis

• The authors categorize the behavior into four
  regimes

Periodic Regime
Transition Regime
Chaotic Regime
Fixed Regime
0.0
1.0
0.5
?
26
Analysis

• The authors note that they observed all 4 classes
  of CA behavior by varying ?, which they suggest
  means ? is a good choice of parameter.
• The transition regime tends to support
  propagating structures (like the gliders from
  the Game of Life). Langton observed that
  propagating structures that collide with static
  structures can produce a new structure that
  propagates in the opposite direction.
• This sort of interaction would seem to allow for
  the storage/transmission/modification of
  information needed for a CA to perform general
  purpose computation.

27
? versus Complexity
28
Phase Transition

• In some experiments, the transition regime seems
  to be just a sharp dividing line between periodic
  and chaotic regimes.
• Other times, it is more of a smooth transition
  range a phase transition between degrading to
  periodic and degrading to chaotic behavior.

29
Analogies

• We can make an analogy between the behavior of
  CAs and fundamental attributes of computation.
• When ? is near the critical value, either chaotic
  or periodic outcomes are possible.
• The transient times are very long (effectively
  infinite, the authors claim), making the question
  of which outcome will occur essentially
  undecidable.
• This is reminiscent of the undecideability of the
  Halting Problem in computer theory. Some turing
  machines can be determined to either halt or not
  halt, but for some systems the problem is
  undecidable.
• Could it be that these problems are near some
  sort of phase transition?

30
CAs explain Computation

• Langton goes so far as to suggest that many
  aspects of general computational theory can be
  explained by phase transitions
• Computability classes (computable, not
  computable) can be explained by the transition
  between ordered/disordered regimes.
• Undecidability is explained by transient times
  approaching infinity at the phase transition.
• Complexity classses (polynomial, exponential,
  etc), can be explained by the increase in
  transient length at the phase transition.
• Universal computation itself is explained as the
  susceptability of the system near the phase
  transition. This means, roughly, the
  responsiveness of the system to small changes in
  state. (Does this mean that ?(a rock) 0.0,
  ?(the ocean) 1.0, and ? (a computer) pc (the
  critical point of phase transition)??)

31
Wow, Neat! . so whats next?

• The authors believe that this is a profoundly new
  way of understanding computing that will have
  many applications
• If phase transitions and information processing
  are closely related, then maybe we can explain
  confusing aspects of dynamical systems through
  the theory of computation.
• There is an analogy between the phases of matter
  (solid/fluid) and the phases of CAs
  (static/chaotic) and the phase transitions
  between them. Could it be that the phases of
  matter are just another dynamical system with a
  phase transition? Then it could be possible to
  reproduce the world of matter in the world of
  computers.

32
The Authors Conclude

• The phase transition manifested in the CA state
  space gives us a new tool to explore and explain
  the concept of computation.
• Additionally, it may help explain the complex
  behavior of dynamical systems near phase
  transitions.
• Case closed.

33
NOT SO FAST!!!

• M.Mitchell, J.P.Crutchfield, and P.T.Hraber.
  Dynamics, computation, and the edge of chaos''
  A re-examination. In G. Cowan, D. Pines, and D.
  Melzner (editors), Complexity Metaphors, Models,
  and Reality. Reading, MA Addison-Wesley, 1994.

34
Another View

• The authors of this paper believe that Langton
  and the other proponents of the ? parameter have
  applied questionable assumptions
• First, they question whether ?, and the rule
  tables it is derived from, are actually the
  drivers of dynamical behavior. In dynamical
  systems theory, it is assumed that functions on
  the equations of motion (the equivalent of
  transition rules) are inadequate to describe the
  behavior of the system.

35
Another View

• Second, they question the assumption that ?, and
  the other parameters Langton analyzed, like
  transient time and entropy, actually converge.
  They say this is not always true.
• Third, and most strongly, they wonder why even
  assume that the statistics Langton describes are
  really the only measures of complexity and
  computational potential?
• The authors support their argument by critical
  analysis of another CA experiment performed by
  Packard (1984).

36
Packards Experiment

• Packard began with a rule set called GKL for a
  one dimensional CA with a neighborhood size of 3.
  
• When using the GKL rule, an input state with less
  than half ones would trend towards a final state
  of all zeroes. When used with an input state of
  more than half ones, it would trend towards a
  final state of all ones.
• The algorithm is not correct in all cases, but it
  is very good (about 98 correct).

37
Packards Experiment

• The problem is not actually as easy to solve as
  it sounds for a 2 state, 3 neighbor CA.
  Essentially, it corresponds to the recognition of
  a non-regular language, and it requires
  transmission of information across long
  distances.
• Packard used a genetic algorithm to try and
  improve the rule.

38
Digression

• Can this problem (determining majority 1s or
  majority 0s) ever by solved perfectly by a CA?
  The papers dont really address this question.
• Thought 1 A CA is a set of DFAs put together,
  which means it is still just a big finite
  automaton. It can never really distinguish an
  infinite, non-regular language.
• Thought 2 True, but with this kind of DFA we add
  many more states as the input size grows. We know
  that some CAs can perform any algorithm, so maybe
  the real question is can a 1-dimensional,
  3-neighbor CA solve the problem, and if so, can
  it be done with only as many cells as there are
  input bits? (If not, how much additional memory
  is required?)

39
Packards Prep

• Before performing his experiment, Packard wanted
  to find the critical ? points for this kind of
  CA (the points of phase transition).
• He did this by statistically analyzing the
  behavior of ? , the difference-pattern spreading
  rate, a measure of chaotic behavior, as a
  function of ?.

40
The Critical Points

• Packard identified critical points at about
  ?0.25 and ?0.8

41
The Genetic Algorithm

• Packard wanted to show that the best rulesets for
  solving the problem would be found near the phase
  transition points, and also that genetic
  algorithms would evolve algorithms towards those
  points.
• The genetic algorithm started with 100 randomized
  CAs, then picked the 50 that were best at solving
  the problem, discarded the rest, and generated
  fifty new ones by mutating the first 50. Each
  repetition of that process is a generation.

42
Packards Result

• Sure enough, in the end, the algorithms evolved
  primarily towards the critical points.

43
Packards Result

• The results seemed to support Packards
  hypothesis that the best algorithms for solving
  the problem would be at or near the phase
  transition points.

44
The Authors Experiment

• The authors attempted to replicate Packards
  experiment with minor differences. Their results,
  however, were quite different

45
The Authors Explain

• The authors have several explanations for why
  their results differed from Packards.
• First Explanation the authors also had two
  towers that the algorithms tended to converge
  to, like Packard, but they were much closer to
  ?0.5 than Packards results, and they were not
  near the critical points. The authors believe
  this is because an optimal solution to the more
  zeroes or more ones problem MUST be close to
  ?0.5. (the initial GKL rule has ?0.5)
• They do not provide a proof in this paper, but
  they state intuitively that, for any CA with ?
  below 0.5, there must be some rules which
  decrease the number of ones in a neighborhood.
  Thus, there must be some initial conditions of
  the CA, with majority 1s, for which the CA will
  decrease the number of 1s rather than increasing
  it, which is the desired behavior. As ? gets
  further away from 0.5, the number of such
  configurations must increase, reducing the
  effectiveness of the algorithm. The reverse
  applies for majority 0.

46
The Authors Explain

• Also, the authors say that the combinatorial
  drift force in genetic algorithms, where random
  actions of mutation tend to push the algorithms
  towards ?0.5.
• The authors explain the dip at ?0.5 as a
  weakness of the genetic algorithm, which seemed
  to pick up and amplify one of two strategies
  (expanding clusters of 1s or expanding clusters
  of 0s) rather than using both of them as the GKL
  rule does.

47
The Authors Explain

• The authors do not claim to know exactly why
  Packards results came about the way they did,
  but they suspect that there was some additional
  randomization or other aspect of Packards
  procedure that has not become public.

48
The Implications

• The authors conclude that ? may not be a good
  indicator of computational ability. First, they
  say that Packards experiment was the only strong
  link between ? and computational capability
  (aside from the fact that the game of life is
  capable of universal computation, and it has ? at
  the critical point), and they call his result
  into question.
• Since they believe they can show that a good
  solution to the problem requires ? be close to
  0.5, they say there is no reason to believe there
  is any generic relationship between ?, the
  critical points, and the computational capability
  of a CA.
• The authors propose to find new ways of analyzing
  the computational potential of a CA, that do not
  rely on rough, statistical parameters like ?.

49
Conclusion

• Between the two papers, the question of the
  relationship between dynamic systems, phase
  transitions, CAs, and computation is uncertain,
  but intriguing.
• Fortunately, group A2 will sort this all out.