Analyzing Evolved FaultTolerant Neurocontrollers

1
Analyzing Evolved Fault-Tolerant Neurocontrollers
Alon Keinan, School of Computer Science, Tel-Aviv
University, Israel (keinanak_at_post.tau.ac.il
www.cns.tau.ac.il/keinan)
Introduction
The Multi-perturbation Shapley value Analysis
(MSA)
Neurally-driven evolved agents are a very
promising model for studying neural processing
and developing methods for its analysis. Being
abstractions, they are missing many qualities of
natural systems. A very important one is
fault-tolerance. In this research,
neurocontrollers that manifest high levels of
fault-tolerance are evolved. Their robustness
increases the challenge of an analysis of their
workings. The MSA (see box) is utilized to
uncover the important neurons and the
interactions between them, revealing the
mechanisms underlying the evolved fault-tolerance.

• Calculating the Shapley value requires full
  knowledge of all possible multi-lesions. Hence,
  the following approximation methods have been
  developedPredicted contributions -
  Contributions based on a prediction of the
  performance in all multi-lesions. The predictor
  is trained with a given multi-lesion data
  set.Estimated contributions - An unbiased
  estimation of the contributions based on a sample
  of multi-lesions.
• In complex networks the importance of an element
  may depend on the state of other elements. Higher
  order descriptions are required for capturing the
  characteristics of such networks. The MSA
  quantifies the interactions between groups of
  elements. Focusing on a two-dimensional analysis,
  the interaction between a pair of elements
  measures how much the whole is greater than the
  sum of its parts (synergism). When the two
  elements exhibit functional overlap (antagonism),
  the interaction is negative.
• Outputs - The contribution of each element to
  the function in question. - A quantification of
  the interaction between groups of elements.
• A more detailed description of the MSA may be
  found in 2.
• Understanding the operations of evolutionary
  neurocontrollers
• Providing insights to the workings of
  neurophysiological models
• Analysis of real neuroscience deactivation/stimul
  ation experiments
• Analysis of gene multi-knockout studies

Background
Goal Identifying the individual roles of a
network's elements.1. Conventionally done by
recording the activity. Disadvantage Identify
correlation, not causality. Example An element
that does not contribute to the processing of a
function is activated by other elements that do
play a role in carrying out the function. 2.
Lesion studies enable a causal identification of
the elements that are responsible for a given
function. Most of the lesion studies in
neuroscience have been single lesion studies
(only one element is lesioned at a
time). Disadvantage limited in the presence of
interacting elements. Example When elements
exhibit a high degree of overlap in their
function, lesioning either element alone will not
reveal its significance.3. Hence -
Multi-lesioning studies, where in each experiment
several elements are concurrently lesioned.
The MSA views each multi-lesion experiment as
a coalition in a coalitional game, borrowing
analytical approaches from the field of Game
Theory. Specifically, the set of contributions
are defined to be the Shapley value 1.
The Evolution

• Agents equipped with fully-recurrent
  neurocontrollers are evolved for solving the
  foraging task from 3. In order to encourage the
  creation of fault-tolerance, faults are
  introduced while an agents fitness is being
  evaluated (as suggested by 4 in the context of
  evolvable hardware)The fitness of an agent is
  evaluated once with each of its neurons lesioned,
  and averaged along with the fitness of the intact
  neurocontroller .Ten evolutionary runs of each
  of the following types were performed
• Incremental evolution An evolutionary run is
  first conducted using the standard fitness,
  resulting in non-robust agents that perform the
  foraging task well. Then, starting from the most
  successful agent in this evolutionary run, agents
  are evolved to be fault-tolerant.
• Direct evolution Agents are evolved to be
  fault-tolerant, starting with random
  neurocontrollers.

The Method
Input Multi-lesion data set
Applications
Results

• The agents are indeed much more fault-tolerant,
  while reaching about the same level of intact
  fitness.
• Though only single lesions are afflicted during
  the evolution, the resulting agents are robust
  against multiple lesions as well.

• Uncovering the underlying fault-tolerance
  mechanisms
• There are many negative interactions in the
  fault-tolerant agents (blue bars), pointing to
  pairs of neurons which backup each others
  function.
• The backup scenario is not a clear-cut case in
  which each redundant neuron has another one
  completely backing it up.
• E.g. each of neurons 4, 9 and 10 backup to some
  extent each of the two others.
• The fault-tolerant agents have much more
  meaningful negative interactions and much less
  positive ones (red). This testifies to the fact
  that the fault-tolerant evolutionary pressure
  encourages the formation of functional overlap
  between the neurons, at the expense of the
  formation of cooperation.

Analyzing the Agents

• Are those results a manifestation of the
  evolution of backup mechanisms or merely an
  outcome of fewer neurons carrying out the same
  function?
• Since the neurocontrollers exhibit
  fault-tolerance to deep levels of lesioning,
  multi-lesion analysis must be used in order to
  address this question
• The MSA reveals the true contributions of the
  neuronsto the function (also of those whose
  importance is missed by the single-lesion
  approach).
• The same number of neurons, on the average, play
  an important part in regular and fault-tolerant
  agents.
• The fault-tolerant agents have a greater number
  of important synapses.
• Hence, the agents are more robust due to the
  evolution of backup mechanisms.

References
1 L. S. Shapley. A value for n-person games. In
Contributions to the Theory of Games, volume II
of Annals of Mathematics Studies, 1953. 2 A.
Keinan, B. Sandbank, C. C. Hilgetag, I.
Meilijson, and E. Ruppin. Fair attribution of
functional contribution in artificial and
biological networks. Neural Computation, 16(9),
2004. 3 R. Aharonov-Barki, T. Beker and E.
Ruppin. Emergence of memory-driven command
neurons in evolved artificial agents. Neural
Computation, 13(3), 2004. 4 A. Thompson.
Evolutionary techniques for fault tolerance. In
Proceedings of CONTROL 96, 1996.