A%20Bifurcation%20Theoretical%20Approach

1
A Bifurcation Theoretical Approach to the
Solving the Neural Coding Problem
June 28 Albert E. Parker
Complex Biological Systems Department
of Mathematical Sciences Center for
Computational Biology Montana State
University Collaborators Tomas Gedeon, Alex
Dimitrov, John Miller, and Zane Aldworth
2
Outline

• The Neural Coding Problem
• A Clustering Problem
• The Role of Bifurcation Theory
• A new algorithm to solve the Neural Coding
  Problem

3
The Neural Coding Problem GOAL To understand
the neural code. EASIER GOAL We seek an answer
to the question, How does neural activity
represent information about environmental stimuli?
The little fly sitting in the flys brain trying
to fly the fly
4
Looking for the dictionary to the neural code
stimulus X
response Y
5
but the dictionary is not deterministic!
Given a stimulus, an experimenter observes many
different neural responses
X
Yi X i 1, 2, 3, 4
6
but the dictionary is not deterministic!
Given a stimulus, an experimenter observes many
different neural responses
X
Yi X i 1, 2, 3, 4
Neural coding is stochastic!!
7
Similarly, neural decoding is stochastic
Y
XiY i 1, 2, , 9
8
Probability Framework
encoder P(YX)
environmental stimuli
neural responses
Y
X
decoder P(XY)
9
The Neural Coding ProblemHow to determine the
encoder P(YX) or the decoder P(XY)?

• Common Approaches parametric estimations, linear
  methods
• Difficulty There is never enough data.
• As we attempt search for an answer to the neural
  coding problem, we proceed in the spirit of John
  Tukey
• It is better to be approximately right than
  exactly wrong.

10
One Approach Cluster the responses
Stimuli
Responses
Clusters
Y
YN
X
q(YN Y)
p(X,Y)
N objects yNi
L objects xi
K objects yi
11
One Approach Cluster the responses
Stimuli
Responses
Clusters
Y
YN
X
q(YN Y)
p(X,Y)
N objects yNi
L objects xi
K objects yi

• To address the insufficient data problem, one
  clusters the
• outputs Y into clusters YN so that the
  information
• that one can learn about X by observing YN ,
  I(XYN), is as
• close as possible to the mutual information
  I(XY)

12
Two optimization problems which use this approach

• Information Bottleneck Method (Tishby, Pereira,
  Bialek 1999)
• min I(Y,YN) constrained by I(XYN) ? I0
• max I(Y,YN) ? I(XYN)
• Information Distortion Method (Dimitrov and
  Miller 2001)
• max H(YNY) constrained by I(XYN) ? I0
• max H(YNY) ? I(XYN)

13
An annealing algorithm to solve

• maxq??(G(q)?D(q)) ?
• Let q0 be the maximizer of maxq G(q), and let ?0
  0. For k ? 0, let (qk , ?k ) be a solution to
  maxq G(q) ? D(q ). Iterate the following
  steps until
• ?K ? max for some K.
• Perform ? -step Let ? k1 ? k dk where dkgt0
• The initial guess for qk1 at ? k1 is qk1(0)
  qk ? for some small perturbation ?.
• Optimization solve maxq (G(q) ? k1 D(q)) to
  get the maximizer qk1 , using initial guess
  qk1(0) .

14
Application of the annealing method to the
Information Distortion problem maxq?? (H(YNY)
? I(XYN)) when p(X,Y) is defined by four
gaussian blobs
q(YN Y)
Y
YN
p(X,Y)
Y
X
52 responses
52 stimuli
52 responses
4 clusters
Stimuli
Responses
15
Evolution of the optimal clustering Observed
Bifurcations for the Four Blob problem
We just saw the optimal clusterings q at some ?
? max . What do the clusterings look like
for ?lt ? max ??
16
(No Transcript)
17
Application to cricket sensory data
18
We have used bifurcation theory in the presence
of symmetries to totally describe how the the
optimal clusterings of the responses must evolve
19
Symmetries??
class 1
class 3
YN
Y
q(YNY) a clustering
N objects yNi
K objects yi
20
Symmetries??
class 3
class 1
YN
Y
q(YNY) a clustering
N objects yNi
K objects yi
21
Observed Bifurcation Structure
22
Observed Bifurcation Structure
Group Structure
23
Observed Bifurcation Structure
q
Group Structure
?
24
Continuation techniques providenumerical
confirmation of the theory
25
(No Transcript)
26
Additional structure!!
27
(No Transcript)
28

• Conclusions
• We have a complete theoretical picture of how the
  clusterings of the responses evolve for any
  problem of the form
• maxq??(G(q)?D(q))
• When clustering to N classes, there are N-1
  bifurcations.
• In general, there are only pitchfork and
  saddle-node bifurcations.
• We can determine whether pitchfork bifurcations
  are either subcritical or supercritical (1st or
  2nd order phase transitions)
• We know the explicit bifurcating directions
• SO WHAT?? This yields a new and improved
  algorithm for solving the neural coding problem

29
A numerical algorithm to solve max(G(q)?D(q))
?

• Let q0 be the maximizer of maxq G(q), ?0 1 and
  ?s gt 0. For k ? 0, let (qk , ?k ) be a solution
  to maxq G(q) ? D(q ). Iterate the following
  steps until ?K ? max for some K.
• Perform ? -step solve
• for and select ? k1 ? k dk
  where
• dk (?s sgn(cos
  ?)) /(??qk 2 ???k 2 1)1/2.
• The initial guess for (qk1,?k1) at ? k1 is
  
  (qk1(0),?k1 (0)) (qk ,?k) dk (?? qk,?? ?k)
  .
• Optimization solve maxq (G(q) ? k1 D(q))
  using pseudoarclength continuation to get the
  maximizer qk1, and the vector of Lagrange
  multipliers ?k1 using initial guess
  (qk1(0),?k1 (0)).
• Check for bifurcation compare the sign of the
  determinant of an identical block of each of
  ?q G(qk) ? k D(qk) and ?q G(qk1) ? k1
  D(qk1). If a bifurcation is detected, then set
  qk1(0) qk d_k u where u is bifurcating
  direction and repeat step 3.

30
(No Transcript)