Clustering using Spiking Neural Networks

1
Clustering using Spiking Neural Networks
2
Biological NeuronThe Elementary Processing Unit
of the Brain
3
Biological NeuronA Generic Structure
Axon
Axon Terminal
Synapse
Soma
Dendrite
4
Biological NeuronNerve Impulse Transiting
Membrane Potential
Action Potential (Spike)
Spike-After Potential
Action Potential (Spike)
Postsynaptic Potential
5
Biological NeuronSoma Firing Behavior
Synchrony is the main factor of soma firing
6
Biological NeuronInformation Coding
Firing rate alone does not carry all the relevant
information
Neurons communicate via exact spike timing
7
Neuroscience Models of NeuronThe Hodgkin-Huxley
Model
Alan Lloyd Hodgkin and Andrew Huxley received the
Nobel Prize in Physiology and Medicine in 1963
The Hodgkin-Huxley model is too complicated model
of neuron to be used in artificial neural networks
8
Neuroscience Models of NeuronLeaky
Integrate-And-Fire Model
or
Leaky Integrate-And-Fire model disregards the
refractory capability of neuron
9
Neuroscience Models of NeuronSpike-Response
Model
Spike-Response model captures the major elements
of a biological neuron behavior
10
Biological Neuron Computational Intelligence
ApproachThe First Generation
The first artificial neuron was proposed by W.
McCulloch W. Pitts in 1943
11
Biological Neuron Computational Intelligence
ApproachThe Second Generation
Multilayered Perception is a universal
approximator
12
Biological Neuron Computational Intelligence
ApproachArtificial Neurons Too Artificial?
From neurophysiology point of view, y is
existence of an output spike
Number of spikes
Time frame
From neurophysiology point of view, y is firing
rate
Spike timing is not considered at all!
13
Biological Neuron Computational Intelligence
ApproachThe Third Generation
Spiking neuron model was introduced by J.
Hopfield in 1995
Spiking neural networks are - biologically
more plausible, - computationally more
powerful, - considerably faster than networks
of the second generation
14
Spiking Neural NetworkOverall Architecture
Spiking neural network is a heterogeneous
two-layered feed-forward network with lateral
connections in the second hidden layer
RN is a receptive neuron
MS is a multiple synapse
SN is a spiking neuron
15
Spiking Neural NetworkPopulation Coding
Input spike
Pool of Receptive Neurons
16
Spiking Neural NetworkMultiple Synapse
Spike-response function
Delayed postsynaptic potential
Total postsynaptic potential
Membrane potential
17
Spiking Neural NetworkHebbian Learning WTA
and WTM
Winner-Takes-All
Winner-Takes-More
Proposed for the first time on the 11th
International Conference on Science and
Technology System Analysis and Information
Technologies (Kyiv, Ukraine, 2009) by Ye.
Bodyanskiy and A. Dolotov
18
Spiking Neural NetworkImage Processing
Original Image
SOM at 50 epoch
SNN at 4 epoch
In Bionics of Intelligence 2007, 2 (67), pp.
21-26 by Ye. Bodyanskiy and A. Dolotov
19
Spiking NeuronThe Laplace Transform Basis
From control theory point of view, action
potential (spike) is a signal in pulse-position
form
Thus, transformation of action potential to
postsynaptic potential taken into synapse is
nothing other than pulse-position
continuous-time transformation, and soma
transformation is just reverse one,
continuous-time pulse-position transformation
20
Spiking Neuron SynapseA 2nd order critically
damped response unit
Proposed for the first time on the 6th
International Conference Information Research
and Applications (Varna, Bulgaria, 2009) by
Ye. Bodyanskiy, A. Dolotov, and I. Pliss
21
Spiking NeuronTechnically Plausible Description
Incoming Spike
Time Delay
Membrane Potential
Spike-Response Function
Relay
Outgoing Spike
Proposed for the first time on the 6th
International Conference Information Research
and Applications (Varna, Bulgaria, 2009) by
Ye. Bodyanskiy, A. Dolotov, and I. Pliss
22
Spiking NeuronAnalog-Digital Architecture
Analog-digital spiking neurons corresponds to
spike-response model entirely
Proposed for the first time in Image Processing
/ Ed. Yung-Sheng Chen In-Teh, Vukovar, Croatia,
pp. 357-380 by Ye. Bodyanskiy and A. Dolotov,
23
Fuzzy Receptive Neurons
Pool of receptive neurons is a linguistic
variable, and a receptive neuron within a pool is
a linguistic term.
Proposed for the first time in Information
Technologies and Computer Engineering 2009,
2(15), pp. 51-55 by Ye. Bodyanskiy and A. Dolotov
24
Fuzzy Spiking Neural NetworkFuzzy Probabilistic
Clustering
There is no need to calculate cluster centers!
Proposed for the first time in Sci. Proc. of
Riga Technical University 2008, 36, P. 27-33 by
Ye. Bodyanskiy and A. Dolotov
25
Fuzzy Spiking Neural NetworkFuzzy Possibilistic
Clustering
Proposed for the first time on the 15th Zittau
East-West Fuzzy Colloquium (Zittau, Germany,
2008) by Ye. Bodyanskiy, A. Dolotov, I. Pliss,
and Ye. Viktorov
26
Fuzzy Spiking Neural NetworkImage Processing
Original image
Training set
SOM at 40th epoch
FSNN at 4th epoch
In Proceeding of the 4th International
School-Seminar Theory of Decision Making
(Uzhhorod, Ukraine, 2008) by Ye. Bodyanskiy, A.
Dolotov, and I. Pliss
27
Fuzzy Spiking Neural NetworkImage Processing
Original image
Training set
FCM at 29th epoch
FSNN at 3rd epoch
In Proceeding of the 11th International
Biennial Baltic Electronics Conference "BEC 2008
(Tallinn/Laulasmaa, Estonia, 2008) by Ye.
Bodyanskiy and A. Dolotov
28
Fuzzy Spiking Neural NetworkImage Processing
Original image
Training set
FSNN at 1st epoch
FSNN at 3rd epoch
FCM at 3rd epoch
FCM at 30th epoch
In Image Processing / Ed. Yung-Sheng Chen
In-Teh, Vukovar, Croatia, pp. 357-380 by Ye.
Bodyanskiy and A. Dolotov