Associative memory using coupled non-linear oscillators

1
Associative memory using coupled non-linear
oscillators

• Semester project
• Final Presentation

Vlad TRIFA
2
Project summary

• Litterature review
• Implementation of an associative memory using
  coupled oscillators and analysis of
  performance/drawbacks.
• Mixture with the BIRG model
• Generalization to complex signals
• Better control on the capacity
• Final discussion about relevant issues concerning
  the performances of both models
• Conclusion

3
Associative Memory

• Animal and human memory works by association.
• Able to retrieve a stored pattern upon
  presentation of a partial and noisy
  representation of an input signal.
• Many models developed since early 80s
• Concepts taken from statisic mechanics and
  hebbian learning rule turned neural networks into
  dynamic systems.
• Useful into understanding dynamics of networks
  (emergence)
• but
• Lack of biologically plausible mechanisms
  (coupling, binary,)
• Low capacity and performance (Global coupling N2
  parameters)

4
Oscillators

• Oscillating systems are very common in nature
  and possess very intersting properties.
• Synchronization
• Energy efficient mechanism for temporal
  correlation
• Many brain processes rely on interaction of
  oscillators
• CPG
• Olfactory and visual cortex
• Temporal correlation hypothesis and binding
  problem
• Information can be stored as phase relationships
  patterns, where coupled oscillators converge.

5
Analyzed model

• Can be found in Borisyuk, 2001.
• Oscillators described by phase, amplitude, and
  frequency.

6
The model
7
(No Transcript)
8
Dynamics
9
Model performance

• The capacity of this model is not easy to derive,
  due to the random phase shifts, and to the
  dynamics of the nonlinear term
• We do not know what percentage of overlapping is
  possible, as memorized patterns can be
  overwritten.
• Implies that error increases as memory is filled.
• Robustness due to distributed memorization.
• But, loss of groups influence strongly the
  retrieval error.

10
Discussion

• The model is interesting as it is based on
  oscillating systems, thus can be easily
  implementable on many oscillating systems (PLL,
  etc).
• A very nice methodology that is embedded in the
  system is proposed in order to decide where to
  store an input signal is proposed.
• Random phase shifts ensure some robustness to the
  system, but too big influence on the performance.

11
Drawbacks

• The all-to-all coupling into groups is not
  efficient computationally and it uses too many
  oscillators.
• Due to the explicit input signal embedded in the
  equations, we can only learn sine functions.
• The input dimension is annoying. Complexity is
  increased with no performance increase.
• The time is reset after each stimulus. We need to
  present the input in-phase with the oscillators.
  We cannot learn sequences.

12
Improvements

• We want to be able to learn complex signals.
• Starting from the model in Righetti et al,
  2005, we want to extend the model to form a
  network.

13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
Discussion

• Simpler model, more computationally efficient.
• We gained a much better control on the amount of
  oscillators to dedicate for a frequency
  component.
• We are able to memorize complex signals in a
  robust and fault tolerant manner, under some
  constraints.
• but
• Unfortunately, the capacity depends on the
  complexity of the signals to store.
• We lost the selection of the storage sites based
  on phase relationships we had with the previous
  model.

23
Future work

• We need to find a mechanism (embedded in the
  dynamics) that can select where each component
  should be stored depending on the signal.
• It would be very intersting to create links
  between different clusters activated by the same
  signal, similar to associative connections
  forming according to the correlation of neural
  activity between assemblies, enhancing robustness
  if attenuated components.
• Reduce parameters, so we need only to select the
  amount of oscillators allocated per component.

24
Conclusion

• This work should be considered as an attempt to
  provide insights on how it is possible to store
  information encoded as a complex signals in a
  reliable manner, simply by using oscillating
  systems with local interactions.
• Our approach is interesting as it uses some
  concepts that are common in biological neuronal
  networks such as
• Oscillating components with local interactions
• No global external process to supervise the
  learning procedure

25
Thank you!

• References
• Borisyuk, 2001
• Righetti et al., 2005
• Singer, 1995