Kein Folientitel

1
Information Processing with Pulsed Neural
Networks
Ulrich Ramacher Corp. Research, Infineon
Technologies AG, Munich ulrich.ramacher_at_infin
eon.com
2
Dilemma (1)
lack of robustness
3
Dilemma (2)
lack of architecture information
4
Dilemma (3)

• Non-invasive long-term recording of neural
  tissue.
• High-density sensor with 128?128 Sensors in
  1?1mm².
• Extended CMOS-process with biocompatible high-k
  surface dielectric.
• Self-calibration circuitry and pre-amplification
  on chip.
• Applications in neurobiology and drug discovery

Infineon Neurochip
Gap between signal processing by few cells and
information processing by large arrays
5
Program

1. Use pulsed IAF neurons and adaptive synapses
   (inhibitory, excitatory)
2. Use experimental evidence
3. Start from simple, complete vision systems
4. Find basic network structures for information
   processing
5. Find quantitative ansatz for information
   processing
6. Demonstrate usefulness for cell phones

6
Neuron Model
Rule 1 Reaching the threshold, a neuron resets
membrane potential to zero and starts sending a
pulse of 1 ms Rule 2 A neuron either receives or
sends
? deterministic dynamical system
7
Experiment 1
8
Experiment 1
Frequency
9
Observations

• Firing patterns dont come to an end
• information processing is not a function of time

? Does frequency of firing patterns
characterize the net ?
Inet -? pn x ln pn
Is entropy a reproducible measure of information?
10
Experiment 2
11
Experiment 2
12
Findings

• distribution function of fps is independent of
  initial conditions

13
Entropy as a function of mean synaptic weight
Each color corresponds to a different realization
of wij
Network size 40 neurons
14
Network size 40 neurons
ISI
Fire Rate
15
Synapse Model (proposed by U. Ramacher, April 99)
Adaptation rule local, causal, simple
16
µ negative
synapses coupled system of damped oscillators
17
µ positive
synapses coupled system of damped
exponentially rising and falling elements
18
Spot Detector Illumination Encoder
19
Pixels 64 x 64
20
LGN
primary visual cortex (V1)
light
retina
optic nerve
Dr. Arne Heittmann
21
convergence
Dr. Arne Heittmann
22
Modeling the Experiment
Bright Stimulus
Dark Stimulus
y0
Dy
Dx
x0
Stimuli projected onto RF of a Simple Cell
H Spot Intensity B Background Intensity
P measured pulse rate Gi Gabor function
23
Pulse difference detector (1)
24
Pulse difference detector (2)
Dynamics of the Synapse W41
25
Pulse difference detector (3)
Characteristic
Neuron 2
Neuron 1
Number of pulses
Neuron 4
i2
i1 0.5 Q 1 WK0 0.08 td 1ms T
0.5s
26
Architecture of feature detector(proposed by A.
Heittmann, 2003)
1
1
1
1
2
2
2
2
- - -
- - -
3
3
3
3
4
4
4
4
5
µ gt 0
µ lt 0
constant
27
Shaping the response-profile of the detector
section of the retina
H Spot Intensity B Background Intensity
28
Results of a detector implementation
measured profile
Gabor-Wavelet

• 256 Gradient detectors
• size of receptive field 17 x 17 Pixel
• T750ms _at_ 1ms Pulse-duration

29
Simulated Filter responses, T750ms
ideal
Real 90
Imaginary 90
Real 0
Imaginary 0
30
The Head-Detector
Restriction
- single scale (keep eye-distance fixed)
31
Check for Robustness in Eye-Brow Zone
Reference Image
Filter Response,horizontal direction
region ofinterest
20x20 Pixel
new eye-browimage
32
A simple memory (1) , 1 zone
Learning Phase
1-1 connection between input and detector
layer 1-1 connection between input and memory
layer
full connection between detector and memory layer
33
Experiment 1 learned image in input and memory
Activity (number of events), 2ms window
Pulse-patterns of input-layer
34
A simple memory (2), 1 Zone
Recognition Phase
Detector Layer
Memory Layer
K
K
1-1 connection between input and detector layer
K
full connection between detector and memory layer
Gabor Layer
35
Experiment 3 recognition of non-learned eye-brow
Activity (number of events), 2ms window
Pulse-patterns of input-layer
36
Experiment 2 Isolated neurons
Activity (number of events), 2ms window
Pulse-patterns of input-layer
37
Zone-Architecture I
Detector Layer 1
Detector Layer 2
Memory Layer
Vertical Orientations
Horizontal Orientations
Gabor-Kernel
Input Image
38
Zone-Architecture II
Zone 5
Zone 1
Zone 6
Zone 2
Zone 7
Zone 3
Zone 4
Zuordnung Zonen zu Bildregionen
Zone für Gabor-Wavlet mit horizontaler
Orientierung
Zone für Gabor-Wavlet mit vertikaler
Orientierung
39
Reference-Image and Test-Images
Reference-Image
Test-Images
Face 0011
Face 0014
Face 0001
40
Face 0001
Activity-Diagram
Memory
Zone 1
Detector
Normalized accumulated activity
Memory
Zone 2
Detector
Memory
Zone 3
Detector
Memory
Zone 4
Detector
Memory
Zone 5
Detector
Time ms
41
Face 0011
Activity-Diagram
Memory
Zone 1
Detector
Normalized accumulated activity
Memory
Zone 2
Detector
Memory
Zone 3
Detector
Memory
Zone 4
Detector
Memory
Zone 5
Detector
Time ms
42
Face 0014
Activity-Diagram
Memory
Zone 1
Detector
Normalized accumulated activity
Memory
Zone 2
Detector
Memory
Zone 3
Detector
Memory
Zone 4
Detector
Memory
Zone 5
Detector
Time ms
43
Binding of zones by synchrony
Binding Layer

1
2
3
4
5
6
7
Zuordnung Zonen zu Neuronen
des Spotdetektors zur Bindung
Memory Layer
44
Results
40ms
40ms
Spot_0001
40ms
Spot_0011
Spot_0014
45
Column Architecture
Image plane
Detector Feature 1Memory Feature 1
Detector Feature 2Memory Feature 2
. . .
robustrecognition
Detector Feature nMemory Feature n



Binding Object 1


Binding Object 2
AssociativeMemory


Binding Object n
46
The vision a 3D-Vision-Cube
The Vision-Cube
CMOS-sensor, sensor array

• 3D-stacking-architecture
• Low-power
• Real time capabilities
• Integration of sensors and information
  processing
• Distributes layers of information
  processing to layers of the stack
• Solves problem of connectivity

Analogue-pulse conversion
Feature-Detection
Gabor-Wavelets, different orientations

Object-recognition Object-detection
47
Design of a testchip for the Synchrony
detector,Base-Chip for the 3D-Stack
pixel memory (1 Pixel)
Infineon 130nm CMOS-Technology
localAER-subcircuit
testcircuits for 3D-interconnects
Test circuit
Integrate-and-Fire-Neuron
3D-vias for supply and digital control signals
adaptive synapses
layout of 1 neuron
rows of 3D-vias for layer-to-layer
signaldistribution
array of 128 x 128 neurons, 64k synapses
control circuit
AER-encoder- circuit
Power consumption 3mW _at_1.5V (analog) ,
40-250mW _at_1.5V (digital)Size 7.6mm x 7.8mm

48
Gabor-Feature-Detector chip (layout) including a
pulse router for the 3D-integration
Infineon 130nm CMOS-Technology
Array of 128x128 (64k) processing elements for
pulse processing and -routing

• Processing element
• photo-Detector
• pulse-processing (gradient detection)
• dynamic routing of pulses

3D- wiring channel for vertical signal
distribution
SRAM for storing routing information

• Digital macro
• routing circuit
• configuration
• AER-circuit (event based)

3D- wiring for power supply and control signals
49
3D-Stacking
Layer 7
Layer 6
Layer 5
12µm
12µm
50
Real 3D !!
Layer 7
Layer 6
Layer 5
Layer 4
Layer 3
13µm
7µm
Layer 2
Bottom Substrate, Layer 1
51
Conclusion

• IAF neurons, adaptive synapses, only
• network built for -- Gabor wavelet
  based feature cascade -- memory
  -- comparison of memory and detector plane
• synchrony of neurons indicative for --
  robust recognition of memory feature at detector
  plane (elastic matching) --
  binding of features as object
• built in 130 nm CMOS -- synchrony
  detector -- Gabor wavelet detector
  -- 3 D stack of 7 silicon chips