Stochastic Neural Network

1
Stochastic Neural Network

• Alokik Kanwal
• Rajeev Rao
• Rutgers University
• ECE Department

2
Project Description

• This project implements a 3-layer by 4-neuron
  perceptron network
• The neurons are used as idealized computing
  elements to model simple digital circuits
• The network has been developed using stochastic
  neurons

3
Introduction to Neural Networks
The Neuron
Threshold Numerical value indicating the bias
associated with the neuron Activation Function
This is a function that determines the conditions
at which the neuron will fire
4
Introduction to Neural Networks
The Network
Wij
Wjk
Whj
Weight Wij Determines the strength of the
connection between neuron i and neuron j A
network is a weighted graph that consists of a
set of neurons (vertices) interconnected by
weight lines (edges)
5
Our Neural Network

• The input and output signal states of the logic
  gate are related through an energy function
• The thresholds and line weights are calculated
  for each neuron such that the minimum-energy
  states correspond to the gates function

6
Creating The Digital Circuit
AND Q P . C
OR F G Q
7
Creating The Digital Circuit
Using the principle of superposition
F G P . C
8
The Stochastic Neuron
9
The Stochastic Neuron
Components

• SRw Shifts the appropriate weight bit
• Xor Xors the input bit with the weight bit
• Tshift Holds and shifts the threshold of the
  neuron
• Counter Implements the 0-1 hard threshold
  limiting activation function
• Sra Shifts the output of the counter to the
  next level

10
Benefits of using the stochastic neuron

• Avoids using complicated hardware units such as
  adders, subtractors and multipliers to compute
  the activation function
• A simple 2-input xor gate along with the shifter
  units performs the same function
• Time multiplexing of neuron I/O lines reduces the
  need for extra wiring between the layers

11
Construction of the multi-layer network

• The outputs of one level are time multiplexed on
  to the inputs of the next level
• The number of logical inputs to a particular
  neuron is equal to the number of neurons in each
  layer
• The final output bit stream is converted back
  into real values using a fast binary counter

12
Conversion of real values to stochastic values

• The modulator is used to assign a probabilistic
  binary value to the output based on the value of
  the input
• We use n modulators in series to convert n real
  bits to n stochastic bits

13
Summary of Design Process

• Behavioral descriptions of the individual
  components of the stochastic neuron as well as
  the entire network was written in VHDL
• Used Schematic editor to create logic and
  switch-level schematics
• Layout was designed unit-by-unit with emphasis
  placed on allowing room on the chip for routing
  different components

14
Simulation Activities

• Behavioral simulation of VHDL code using vhdldbx
• Analog simulation of individual components as
  well as composite units
• Logic simulation of some of the components

15
Sample Schematic Neuron Counter

• Logic Schematic

16
Sample Layout Neuron Counter
17
Sample Layout Complete Neuron
Counter
SRw
Mux
T_Shift
SRa
18
Full Chip Layout
Internal
Full
19
Pin Assignment
1
2
3
4
5
6
7
8
26
25
24
23
22
21
20
19
16
15
14
13
12
11
10
9
18
17
20
Problems and Challenges

• Adapting the stochastic neuron design to our
  neural network model We developed intermediate
  control units between layers to synchronize
  signals
• Problems with designing a neural network model
  for a digital circuit We picked the unitary
  part of a carry-lookahead adder for an example
  circuit
• Problems with Cadence tools DRC, LVS, Layout
  Simulator etc.,

21
Testing Approach

• The circuit was tested at each level of
  development individual modules were simulated
  thoroughly
• VHDL test-benches were written to test all of the
  VHDL code
• Due to the limited number of inputs and the
  simplicity of the interconnections between
  modules, test pattern generators are suitable for
  testing

22
Transistor Count and Chip Area

• Each Neuron has 822 transistors. Area is
  247.05mm by 90.45mm
• Each Weight_Unit has 740 transistors. Area is by
  221.10mm by 100.65mm
• Bit_Modulator has 2170 transistors. Area is
  650.10mm by 138.30mm
• PRSG_Input has 1235 transistors. Area is
  835.20mm by 63.15mm
• Full Chip has 12 Neurons, 12 Weight_Units, 1
  Bit_Modulator and 1 PRSG_Input.
• Total Number of transistors 22,149

23
Preliminary Guesstimates

• Timing Calculations A 2.0 MHz clock signal will
  be appropriate for the circuit. Critical path
  delay is about 520ns.
• Power Consumption

24
Status of Design

• All the components of the chip (Neuron,
  Weight_Unit etc.,) are complete. They have been
  assembled on the chip.
• Analog simulation of individual units is
  complete.
• Documentation (data sheet, final report) is
  complete

25
References

• Max van Daalen, Peter Jeavons, John Shawe-Taylor,
  A stochastic neural architecture that exploits
  dynamically reconfigurable FPGAs, IEEE Workshop
  on FPGAs for Custom Computing Machines, pp
  202-211, 1993
• Max van Daalen, Peter Jeavons, John Shawe-Taylor,
  David Cohen, Device for generating binary
  sequences for Stochastic Computing, Electronics
  Letters, Vol 29, No 1, Jan 1993, pp 80-81
• S. T. Chakradhar, V. D. Agrawal, M. L. Bushnell,
  Neural Models and Algorithms for digital
  testing, Kluwer Academic Publishers, Boston,
  c1991

26
Any Questions ?