Hopfield Neural Networks for Optimization

1
Hopfield Neural Networksfor Optimization

• ???

??????? ?????????
2
Content

• Introduction
• A Simple Example ? Race Traffic Problem
• Example ? A/D Converter
• Example ? Traveling Salesperson Problem

3
Hopfield Neural Networksfor Optimization

• Introduction

??????? ?????????
4
Energy Function of a Hopfield NN
Interaction btw neurons
Interaction to the external
constant
Running a Hopfield NN asynchronously, its energy
is monotonically non-increasing.
5
Solving Optimization Problems Using Hopfield NNs

• Reformulating the cost of a problem in the form
  of energy function of a Hopfield NN.
• Build a Hopfield NN based on such an energy
  function.
• Running the NN asynchronously until the NN
  settles down.
• Read the answer reported by the NN.

6
Hopfield Neural Networksfor Optimization

• A Simple Example
• Race Traffic Problem

??????? ?????????
7
A Simple Hopfield NN
8
The Race Traffic Problem
9
The Race Traffic Problem
10
The Race Traffic Problem
?1
?1
1
Stable State
11
The Race Traffic Problem
?1
?1
1
Stable State
12
The Race Traffic Problem
?1
?1
How about if to run synchronously?
13
Hopfield Neural Networksfor Optimization

• Example
• A/D Converter

??????? ?????????
14
Reference
Tank, D.W., and Hopfield, J.J., Simple "neural"
optimization networks An A/D converter, signal
decision circuit and a linear programming
circuit, IEEE Transactions on Circuits and
Systems, Vol. CAS-33 (1986) 533-541.
15
A/D Converter
A/D
I
Analog
Using Unipolar Neurons
16
A/D Converter
Using Unipolar Neurons
17
A/D Converter
18
Hopfield Neural Networksfor Optimization

• Example
• Traveling Salesperson Problem

??????? ?????????
19
Reference
J. J. Hopfield and D. W. Tank, Neural
computation of decisions in optimization
problems, Biological Cybernetics, Vol. 52,
pp.141-152, 1985.
20
Traveling Salesperson Problem
21
Traveling Salesperson Problem
Given n cities with distances dij, what is the
shortest tour?
22
Traveling Salesperson Problem
2
3
4
1
5
6
9
11
8
10
7
23
Traveling Salesperson Problem
Distance Matrix
Find a minimum cost Hamiltonian Cycle.
24
Search Space
Assume we are given a fully connection graph with
n vertices and symmetric costs (dijdji).
The size of search space is
Find a minimum cost Hamiltonian Cycle.
25
Problem Representation Using NNs
Time
2
1
4
City
3
5
26
Problem Representation Using NNs
The salesperson reaches city 5 at time 3.
Time
2
1
4
City
3
5
27
Problem Representation Using NNs
Goal Find a minimum cost Hamiltonian
Cycle.
Time
2
1
4
City
3
5
28
The Hamiltonian Constraint
Goal Find a minimum cost Hamiltonian
Cycle.
Time

• Each row and column can have only one neuron
  on.
• For a n-city problem, n neurons will be on.

2
1
4
City
3
5
29
Cost Minimization
Goal Find a minimum cost Hamiltonian
Cycle.
Time

• The total distance of the valid tour have to be
  very low.

2
1
4
City
The summation of these dijs is very low.
3
5
30
Indices of Neurons
i
Time
vxi
City
x
31
Energy Function
32
Energy Function
n neurons on
Each row one or zero neuron on
Each column one or zero neuron on
33
Energy Function
Total distance of the tour
34
Energy Function
35
Build NN for TSP
Mapping
Energy function of a 2-D neural network
36
Analog Hopfield NN for 10-City TSP
37
Analog Hopfield NN for 10-City TSP
The shortest path
38
Analog Hopfield NN for 10-City TSP
The shortest path
39
Analog Hopfield NN for 30-City TSP
40
Analog Hopfield NN for 30-City TSP