Supervised

1
CHAPTER 7

• Supervised
• Hebbian
• Learning

2
Objectives

• The Hebb rule, proposed by Donald Hebb in 1949,
  was one of the first neural network learning
  laws.
• A possible mechanism for synaptic modification
  in the brain.
• Use the linear algebra concepts to explain why
  Hebbian learning works.
• The Hebb rule can be used to train neural
  networks for pattern recognition.

3
Hebbs Postulate

• Hebbian learning
• (The Organization of Behavior)
• When an axon of cell A is near enough to excite
  a cell B and repeatedly or persistently takes
  part in firing it some growth process or
  metabolic change takes place in one or both cells
  such that As efficiency, as one of the cells
  firing B, is increased.
• ???A??????B??????????,?????????B,?????????????????
  ??????????,???A?B??????

4
Linear Associator
5
Hebb Learning Rule
?Unsupervised learning rule
?Supervised learning rule
6
Supervised Hebb Rule
7
Performance Analysis
8
Performance Analysis
error
9
Orthonormal Case
Success!!
10
Not Orthogonal Case
The outputs are close, but do not quite match the
target outputs.
11
Solved Problem P7.2
i.
Orthogonal, not orthonormal,
ii.
12
Solutions of Problem P7.2
iii.
Hamming dist. 1
Hamming dist. 2
13
Pseudoinverse Rule
?
? Performance index
?
14
Pseudoinverse Rule
15
Moore-Penrose Pseudoinverse
16
Example of Pseudoinverse Rule
17
Autoassociative Memory
18
Corrupted Noisy Versions
19
Variations ofHebbian Learning
20
Variations ofHebbian Learning
21
Solved Problem P7.6
22
Solved Problem P7.6
Wp b 0
23
Solved Problem P7.7
, where 1 is a vector of ones.
24
Binary Associative Network