Ch10_pres

1
Widrow-Hoff Learning (LMS Algorithm)
2
ADALINE Network
3
Two-Input ADALINE
4
Mean Square Error
Training Set
Input
Target
Notation
Mean Square Error
5
Error Analysis
The mean square error for the ADALINE Network is
a quadratic function
6
Stationary Point
Hessian Matrix
The correlation matrix R must be at least
positive semidefinite. If there are any zero
eigenvalues, the performance index will either
have a weak minumum or else no stationary point,
otherwise there will be a unique global minimum
x.
If R is positive definite
7
Approximate Steepest Descent
Approximate mean square error (one sample)
Approximate (stochastic) gradient
8
Approximate Gradient Calculation
9
LMS Algorithm
10
Multiple-Neuron Case
Matrix Form
11
Analysis of Convergence
For stability, the eigenvalues of this matrix
must fall inside the unit circle.
12
Conditions for Stability
(where li is an eigenvalue of R)
Therefore the stability condition simplifies to
13
Steady State Response
If the system is stable, then a steady state
condition will be reached.
The solution to this equation is
This is also the strong minimum of the
performance index.
14
Example
Banana
Apple
15
Iteration One
Banana
16
Iteration Two
Apple
17
Iteration Three
18
Adaptive Filtering
Tapped Delay Line
Adaptive Filter
19
Example Noise Cancellation
20
Noise Cancellation Adaptive Filter
21
Correlation Matrix
22
Signals
23
Stationary Point
0
0
24
Performance Index
25
LMS Response
26
Echo Cancellation