Title: A Fast-Nonegativity-Constrained Least Squares Algorithm
1A Fast-Nonegativity-Constrained Least Squares
Algorithm
- R. Bro, S. D. Jong
- J. Chemometrics,11,393-401, 1997
By Maryam Khoshkam
2- Introduction
- Algorithm of classical non-negative least square
(nnls) - A numerical example
- Fast nonegative least square (Fnnls)
- Results and discussion
3introduction
Estimation of models subject to non-negativity
constraints is of practical importance in
chemistry.
The time required for estimating true least
squares non-negativity constrained model is
typicaly many times longer than for estimating
unconstrained model.
Approximations procedures
Unconstrained estimation Setting negative value
to zero
What is the problem of force to zero algorithm????
4- There is no guarantee whatsoever for the quality
of the model. - When included in multiway algorithm, it can cause
the algorithm to diverge. (specially, in noisy
data or difficult models)
Non-negativity constrained linear least square
5NNLS
NNLS will be stated using the following
nomenclature
For all m
lsqnonneg command in MATLAB is based on this
algorithm
Lawson, C.L. and R.J. Hanson, Solving
Least-Squares Problems, Prentice-Hall, Chapter
23, p. 161, 1974.
6Some important aspects of the algorithm
- This is an active set algorithm.
Thus the nnls first find the true passive and
active sets, then perform the least square on
corresponding columns of Z.
HOW????
First of all, it suppose that all elements of d,
are in active set.
If the all elements of d are active, estimate
that how the form of initial d.?????
R1,2,.,M
P
7- Initial solution vector d is feasible and set
equal to a M1 zero vector
- Then it removes the passive sets elements of d,
one by one, and STEP by STEP
HOW????
8wZT ( x Z d )
One necessary condition for optimally of a
solution , is that the derivatives with respect
to the parameters of the passive set be zero. WHY
??
How is the f (dm) if m is an active set ????
9Thus at optimal solution, we expect that
When we are being at optimal condition, wm gt0.
wht is the meaning of positive value for wm?
10- positive wm shows that by increasing dm to a more
positive value, the change in residual is
negative (f ', which is the slope is negative). - It means that the residual becomes less, and more
close to zero, when dm goes toward positive.
11Algorithm nnls
- PØ
- R1,2,,M
- d0
- wZT(x-Zd)
loop A
No
Opt. sol. Of d
yes
Loop B Build sp
ds wZT(x-Zd)
yes
No
Loop C
all spgt0
12A simple numerical example
loop A
Max(wn)gt1e-15 R?Ø
Loop B
13Loop B
No
Max(wn)gt1e-15 R?Ø
Opt. sol. Of d
14Graphical representation of nonlinear least
square algorithm
H3A ? H H2A- pka12.6 H2A- ? H
HA2- pka24.0 HA2- ? H A3-
pka36.3
D
Z
DX.Z (ILS)
15x10x1X(,1)
Z(10x4)
D(4x100)
X(10x100)
d(4x1)
Step1
- P
- R1,2,3,4
0.005 x D(,1)
w
16nnls
Force to zero
Variation of w during nonnegative least square
17w
Residual matrix
wZT(x-Zd)
wi in each ?, is the contribution of ith species
in in residual vector in each ?.
w(41)
r(10x1)
Z (10x4)T
18(No Transcript)
19PARAFAC-ALS
Least square
20If the size of X is 10x200x5 in a 3 component
system, the size of Z is 500x3 And so on.
Computation of Z, can be computationally costly
for large arrayes Excessive memory is required
to calculate X(IxJK), X(JxIK) and X(K,IJ)
21And in similar way for estimation of B and C
22Nnls can not used for this simplified version of
PARAFAc, Why??
23In estimating A, and using non-negativity
constraint on it,
A(IxF)
X(IxJK)
Z
SLOW!!!!!!!
24Fast non-negativity least square
1. Accept the cross product (xZ and ZZ) instead
of the row data
wZT ( x Z d ) ? wZTx (ZTZ)d
SP(ZP)TZP-1(ZP)Tx? SP(ZTZ)P-1(ZTx)
2. Set the passive and active variables, before
enter to loop B
d is not the zero vector in this case
25Thanks
26Ex.
dZ\x
Force to zero
nnlsq
RMS20
RMS103
27In unconstrained solution of d, the mth negative
element is active set
3th element of d
In constrained solution of d, the mth zero
element is active set
2th and 3th elements of d
Thus it is not possible to realize the true
active set form unconstraint least square
solution.
What happen if the true active set is known?
28For example, if we know that 2th and 3th elements
of d are true active sets,
Then perform the simple unconstrained least
square with the 1st column of Z. (columns
corresponding to passive sets. selectivity
constraint !!!!
29Loop C
1)
2)
3)
Update R and P
4)
SP(ZP)TZP-1(ZP)Tx
5)
SR0