Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks

1
Consensus-Based Distributed Least-Mean Square
Algorithm Using Wireless Ad Hoc Networks

• Gonzalo Mateos, Ioannis Schizas and Georgios B.
  Giannakis
• ECE Department, University of Minnesota
• Acknowledgment ARL/CTA grant no.
  DAAD19-01-2-0011

2
Motivation

• Estimation using ad hoc WSNs raises exciting
  challenges
• Communication constraints
• Limited power budget
• Lack of hierarchy / decentralized processing
  Consensus
• Unique features
• Environment is constantly changing (e.g., WSN
  topology)
• Lack of statistical information at sensor-level
• Bottom line algorithms are required to be
• Resource efficient
• Simple and flexible
• Adaptive and robust to changes

Single-hop communications
3
Prior Works

• Single-shot distributed estimation algorithms
• Consensus averaging Xiao-Boyd 05,
  Tsitsiklis-Bertsekas 86, 97
• Incremental strategies Rabbat-Nowak etal 05
• Deterministic and random parameter estimation
  Schizas etal 06
• Consensus-based Kalman tracking using ad hoc WSNs
• MSE optimal filtering and smoothing Schizas etal
  07
• Suboptimal approaches Olfati-Saber 05, Spanos
  etal 05
• Distributed adaptive estimation and filtering
• LMS and RLS learning rules Lopes-Sayed 06 07

4
Problem Statement

• Ad hoc WSN with sensors
• Single-hop communications only. Sensor s
  neighborhood
• Connectivity information captured in
• Zero-mean additive (e.g., Rx, quantization) noise
• Each sensor , at time instant
• Acquires a regressor and scalar
  observation
• Both zero-mean w.l.o.g and spatially uncorrelated
• Least-mean squares (LMS) estimation problem of
  interest

5
Centralized Approaches

• If , jointly stationary Wiener
  solution
• If global (cross-) covariance matrices ,
  available
• Steepest-descent converges avoiding matrix
  inversion
• If (cross-) covariance info. not available or
  time-varying
• Low complexity suggests (C-) LMS adaptation

Goal develop a distributed (D-) LMS algorithm
for ad hoc WSNs
6
A Useful Reformulation

• Introduce the bridge sensor subset
• For all sensors , such that
• For , there must such that
  
• Consider the convex, constrained optimization

7
Algorithm Construction

• Problem of interest
• Two key steps in deriving D-LMS
• Resort to the alternating-direction method of
  multipliers
• Gain desired degree of parallelization
• Apply stochastic approximation ideas
• Cope with unavailability of statistical
  information

8
Derivation of Recursions

• Associated augmented Lagrangian
• Alternating-direction method of Lagrange
  multipliers
• Three-step iterative update process
• Multipliers Dual iteration
• Local estimates Minimize
  w.r.t.
• Bridge variables Minimize
  w.r.t.

9
Multiplier Updates

• Recall the constraints
• Use standard method of multipliers type of update
• Requires from the bridge neighborhood

10
Local Estimate Updates

• Given by the local optimization
• First order optimality condition
• Proposed recursion inspired by Robbins-Monro
  algorithm
• is the
  local prior error
• is a constant step-size
• Requires
• Already acquired bridge variables
• Updated local multipliers

11
Bridge Variable Updates

• Similarly,
• Requires
• from the neighborhood
• from the neighborhood in a startup phase

12
D-LMS Recap and Operation

• In the presence of communication noise, for
• Simple, fully distributed, only single-hop
  exchanges needed

Step 1
Step 2
Step 3
13
Further Insights

• Manipulating the recursions for and
  yields
• Introduce the instantaneous consensus error at
  sensor
• The update of becomes
• Superposition of two learning mechanisms
• Purely local LMS-type of adaptation
• PI consesus loop tracks the consensus
  set-point

14
D-LMS Processor

• Network-wide information enters through the
  set-point
• Expect increased performance with
  Flexibility

15
Mean Analysis

• Independence setting signal assumptions for
• (As1) is a zero-mean white random
  vector
• , with spectral
  radius
• (As2) Observations obey a linear model
• where is a zero-mean white
  noise
• (As3) and are statistically
  independent
• Define and
• Goal derive sufficient conditions under which

16
Dynamics of the Mean

• Lemma Under (As1)-(As3), consider the D-LMS
  algorithm initialized with
  .
• Then for , is given by the
  second-order recursion
• with
  and
• 
  , where
• Equivalent first-order system by state
  concatenation

17
First-Order Stability Result

• Proposition Under (As1)-(As3), the D-LMS
  algorithm whose positive step-sizes
  and relevant parameters are chosen
• such that ,
  achieves consensus in the mean sense i.e.,
• Step-size selection based on local information
  only
• Local regressor statistics
• Bridge neighborhood size

18
Simulations

• node WSN,
• Regressors i.i.d.
• Observations
• D-LMS ,

True time-varying weight
19
Loop Tuning

• Adequately selecting actually does make a
  difference
• Compared figures of merit
• MSE (Learning curve)
• MSD (Normalized estimation error)

20
Concluding Summary

• Developed a distributed LMS algorithm for general
  ad hoc WSNs
• Intuitive sensor-level processing
• Local LMS adaptation
• Tunable PI loop driving local estimate to
  consensus
• Mean analysis under independence assumptions
• step-size selection rules based on local
  information
• Simulations validate mss convergence and tracking
  capabilities
• Ongoing research
• Stability and performance analysis under general
  settings
• Optimality selection of bridge sensors,
• D-RLS. Estimation/Learning performance Vs
  complexity tradeoff