Compressed Sensing for Networked Information Processing

1
Compressed Sensing for Networked Information
Processing

• Reza Malek-Madani, 311/ Computational Analysis
• Don Wagner, 311/ Resource Optimization
• Tristan Nguyen, 311/ Computational Analysis
• 28 November 2007

2
Ubiquitous Compressibility

• DoD acquires and uses huge amount of data
• In many scenarios, most of the data in a signal
  can be discarded with almost no perceptual loss
• E.g., lossy compression formats for sounds and
  images
• Key Questions
• Why acquire all the data when most will be
  discarded?
• Can we directly measure the relevant information?
• Challenge Develop mathematical and
  computational techniques that allow us to
  directly acquire relevant information from
  signals and images in compressed form.
• Key Words Adaptivity, Parallelization,
  Stability, Nonlinearity, Noise

3
What is Compressed Sensing?

• Underlying Assumption Most signals are
  compressible in some representation (i.e., most
  coefficients are small relative to some basis)
• Compressed Sensing
• Measure the signal via a random projection to
  yield a compact representation
• Reconstruct the signal from its compact
  representation

4
Nyquist vs. Compressed Sensing

• Nyquist rate samples of wideband signal (sum of
  20 wavelets)
• N 1024 samples/second
• Reconstruction from compressed sensing
• M 150 random measurements/second

MSE lt 2 of signal energy
5
Nyquist vs. Compressed Sensing

• Nyquist rate samples of image
• N 65536 pixels
• Reconstruction from compressed sensing
• M 20000 projections

MSE lt 3 of signal energy
6
Compressed Sensing

• Forward Problem Random projection is the key
  idea
• Inverse Problem Reconstruct from this
  is an ill-posed problem

Candes-Romberg-Tao, Donoho, 2004
7
CS Signal Recovery

• Reconstruction find given
• Classical L2 approach
• L2 algorithm is fast, but unfortunately it is
  wrong

8
CS Signal Recovery

• Reconstruction given(ill-posed inverse
  problem) find
• L2 fast, wrong
• L0 correct, slow
• L1 correct, mild oversampling Candes et al,
  Donoho

Linear-programming problem
9
Theoretical Result (Donoho, 2004)

• Theorem There is a function g, from the
  interval (0, 1 to itself, with the following
  characteristics
• Fix e gt 0.
• If K/M gt g(M/N)(1 e) then, with overwhelming
  probability for large N,
• .
• If K/M lt g(M/N)(1 e ), then does not
  equal .

10
Applications

• Image Understanding (feature detection)
• Communication
• Underwater Communication (RF)
• Wireless Communication
• Channel Parameter Identification (cognitive
  radio, radar)
• Distributed Sensing (fusion of partial
  information)
• .

11
Mathematical Challenges

• Randomness versus determinism
• Can random sensing matrices be replaced by
  deterministic ones?
• What is the impact on the theoretical
  development?
• Can rigorous bounds be developed for the
  equivalents of K, M and N?
• Faster optimization algorithms
• Reconstruction via L1 minimization is relatively
  slow
• Other algorithmic ideas need to be developed
• First-order vs. second-order methods?
• Combinatorial vs. linear vs. nonlinear methods?
• Need to create baselines for comparing algorithms
  in terms of reconstruction speed and accuracy
• Multiple sensors/multiple targets
• Can the underlying theory be extended to handle
  distributed, networked sensors and multiple
  targets?
• Develop the mathematical tools needed to take
  advantage of the statistical correlations among
  signals to perform multi-signal reconstruction

12
Mathematical Challenges

• How many measurements are needed?
• One can get by with much fewer measurements, but
  at the expense of having to solve a tougher
  (i.e., non-convex) optimization problem. What is
  the tradeoff?
• Important extensions needed
• No development to date of distributed
  reconstruction algorithms
• Very important for distributed sensor networks
• Does the theoretical developments to date
  adequately address the issue of noise?
• Random vs. pseudo-random vs. deterministic
  sensing matrices

13
Tangential Issues

• Extension of compressed-sensing optimization to
  affine rank minimization
• Potentially very important in data mining
• Numerical partial differential equation solvers
• Can de-aliasing techniques benefit from the
  compressed-sensing approach?

14
Budget

• Current seed investment 250K/year
• Proposed First Year 1.2M/year
• 40 for analytical/theoretical development
• 40 for algorithmic/computational development
• 20 for application/sensor development
• Outyear growth towards applications

15
Summary

• Compressed Sensing is an important emerging area
• Cuts across of sciences and engineering
• Pioneering foundations are in place
• ONR is well positioned to be a leader