Compressive Sampling (of Analog Signals)

1
Compressive Sampling(of Analog Signals)

• Moshe Mishali Yonina C. Eldar
• Technion Israel Institute of Technology
• http//www.technion.ac.il/moshiko moshiko_at_tx.te
  chnion.ac.il
• http//www.ee.technion.ac.il/people/YoninaEldar
  yonina_at_ee.technion.ac.il

Advanced topics in sampling (Course
049029) Seminar talk November 2008
2
Context - Sampling
Digital world
Analog world
Sampling A2D
Reconstruction D2A
3
Compression
Original 2500 KB100
4
Outline

• Mathematical background
• From discrete to analog
• Uncertainty principles for analog signals
• Discussion

5
References

• M. Mishali and Y. C. Eldar, "Reduce and Boost
  Recovering Arbitrary Sets of Jointly Sparse
  Vectors," IEEE Trans. on Signal Processing, vol.
  56, no. 10, pp. 4692-4702, Oct. 2008.
• M. Mishali and Y. C. Eldar, "Blind Multi-Band
  Signal Reconstruction Compressed Sensing for
  Analog Signals," CCIT Report 639, Sep. 2007, EE
  Dept., Technion.
• Y. C. Eldar, "Compressed Sensing of Analog
  Signals", submitted to IEEE Trans. on Signal
  Processing, June 2008.
• Y. C. Eldar and M. Mishali, "Robust Recovery of
  Signals From a Union of Subspaces", arXiv.org
  0807.4581, submitted to IEEE Trans. Inform.
  Theory, July 2008.
• Y. C. Eldar, "Uncertainty Relations for Analog
  Signals", submitted to IEEE Trans. Inform.
  Theory, Sept. 2008.

6

• Mathematical background
• Basic ideas of compressed sensing
• Single measurement model (SMV)
• Multiple- and Infinite- measurement models (MMV,
  IMV)
• The Continuous to finite block (CTF)

7
Compressed Sensing
AnalogAudioSignal
Nyquist rateSampling
Compression(e.g. MP3)
CompressedSensing
High-rate
Low-rate
8
Concept
Goal Identify the bucket with fake coins.
Weigh a coinfrom each bucket
Compression
Nyquist
Bucket
numbers
1 number
Weigh a linear combinationof coins from all
buckets
Compressed Sensing
Bucket
1 number
9
Mathematical Tools
non-zero entries ? at least measurements
Recovery brute-force, convex optimization,
greedy algorithms, and more
10
CS theory on 2 slides
Compressed sensing (2003/4 and on) Main results
is uniquely determined by
Donoho and Elad, 2003
Maximal cardinality of linearly independent
column subsets
Hard to compute !
11
CS theory on 2 slides
Compressed sensing (2003/4 and on) Main results
is uniquely determined by
Donoho and Elad, 2003
with high probability
is random
Donoho, 2006 and Candès et. al., 2006
Convex and tractable
Donoho, 2006 and Candès et. al., 2006
Greedy algorithms OMP, FOCUSS, etc.
NP-hard
Tropp, Cotter et. al. Chen et. al. and many other
12
Sparsity models
unknowns
measurements
MMV
Joint sparsity
SMV
IMV Infinite Measurement Vectors (countable or
uncountable) with joint sparsity prior How can
be found ?
Infinite many variables
Infinite many constraints
Exploit prior ? Reduce problem dimensions
13
Reduction Framework
Find a frame for
Solve MMV
Theorem
Mishali and Eldar (2008)
Deterministicreduction
IMV
MMV
Infinite structure allows CS for analog signals
14

• From discrete to analog
• Naïve extension
• The basic ingredients of sampling theorem
• Sparse multiband model
• Rate requirements
• Multicoset sampling and unique representation
• Practical recovery with the CTF block
• Sparse union of shift-invariant model
• Design of sampling operator
• Reconstruction algorithm

15
Naïve Extension to Analog Domain
Standard CS Discrete Framework
Analog Domain
Sparsity prior
what is a sparse analog signal ?
Generalized sampling
Continuoussignal
Infinite sequence
Operator
Finite dimensional elements
Stability
Randomness ? Infinitely many
Random is stable w.h.p
Need structure for efficient implementation
Reconstruction
Finite program, well-studied
Undefined program over a continuous signal
16
Naïve Extension to Analog Domain
Standard CS Discrete Framework
Analog Domain

• Questions
• What is the definition of analog sparsity ?
• How to select a sampling operator ?
• Can we introduce stucture in sampling and still
  preserve stability ?
• How to solve infinite dimensional recovery
  problems ?

Sparsity prior
what is a sparse analog signal ?
Generalized sampling
Continuoussignal
Infinite sequence
Operator
Finite dimensional elements
Stability
Randomness ? Infinitely many
Random is stable w.h.p
Need structure for efficient implementation
Reconstruction
Finite program, well-studied
Undefined program over a continuous signal
17
A step backward
Every bandlimited signal ( Hertz) can be
perfectly reconstructed from uniform sampling
if the sampling rate is greater than
18
A step backward
Every bandlimited signal ( Hertz) can be
perfectly reconstructed from uniform sampling
if the sampling rate is greater than
Fundamental ingredients of a sampling theorm

• A signal model
• A minimal rate requirement
• Explicit sampling and reconstruction stages

19

• Discrete Compressed Sensing
• Analog Compressive Sampling

20
Analog Compressed Sensing
What is the definition of analog sparsity ?

• A signal with a multiband structure in some basis

1. Each band has an uncountable number of non-zero
   elements

1. Band locations lie on an infinite grid

1. Band locations are unknown in advance

(Mishali and Eldar 2007)
(Eldar 2008)
21
Multi-Band Sensing Goals
bands
Sampling
Reconstruction
Analog
Infinite
Analog
Goal Perfect reconstruction
Constraints

1. Minimal sampling rate
2. Fully blind system

What is the minimal rate ? What is the sensing
mechanism ?
How to reconstruct from infinite sequences ?
22
Rate Requirement
Theorem (non-blind recovery)
Landau (1967)
Average sampling rate

• Subspace scenarios
• Minimal-rate sampling and reconstruction (NB)
  with known band locations (Lin and
  Vaidyanathan 98)
• Half blind system (Herley and Wong 99,
  Venkataramani and Bresler 00)

23
Sampling
Multi-Coset Periodic Non-uniform on the Nyquist
grid
In each block of samples, only are kept,
as described by
2
Analog signal
Point-wise samples
0
0
3
3
2
0
3
2
Bresler et. al. (96,98,00,01)
24
The Sampler
DTFT of sampling sequences
Constant
Problems

1. Undetermined system non unique solution
2. Continuous set of linear systems

is jointly sparse and unique under
appropriate parameter selection (

)
25
Paradigm
Solve finiteproblem
Reconstruct
0
S non-zero rows
1
2
3
4
5
6
26
Continuous to Finite
CTF block
MMV

• span a finite space
• Any basis preserves the sparsity

Continuous
Finite
27
Algorithm
Continuous-to-finite block Compressed sensing
for analog signals

• Perfect reconstruction at minimal rate
• Blind system band locations are unkown
• Can be applied to CS of general analog signals
• Works with other sampling techniques

28
Blind reconstruction flow
Multi-coset with Universal
SBR4
Yes
CTF
No
SBR2
No
CTF
Bi-section
Yes
Uniform at
Ideal low-pass filter
Spectrum-blind Sampling
Spectrum-blind Reconstruction
29
Final reconstruction (non-blind)

Bresler et. al. (96,00)
30
Framework Analog Compressed Sensing
Sampling signals from a union of shift-invariant
spaces (SI)
Subspace
generators
31
Framework Analog Compressed Sensing
What happen if only KltltN sequences are not zero ?
Not a subspace !
Only k sequences are non-zero
32
Framework Analog Compressed Sensing
Step 1 Compress the sampling sequences
Step 2 Push all operators to analog domain
CTF
System A
High sampling rate m/TPost-compression
Only k sequences are non-zero
33
Framework Analog Compressed Sensing
Low sampling rate p/TPre-compression
System B
CTF
Theorem
Eldar (2008)
34

• Does it work ?

35
Simulations
Sampling rate
Sampling rate
Brute-Force
M-OMP
36
Simulations (2)
0 Recovery
100 Recovery
0 Recovery
100 Recovery
Noise-free
Sampling rate
Sampling rate
SBR4
SBR2
Empirical recovery rate
37
Simulations (3)
Signal
Reconstruction filter
Amplitude
Amplitude
Output
Time (nSecs)
Time (nSecs)
38

• Break
• (10 min. please)

39

• Uncertainty principles
• Coherence and the discrete uncertainty principle
• Analog coherence and principles
• Achieving the lower coherence bound
• Uncertainty principles and sparse representations

40
The discrete uncertainty principle
Uncertainty principle
41
Discrete coherence
Which bases achieve the lowest coherence ?
42
Discrete coherence
Which signal achieves the uncertainty bound ?
Spikes
Fourier
43
Discrete to analog

• Shift invariant spaces
• Sparse representations

• Questions
• What is the analog uncertainty principle ?
• Which bases has the lowest coherence ?
• Which signal achieves the lower uncertainty
  bound ?

44
Analog uncertainty principle
Theorem
Eldar (2008)
45
Bases with minimal coherence
In the DFT domain
Fourier
Spikes
What are the analog counterparts ?

• Constant magnitude
• Modulation

• Single component
• Shifts

46
Bases with minimal coherence
In the frequency domain
47
Tightness
48
Sparse representations

• In discrete setting

49
Sparse representations

• Analog counterparts

Undefined program !
But, can be transformed into an IMV model
50
Discussion

• IMV model as a fundamental tool for treating
  sparse analog signals
• Should quantify the DSP complexity of the CTF
  block
• Compare approach with the analog model
• Building blocks of analog CS framework.

51

• Thank you