Approaches to Superresolution

1
Approaches to Super-resolution

• Speaker Chia-Hung Lee

2
Outline

• Introduction
• Problem Setup
• Time Domain Approach
• Frequency Domain Approach
• Time-Frequency Domain Approach
• Simulation
• Conclusion

3
Introduction

• Sampling
• Uniform Sampling
• Non-uniform Sampling

4
Multichannel Sampling
5
Resolution
6
Resolution
7
Resolution
8
Resolution

• Ideally, we have unlimited details

9
Resolution

• Ideally, we have unlimited details

10
Resolution

• Ideally, we have unlimited details

11
Resolution

• In practice, resolution is limited

12
Resolution

• In practice, resolution is limited

13
Resolution

• In practice, resolution is limited

14
Resolution

• Resolution is related to resolving power the
  ability to distinguish details in an image
• Resolution of an image depends on
• Number of pixels on the sensor
• Optics
• Processing in the camera

15
Super-resolution

• Why?
• Time Interleaved Analog to Digital Convertor
  (TI-ADC)
• Digital Camera
• Satellite

16
Super-Resolution

• Capture multiple images with small camera motion

17
Super-Resolution

• Align the images

18
Super-Resolution

• Put all the pixels on a common grid

19
Super-Resolution

• Interpolate a higher resolution image

20
Spectrum Analysis

• Fourier Transform (FT)
• Time-Frequency Analysis
• Short Time Fourier Transform
• Gabor Transform

21
Gabor Transform

• Definition
• Why?

22
Problem Setup

• Registration
• Reconstruction

23
Nyquist Criterion

• Perfect Reconstruction
• Below Nyquist?

24
Registration
25
Time Domain Approach

• Mathematic formulation
• An instance, Fourier series
• Numerical methods

26
Pro Con

• Advantage accurate
• Disadvantage high complexity

27
Frequency Domain Approach

• Property of Fourier transform

28
Relation For FT

• Relation between two channels

29
Aliasing

• When below Nyquist, aliasing occurs
• Solution consider the aliasing-free part only

30
Pro Con

• Advantage fast and simple
• Disadvantage only works for partially-aliasing
  signal

31
Time-Frequency Domain Approach

• Property similar to FT

32
Relation For Gabor transform

• Property of Gabor transform

33
Aliasing

• How to deal with aliasing?
• For those aliasing-free parts, the solution is
  exactly
• For those pure (not mixed with the aliasing-free
  parts) aliased parts, the solution is (f
  is the corresponding frequency before aliasing
  f is the corresponding frequency after aliasing)

34
Histogram

• The error solutions depend on frequency
• The error solutions dispersed
• Histogram the global maximum is expected to be
  our estimated offset
• Enhancement region dispersion

35
Pro Con

• Advantage works for almost all cases, with
  limited sacrifice
• Disadvantage hard to extend to 2-D case

36
Simulation -- A (1/5)

• Combination of sinusoidal signals
• Maximum frequency 48 Hz
• Sampling rate 50 Hz
• Offset is 0.002 sec

37
Simulation -- A (2/5)

• The time domain frequency domain spectrum

38
Simulation -- A (3/5)

• The time-frequency domain spectrum

39
Simulation -- A (4/5)

• Histogram

40
Simulation -- A (5/5)

• Result comparison
• The true offset is 0.002 sec

41
Simulation -- B (1/5)

• Acoustic signals
• Maximum frequency 4000 Hz
• Sampling rate 3675 Hz
• Offset is 0.090703 ms

42
Simulation -- B (2/5)

• The time domain frequency domain spectrum

43
Simulation -- B (3/5)

• The time-frequency domain spectrum

44
Simulation -- B (4/5)

• Histogram

45
Simulation -- B (5/5)

• Result comparison
• The true offset is 0.090703 ms

46
Simulation -- C (1/5)

• Speech signals
• Maximum frequency 2000 Hz
• Sampling rate 2667 Hz
• Offset is 0.125 ms

47
Simulation -- C (2/5)

• The time domain frequency domain spectrum

48
Simulation -- C (3/5)

• The time-frequency domain spectrum

49
Simulation -- C (4/5)

• Histogram

50
Simulation -- C (5/5)

• Result comparison
• The true offset is 0.125 ms

51
Conclusion

• The time-frequency domain approach is more
  powerful than the frequency domain approach
• Our algorithm works when the signal is partially
  aliased in the time-frequency domain
• In practice, our algorithm works for most real
  signals
• Trade-off depends on application

52
Reference

• 1 P. Vandewalle, S. Susstrunk, and M. Vetterli,
  A Frequency Approach to Registration of Aliased
  Images with Application to Super-resolution,
  EURASIP Journal on Applied Signal Processing,
  vol. 2006, p.p. 1-14
• 2 P. Vandewalle, Super-resolution from
  unregistered aliased images, Ph.D. Thesis, 2006.
• 3 M. Barni and F. Perez-Gonzalez, Pushing
  Science into Sgnal Processing, IEEE Signal
  Processing Magazine, vol. 22, no. 4, pp. 119120,
  Jul 2005.
• 4 S. Baker and T. Kanade, Limits on
  super-resolution and how to break them, IEEE
  Transactions on Pattern Analysis and Machine
  Intelligence, vol. 24, no. 9, pp. 11671183,
  Sept. 2002