CSCE 641 Computer Graphics: Image Sampling and Reconstruction

1
CSCE 641 Computer GraphicsImage Sampling and
Reconstruction

• Jinxiang Chai

2
Review 1D Fourier Transform

• A function f(x) can be represented as a weighted
  combination of phase-shifted sine waves
• How to compute F(u)?

Inverse Fourier Transform
Fourier Transform
3
Review Box Function
f(x)
x
F(u)
u
If f(x) is bounded, F(u) is unbounded
4
Review Cosine
?
1
-1
If f(x) is even, so is F(u)
5
Review Gaussian
If f(x) is gaussian, F(u) is also guassian.
6
Review Properties

• Linearity
• Time shift
• Derivative
• Integration
• Convolution

7
Review Properties

• Linearity
• Time shift
• Derivative
• Integration
• Convolution

8
Review Properties

• Linearity
• Time shift
• Derivative
• Integration
• Convolution

9
Outline

• 2D Fourier Transform
• Nyquist sampling theory
• Antialiasing
• Gaussian pyramid

10
Extension to 2D

• Fourier Transform
• Inverse Fourier transform

11
Building Block for 2D Transform
Building block Frequency Orientation
Oriented wave fields
12
Building Block for 2D Transform
Building block Frequency Orientation
Oriented wave fields
A function f(x,y) can be represented as a
weighted combination of phase-shifted oriented
wave fields.
Higher frequency
13
Some 2D Transforms
From Lehar
14
Some 2D Transforms
From Lehar
15
Some 2D Transforms
From Lehar
16
Some 2D Transforms
From Lehar
17
Some 2D Transforms
From Lehar
18
Some 2D Transforms
Why we have a DC component?
From Lehar
19
Some 2D Transforms
Why we have a DC component? - the sum of all
pixel values
From Lehar
20
Some 2D Transforms
Why we have a DC component? - the sum of all
pixel values Oriented stripe in spatial domain
an oriented line in spatial domain
From Lehar
21
2D Fourier Transform
Why? - Any relationship between two slopes?
22
2D Fourier Transform
Why? - Any relationship between two slopes?
Linearity
23
2D Fourier Transform
Why? - Any relationship between two slopes?
Linearity
The spectrum is bounded by two slopes.
24
Online Java Applet

• http//www.brainflux.org/java/classes/FFT2DApplet.
  html

25
2D Fourier Transform Pairs
Gaussian
Gaussian
26
2D Image Filtering
Inverse transform
Fourier transform
From Lehar
27
2D Image Filtering
Inverse transform
Fourier transform
Low-pass filter
From Lehar
28
2D Image Filtering
Inverse transform
Fourier transform
Low-pass filter
high-pass filter
From Lehar
29
2D Image Filtering
Inverse transform
Fourier transform
Low-pass filter
high-pass filter
band-pass filter
From Lehar
30
Aliasing

• Why does this happen?

31
Aliasing

• How to reduce it?

32
Sampling Analysis
Sampling
33
Sampling Analysis
Sampling
Reconstruction
34
Sampling Analysis
Sampling
Reconstruction

• What sampling rate (T) is sufficient to
  reconstruct the continuous version of the sampled
  signal?

35
Sampling Theory

• How many samples are required to represent a
  given signal without loss of information?
• What signals can be reconstructed without loss
  for a given sampling rate?

36
Sampling Analysis Spatial Domain
f(x)
X


x
T
2T
-2T
-T
0
x
continuous signal
comb function
?
discrete signal (samples)
37
Sampling Analysis Spatial Domain
f(x)
X


x
T
2T
-2T
-T
0
x
continuous signal
comb function
?
What happens in Frequency domain?
discrete signal (samples)
38
Sampling Analysis Spatial Domain
f(x)
X


x
T
2T
-2T
-T
0
x
continuous signal
comb function
?
What happens in Frequency domain?
discrete signal (samples)
39
Fourier Transform of Dirac Comb
T
40
Review Dirac Delta and its Transform
f(x)
x
F(u)
1
u
Fourier transform and inverse Fourier transform
are qualitatively the same, so knowing one
direction gives you the other
41
Review Fourier Transform Properties

• Linearity
• Time shift
• Derivative
• Integration
• Convolution

42
Fourier Transform of Dirac Comb
T
43
Fourier Transform of Dirac Comb
44
Fourier Transform of Dirac Comb
T
1/T
- Fourier transform of a Dirac comb is a Dirac
comb as well. - Moving the spikes closer together
in the spatial domain moves them farther apart in
the frequency domain!
45
Sampling Analysis Spatial Domain
f(x)
X


x
T
2T
-2T
-T
0
x
continuous signal
comb function
What happens in Frequency domain?
?
discrete signal (samples)
46
Review Properties

• Linearity
• Time shift
• Derivative
• Integration
• Convolution

47
Sampling Analysis Freq. Domain
1/T

-1/T

0
u
48
Sampling Analysis Freq. Domain
1/T

-1/T

0
u
How does the convolution result look like?
49
Sampling Analysis Freq. Domain
1/T

-1/T

0
u
50
Sampling Analysis Freq. Domain
1/T

-1/T

0
u
51
Sampling Analysis Freq. Domain
1/T

-1/T

0
u
G(0)? G(fmax)? G(u)?
52
Sampling Analysis Freq. Domain
1/T

-1/T

0
u
G(0) F(0) G(fmax) F(fmax) G(u) F(u)
53
Sampling Analysis Freq. Domain
How about
1/T

-1/T

0
u
Fs(u)
u
fmax
-fmax
-1/T
1/T
54
Sampling Analysis Freq. Domain
How about
1/T

-1/T

0
u
Fs(u)
u
fmax
-fmax
-1/T
1/T
55
Sampling Analysis Freq. Domain
1/T

-1/T

0
u
Fs(u)
u
fmax
-fmax
-1/T
1/T
56
Sampling Theory

• How many samples are required to represent a
  given signal without loss of information?
• What signals can be reconstructed without loss
  for a given sampling rate?

57
Sampling Analysis Freq. Domain
1/T

-1/T

0
u
How can we reconstruct the original signal?
Fs(u)
u
fmax
-fmax
-1/T
1/T
58
Reconstruction in Freq. Domain
box(u)
u
fmax
-fmax
u
fmax
-fmax
59
Signal Reconstruction in Freq. Domain
Fourier transform
Inverse Fourier transform
60
Signal Reconstruction in Spatial Domain
sinc(x)
fs(x)
x


T
2T
-2T
-T
0
x
61
Aliasing

• Why does this happen?

62
Sampling Analysis
When does aliasing happen?
f(x)
F(u)
u
x
fmax
-fmax
fs(x)
x


T
-T
0
63
Sampling Analysis
When does aliasing happen?
f(x)
F(u)
u
x
fmax
-fmax
fs(x)
x


T
-T
0
64
Sampling Analysis
When does aliasing happen?
f(x)
F(u)
u
x
fmax
-fmax
fs(x)
x


T
-T
0
65
Sampling Analysis
When does aliasing happen?
f(x)
F(u)
u
x
fmax
-fmax
fs(x)
x


T
-T
0
66
Sampling Analysis
Sampling
Reconstruction

• What sampling rate (T) is sufficient to
  reconstruct the continuous version of the sampled
  signal?

67
Sampling Analysis
Sampling
Reconstruction

• What sampling rate (T) is sufficient to
  reconstruct the continuous version of the sampled
  signal?

• Sampling Rate 2 max frequency in the signal
• this is known as the Nyquist Rate

68
Antialiasing
69
Antialiasing

• Increase the sampling rate to above twice the
  highest frequency

70
Antialiasing

• Increase the sampling rate to above twice the
  highest frequency
• Introduce an anti-aliasing filter to reduce fmax

71
Antialiasing

• Increase the sampling rate to above twice the
  highest frequency
• Introduce an anti-aliasing filter to reduce fmax

Fs(u)
u
fmax
-fmax
72
Sinc Filter

• Is this a good filter?

F(u)
u
fmax
-fmax
73
Sinc Filter

• Is this a good filter?

F(u)
Inverse Fourier transform
u
fmax
-fmax
74
Sinc Filter

• Is this a good filter?

F(u)
Inverse Fourier transform
u
fmax
-fmax
Multiplying with a box function in frequency
domain
Convolution with a sinc function in spatial domain
75
Sinc Filter

• Is this a good filter?

F(u)
Inverse Fourier transform
u
fmax
-fmax
Multiplying with a box function in frequency
domain
Convolution with a sinc function in spatial domain
Good removes all frequency components above a
given bandwidth Bad - an infinite support
in spatial domain and hard to do in the spatial
domain - negative weights (not good for
filtering pixels) - fluctuation causes ripples
76
Good Pre-filtering filters

• Finite support in frequency domain
• - cut-off frequency
• Finite support in spatial domain
• - implementation in spatial domain
• Positive weights
• - good for filtering pixels

77
Other Filter Choices
F(u)
sinc(u)

• Mean filter
• Triangular filter
• Gaussian filter

u
Cannot have a filter with finite support in both
spatial and frequency domain (box, mean,
triangular) - box filter finite in freq.
and infinite in spatial domain - mean and
triangular infinite in freq. and finite in
spatial domain Gaussian filter provides a good
tradeoff between two domains
78
Other Filter Choices
F(u)
sinc(u)

• Mean filter
• Triangular filter
• Gaussian filter

u
Cannot have a filter with finite support in both
spatial and frequency domain (box, mean,
triangular) - box filter finite in freq.
and infinite in spatial domain - mean and
triangular infinite in freq. and finite in
spatial domain Gaussian filter provides a good
tradeoff between two domains
79
Gaussian Filters
Cannot have a filter with finite support in both
spatial and frequency domain (box, mean,
triangular) - box filter finite in freq.
and infinite in spatial domain - mean and
triangular infinite in freq. and finite in
spatial domain Gaussian filter provides a good
tradeoff between two domains
80
Gaussian Filters
Cannot have a filter with finite support in both
spatial and frequency domain (box, mean,
triangular) - box filter finite in freq.
and infinite in spatial domain - mean and
triangular infinite in freq. and finite in
spatial domain Gaussian filter provides a good
tradeoff between two domains
81
Gaussian Filter
82
Gaussian Filter
Inverse Fourier transform
83
Gaussian Filter
Inverse Fourier transform
Multiplying with gaussian function in frequency
domain
Convolution with a gaussian function in spatial
domain
84
Gaussian Filter
Inverse Fourier transform
Multiplying with gaussian function in frequency
domain
Convolution with a gaussian function in spatial
domain

• Since the Gaussian function decays rapidly, it
  is reasonable to truncate the filter window
• Easy to implement it in the spatial domain
• Its standard deviation controls the smoothness
  of filtered signal large s in spatial domain
  lower cutoff frequency

85
Review Gaussian
If f(x) is gaussian, F(u) is also guassian.
86
Filtering in Spatial Domain

Filter function
Input image
Filtered image
87
Gaussian Filtering in Spatial Domain
filtering window 5X5
?
Step 1 evaluate the filtering function at each
grid Step 2 Normalize each weight to ensure the
sum of all weights is one
Discrete approximation to Gaussian function with
s 1.0
88
Gaussian Filtering in Spatial Domain
filtering window 5X5
G(0,0)
Step 1 evaluate the filtering function at each
grid Step 2 Normalize each weight to ensure the
sum of all weights is one
Discrete approximation to Gaussian function with
s 1.0
89
Gaussian Filtering in Spatial Domain
filtering window 5X5
?
G(0,0)
Step 1 evaluate the filtering function at each
grid Step 2 Normalize each weight to ensure the
sum of all weights is one
Discrete approximation to Gaussian function with
s 1.0
90
Gaussian Filtering in Spatial Domain
filtering window 5X5
G(-2,-2)
G(0,0)
Step 1 evaluate the filtering function at each
grid Step 2 Normalize each weight to ensure the
sum of all weights is one
Discrete approximation to Gaussian function with
s 1.0
91
Gaussian Filtering in Spatial Domain
Step 1 evaluate the filtering function at each
grid Step 2 Normalize each weight to ensure the
sum of all weights is one
Discrete approximation to Gaussian function with
s 1.0
92
Filtering in Spatial Domain
Discrete approximation to Gaussian function with
s 1.0
93
Filtering in Spatial Domain
Discrete approximation to Gaussian function with
s 1.0
94
Filtering in Spatial Domain
Discrete approximation to Gaussian function with
s 1.0
95
Filtering in Spatial Domain
Discrete approximation to Gaussian function with
s 1.0
Filtered_I45
X
96
Gaussian Pre-filtering
G 1/8
G 1/4
Gaussian 1/2

• Solution filter the image, then subsample
• Filter size should double for each ½ size
  reduction. Why?

97
Gaussian Prefiltering
G 1/4
G 1/8
Gaussian 1/2

• Solution filter the image, then subsample
• Filter size should double for each ½ size
  reduction. Why?

98
Without Prefiltering
1/4 (2x zoom)
1/8 (4x zoom)
1/2
99
Image Pyramids

• Known as a Gaussian Pyramid
• MipMap (Williams, 1983)

100
MipMap
Mip (latin phase) many things in a small
place Used for texture mapping Small-scale
texture used for rendering distant objects
101
Gaussian Pyramid
filter mask

• Repeat
• Filter
• Subsample
• Until minimum resolution reached
• can specify desired number of levels (e.g.,
  3-level pyramid)
• The whole pyramid is only 4/3 the size of the
  original image!

102
Gaussian Pyramid
103
Laplacian Pyramid

• Laplacian Pyramid (subband images)
• - Created from Gaussian pyramid by subtraction
• - Encode fine to course structure in a
  different level

104
Laplacian Pyramid

• What happens in frequency domain?

105
Where Are They Used?

• Texture mapping
• - Mipmap
• Image processing
• - Image composition/blending
• - Image compression
• Computer vision
• - Image matching
• - Motion tracking
• - Optical flow estimation

106
Summary

• 2D Fourier Transform
• Nyquist sampling theory
• - Sampling Rate 2 max frequency in the
  signal
• Antialiasing
• - prefiltering
• Gaussian pyramid
• - Mipmap

107
Next Lecture

• Image Processing
• - Filtering noise
• - Bilateral filter
• - Feature extraction