Orthogonal Transforms

1
Orthogonal Transforms

• Fourier

2
Review

• Introduce the concepts of base functions
• For Reed-Muller, FPRM
• For Walsh
• Linearly independent matrix
• Non-Singular matrix
• Examples
• Butterflies, Kronecker Products, Matrices
• Using matrices to calculate the vector of
  spectral coefficients from the data vector

Our goal is to discuss the best approximation of
a function using orthogonal functions
3
Orthogonal Functions
4
Orthogonal Functions
5
Note that these are arbitrary functions, we do
not assume sinusoids
6
Illustrate it for Walsh and RM
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
Mean Square Error
11
Mean Square Error
12
(No Transcript)
13
(No Transcript)
14
Important result
15

• We want to minimize this kinds of errors.
• Other error measures are also used.

16
Unitary Transforms
17
Unitary Transforms

• Unitary Transformation for 1-Dim. Sequence
• Series representation of
• Basis vectors
• Energy conservation

Here is the proof
18

• Unitary Transformation for 2-Dim. Sequence
• Definition
• Basis images
• Orthonormality and completeness properties
• Orthonormality
• Completeness

19

• Unitary Transformation for 2-Dim. Sequence
• Separable Unitary Transforms
• separable transform reduces the number of
  multiplications and additions from to
• Energy conservation

20
Properties of Unitary Transform
transform
Covariance matrix
21
Example of arbitrary basis functions being
rectangular waves
22
This determining first function determines next
functions
23
1
0
24
(No Transcript)
25
Small error with just 3 coefficients
26
This slide shows four base functions multiplied
by their respective coefficients
27
This slide shows that using only four base
functions the approximation is quite good
End of example
28
Orthogonality and separability
29
Orthogonal and separable Image Transforms
30
Extending general transforms to 2-dimensions
31
Forward transform
inverse transform
separable
32
Fourier Transforms in new notations

• We emphasize generality
• Matrices

33
Fourier Transform
separable
34
Extension of Fourier Transform to two dimensions
35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
Discrete Fourier Transform (DFT)
New notation
39
(No Transcript)
40
Fast Algorithms for Fourier Transform
Task for students Draw the butterfly for these
matrices, similarly as we have done it for Walsh
and Reed-Muller Transforms
2
Pay attention to regularity of kernels and order
of columns corresponding to factorized matrices
41
Fast Factorization Algorithms are general and
there is many of them
42

• 1-dim. DFT (cont.)
• Calculation of DFT Fast Fourier Transform
  Algorithm (FFT)
• Decimation-in-time algorithm

Derivation of decimation in time
43
Decimation in Time versus Decismation in Frequency
44

• 1-dim. DFT (cont.)
• FFT (cont.)
• Decimation-in-time algorithm (cont.)

Butterfly for Derivation of decimation in time
Please note recursion
45

• 1-dim. DFT (cont.)
• FFT (cont.)
• Decimation-in-frequency algorithm (cont.)

Derivation of Decimation-in-frequency algorithm
46
Decimation in frequency butterfly shows recursion

• 1-dim. DFT (cont.)
• FFT (cont.)
• Decimation-in-frequency algorithm (cont.)

47
Conjugate Symmetry of DFT

• For a real sequence, the DFT is conjugate symmetry

48

• Use of Fourier Transforms for fast convolution

49
Calculations for circular matrix
50
By multiplying
51
W ? Cw
In matrix form next slide
52
w ? Cw
53
Here is the formula for linear convolution, we
already discussed for 1D and 2D data, images
54
Linear convolution can be presented in matrix
form as follows
55
As we see, circular convolution can be also
represented in matrix form
56
Important result
57
Inverse DFT of convolution
58

• Thus we derived a fast algorithm for linear
  convolution which we illustrated earlier and
  discussed its importance.
• This result is very fundamental since it allows
  to use DFT with inverse DFT to do all kinds of
  image processing based on convolution, such as
  edge detection, thinning, filtering, etc.

59
2-D DFT
60
2-D DFT
61
Circular convolution works for 2D images
62
Circular convolution works for 2D images So we
can do all kinds of edge-detection, filtering etc
very efficiently

• 2-Dim. DFT (cont.)
• example

63

• 2-Dim. DFT (cont.)
• Properties of 2D DFT
• Separability

64

• 2-Dim. DFT (cont.)
• Properties of 2D DFT (cont.)
• Rotation

65

• 2-Dim. DFT (cont.)
• Properties of 2D DFT
• Circular convolution and DFT
• Correlation

66

• 2-Dim. DFT (cont.)
• Calculation of 2-dim. DFT
• Direct calculation
• Complex multiplications additions
• Using separability
• Complex multiplications additions
• Using 1-dim FFT
• Complex multiplications additions ???

Three ways of calculating 2-D DFT
67
Questions to Students

• You do not have to remember derivations but you
  have to understand the main concepts.
• Much software for all discussed transforms and
  their uses is available on internet and also in
  Matlab, OpenCV, and similar packages.

• How to create an algorithm for edge detection
  based on FFT?
• How to create a thinning algorithm based on DCT?
• How to use DST for convolution show example.
• Low pass filter based on Hadamard.
• Texture recognition based on Walsh