??? (Jian-Jiun Ding)

1
??? (Jian-Jiun Ding) National Taiwan
University ??????723?, ??????531?
???? [phone] MajorDigital Signal
Processing Digital Image
Processing
2
Research Fields A. Image Processing (1) Image
Compression (page 4) (2) Edge
and Corner Detection (page 14) (3)
Segmentation (page
17) (4) Pattern Recognition (Face, Character,
Video) (page 21) (5) Optical Image Processing
(page 25) (6) Others
Biomedical Image Processing, Banknote
Reconstruction, Dehaze, Scene
Classification (page
33) B. Time-Frequency Analysis (7)
Time-Frequency Analysis (page 40) (8)
Music Signal Analysis (page
58) (9) Wavelet Transform
(page 62)
??????????????
3
C. Fast Algorithms (10) Integer Transforms
(including Walsh transforms, Number Theory) (page
66) D. Applications of Signal Processing
(11) Bioinformatics
(page 72) (12) 3-D Accelerometer
(page 76) E. Other Topics
(13) ECG Signal Analysis
(page 79) (14) Structure Similarity (15) Others
(Quaternion, Filter Design, ) ??????
(page 83)
4
1. Image Compression
Conventional JPEG method
Separate the original image into many 88 blocks,
then using the DCT to code each blocks.
DCT discrete cosine transform
PS ?? 2008?????????
5
??????? ????? discrete cosine transform (DCT)
??,????????????
DCT
6
JPEG ?????????????? ?????????,??? blocking effect
Compression ratio 53.4333 RMSE 10.9662
7
New Method Edge-Based Segmentation and
Compression
???????????
8

• Segmentation-based image compression

Boundary
Boundary Compression
Image Segmentation
Bit stream
An image
Image Segment Compression
Image Segment
9
Original Image
By JPEG
By Proposed Method
An 100x100 image
Bytes 1295, RMSE 2.39
Bytes 456, RMSE 2.54
10
?? (10000 bytes)
?? JPEG (692 bytes)
?? JPEG (233 bytes)
????? (165 bytes)
11
????? ???? 8?8 ???????,??????????? Triangular
and Trapezoid (??) Block Segmentation
J. J. Ding, Y. W. Huang, P. Y. Lin, S. C. Pei, H.
H. Chen, and Y. H. Wang, "Two-dimensional
orthogonal DCT expansion in trapezoid and
triangular blocks and modified JPEG image
compression," IEEE Trans. Image Processing, vol.
22, issue 9, pp. 3664-3675, Sept. 2013
12
?????? (1) ?????????????(???) (2) ??????????,??
orthogonal transform (???) (3) ???????????????????
??? (???) (4) ?????????,????????? (???) (5)
?????????,????????? (???) (6) ?????????????
(???) (7) ?? buffer size ???,???????????
(???) (8) ??????????????,?????????? (???)
J. J. Ding, P. Y. Lin, J. D. Huang, T. H. Lee,
and H. H. Chen, Morphology-based shape adaptive
compression, Lecture Notes in Computer Science,
vol. 6524, pp. 168-176, Jan. 2011
J. J. Ding, H. H. Chen, and W. Y. Wei, Adaptive
Golomb code for joint geometrically distributed
data and its application in image coding, IEEE
Trans. Circuits Syst. Video Technol., vol. 23,
issue 4, pp. 661-670, Apr. 2013
13
?????????????
14
2. Edge and Corner Detection
Why should we perform edge and corner detection?
--Segmentation --Compression --Efficient for
Processing
15
The most efficient way to trace an object in
video (1) Edges (2) Corners (3) SIFT
Points (4) SURF, FAST, BRISK, ORB, FREAK..
Other Feature Points
?? edge detection ?? ???????? (Ex Cannys
algorithm) ? corner detection ???,???? noise ??
16
? Corner Detection
by Harris algorithm
by proposed algorithm
17
3. Segmentation
Important for (i) compression
(ii) biomedical engineering
(iii) pattern recognition, object identification
18
Conventional method 97.87 sec
New method 1.02 sec
19
(No Transcript)
20
(No Transcript)
21
4. Pattern Recognition
including
face recognition character recognition
???? security,
identification, computer
vision
22
???? (1) ??????? (2) ???? (?????)
??????,???????
????
A B 1
23
????
Class 1
Class 2
Class 3
24
?????? matched filter
?????????.
scaling shadow rotation partially distortion
???????? Feature Extraction Machine Learning
???????
25
5. Optical Image Processing
??????,?????????? Qualcomm ??
Depth recovery
?????????????,???????? ?????????????
26
Model for Blurred Images
im, n the original image
bm, n blurred image
km, n some point spread function
convolution
sm, n noise
A blurred image may cause from (1) defocus
(??????) (2) hand-shaking (? Qualcomm ??)
27
Simplest way
What is the problem?
Alternative ways (1) Wiener filter
(2) Richardson-Lucy Methods (3) Fourier Optics
(4) Norm-Prior Based Methods (Levin,
Krishnan) (5) Others
28
Reconstructed Image
Blurred Image
Blurred Image
Reconstructed Image
W. D. Chang, J. J. Ding, Y. Chen, C. W. Chang,
and C. C. Chang, Edge-membership based blurred
image reconstruction algorithm, APSIPA Annual
Summit and Conference, Hollywood, USA, Dec. 2012
29
Blurred Image
30
Reconstructed Image
Y. Chen, J. J. Ding, W. S. Lai, Y. J. Chen, C. W.
Chang, and C. C. Chang, High quality image
deblurring scheme using the pyramid
hyper-Laplacian L2 norm priors algorithm,
Advances in Multimedia Information Processing,
Lecture Notes in Computer Science, vol. 8294, pp.
134-145, Dec. 2013
31
??????????????

• Blurred
• Deblurred

32
lens, (focal length f)
free space, (length z1)
free space, (length z2)

f z1 z2 ? Fourier Transform
f ? z1, z2 but z1 z2 ? Fractional Fourier
Transform f ? z1 ? z2 ? Fractional Fourier
Transform multiplied by a chirp
33
6. Other Applications of Image Processing
(1) Biomedical Image Processing
(????????????) (2) Banknote Reconstruction
(?????????) (3) Image Dehaze (??????,???
Qualcomm ??) (4) Scene Classification (????
Qualcomm ??)
34
???????????
??????????
35
??????????????????
36
?????????????????
J. J. Ding, Y. H. Wang, L. L. Hu, W. L. Chao, and
Y. W. Shau, Muscle injury determination by image
segmentation, VCIP, accepted, Tainan, Nov. 2011
37
???????? (Brain MRI Image)
(a) Brain MRI Image
(b) White Matter (??)
(c) Gray Matter (??)
(d) ?????,???
http//mouldy.bic.mni.mcgill.ca/brainweb/
38
Banknote Reconstruction
39
Dehaze
??????????????????,????????????
Scene Classification
Mountain
River
Outdoors
Ocean
Images
Street
Indoors
40
7. Time-Frequency Analysis
http//djj.ee.ntu.edu.tw/TFW.htm Fourier
transform (FT)
Time-Domain ? Frequency Domain Some
things make the FT not practical (1) Only the
case where t0 ? t ? t1 is interested. (2) Not
all the signals are suitable for analyzing in the
frequency domain. It is hard to analyze the
signal whose instantaneous frequency varies with
time.
41
Example x(t) cos(? t) when t lt 10, x(t)
cos(3? t) when 10 ? t lt 20, x(t) cos(2? t)
when t ? 20 (FM signal)
42
Using Time-Frequency analysis
x(t) cos(? t) when t lt 10,
x(t) cos(3? t) when 10 ? t lt 20, x(t)
cos(2? t) when t ? 20 (FM
signal) Leftusing Gray level to
represent the amplitude of X(t, f) Rightslicing
along t 15
f -axis
t -axis
t -axis
43
Several Time-Frequency Distribution
Short-Time Fourier Transform (STFT) with
Rectangular Mask
Gabor Transform
avoid cross-term less clarity
Wigner Distribution Function
with cross-term high clarity
Gabor-Wigner Transform (Proposed)
avoid cross-term high clarity
44
Cohens Class Distribution
where
S Transform
Hilbert-Huang Transform
45
?????????????????
?? ???? Doppler effect seismic waves Optics radar
system, rectangular function,
In fact, in addition to sinusoid-like functions,
the instantaneous frequencies of other functions
will inevitably vary with time.
46
Applications of Time-Frequency Analysis
(1) Finding Instantaneous Frequency (2) Signal
Decomposition (3) Filter Design (4) Sampling
Theory (5) Modulation and Multiplexing
(6) Electromagnetic Wave Propagation (7)
Optics (8) Radar System Analysis (9) Random
Process Analysis (10) Music Signal Analysis
(11) Biomedical Engineering
(12) Acoustics (13) Spread Spectrum Analysis
(14) System Modeling (15) Image Processing (16)
Economic Data Analysis (17) Signal
Representation (18) Data Compression (19)
Seismology (20) Geology (21) Astronomy, Space
Technology (22) Oceanography
47
Conventional Sampling Theory
Nyquist Criterion
New Sampling Theory
(1) ?t can vary with time (2) Number of sampling
points Area of time frequency distribution
48
Modulation and Multiplexing
spectrum of signal 1
B1
-B1
spectrum of signal 2
not overlapped
B2
-B2
49
Filter Design
(1) Adaptive cutoff criterion
(2) With the fractional Fourier transform
50

• Fractional Fourier Transform
• Performing the Fourier transform a times (a can
  be non-integer)
• ? Fourier Transform (FT)
• 
  generalization
• ? Fractional Fourier Transform (FRFT)
  
• 
  
  , ? ?a/2
• When ? 0.5?, the FRFT becomes the FT.
  

When ? 0.1? ? performing the FT 0.2 times
When ? 0.25? ? performing the FT 0.5 times
When ? ?/6 ? performing the FT 1/3 times
51
? Physical Meaning Transform a Signal into the
Fractional domain, which is the intermediate of
the time domain and the frequency domain.
52
????????,????????? f-axis ????
f-axis
stop band
f0
cutoff line
pass band
t-axis
?? fractional Fourier transform ???????,?????????
f-axis
?
stop band
cutoff line ? f-axis ?????????? ?
u0
pass band
cutoff line
53
? Gabor Transform for signal 0.3expj0.06(t?1)3
? j7t

fractional axis
t-axis
Advantage ? Easy to estimate
the character of a signal in the fractional
domain ? Proposed an
efficient way to find the optimal parameter ?
54
Improvement of Time-Frequency Analysis
(1) Computation Time (2) Tradeoff of the cross
term problem and clarification
??????????,??????????? (???????)

55
??????????
56
???????????????
astronomy
satellite
over 700 km
communication signal
speech, music, voice
vocal signal
over 1000m
oceanography
geology
ocean crust
57
??????????????,????,????????????
58
8. Music Signal Analysis
?? ?????? (Query by Humming)
(?????????) ?????? (??
MP3 ??????????? 1/5)
59
Using the time-frequency analysis
???http//djj.ee.ntu.edu.tw/Chord.wav
La Fa Re
So Mi Do
La Mi Do
60
???http//djj.ee.ntu.edu.tw/air.mp3
time-frequency analysis
61
?????20??????,?????? 100
????????????
62
9. Wavelet Transform
???????????????????,????? Fourier transform (?
2-D ???,???????)
gn
x1,Ln
? 2
??????
xn
? 2
hn
x1,Hn
??????
Example gn 1, 1, hn 1,
-1 or
63
2-D ???
gm
? 2
x1,Lm, n
L-points
m ??, n ??
along m
gn
? 2
v1,Lm, n
along n
hm
? 2
x1,H1m, n
M N
m ??, n ??
xm, n
along m
L-points
x1,H2m, n
gm
? 2
hn
? 2
v1,Hm, n
m ??, n ??
along m
along n
? 2
x1,H3m, n
hm
along m
m ??, n ??
64
The result of the wavelet transform for a 2-D
image
lowpass for x highpass for y
lowpass for x lowpass for y
highpass for x lowpass for y
highpass for x highpass for y
65
Applications for Wavelets
-- JPEG 2000 (image compression) -- filter
design -- edge and corner detection -- pattern
recognition -- biomedical engineering
66
10. Integer Transform
DFT, DCT ???,??,???????..
Discrete Fourier transform (DFT)
Discrete cosine transform (DCT)
RGB to YCbCr Transform
????? entries ???? (?? C/2b) ????
67

• Integer Transform The discrete linear operation
  whose entries are
• summations of 2k.

, ak 0 or 1 or ,
C is an integer.
68
Problem Most of the discrete transforms are
non-integer ones. DFT, DCT,
Karhunen-Loeve transform, RGB to YCbCr color
transform --- To implement them
exactly, we should use floating-point processor
--- To implement them by fixed-point
processor, we should approximate it by an integer
transform. However, after
approximation, the reversibility property is
always lost.
69
Walsh transform
(applied by CDMA)
70
Integer RGB to YCbCr Transform
This is used in JPEG 2000.
71
Integer Transform Conversion Converting all
the non-integer transform into an integer
transform that achieve the following 6 Goals
A, A-1 original non-integer transform pair,
B, B integer transform pair (Goal 1)
Integerization ,
, bk and bk are integers. (Goal
2) Reversibility .
(Goal 3) Bit Constraint The denominator 2k
should not be too large. (Goal 4)
Accuracy B ? A, B ? A-1 (or B ? ?A,
B ? ?-1A-1) (Goal 5) Less Complexity (Goal 6)
Easy to Design
72

11. Bioinformatics
? There are four types of nucleotide in a DNA
sequence adenine (A), guanine (G), thymine
(T), cytosine (C) ? Unitary Mapping bx? 1
if x? A, bx? ?1 if x?
T, bx? j if x? G,
bx? ?j if x? C. y AACTGAA,
? by 1, 1, ?j, ?1, j, 1, 1.
73
? Discrete Correlation Algorithm for DNA Sequence
Comparison For two DNA sequences x and y, if

where Then there are sn nucleotides of xn?
that satisfies xn? y?. ? Example x
GTAGCTGAACTGAAC, y AACTGAA,


. x
GTAGCTGAACTGAAC, y (shifted 7
entries rightward) AACTGAA.
74
? Example x GTAGCTGAACTGAAC, y
AACTGAA, sn

. Checking x
GTAGCTGAACTGAAC, y
AACTGAA. (no entry
match) x
GTAGCTGAACTGAAC, y (shifted 2
entries rightward) AACTGAA. (6
entries match) x
GTAGCTGAACTGAAC, y (shifted 7
entries rightward) AACTGAA. (7
entries match)
75
? Advantage of the Discrete Correlation
Algorithm ---The complexity of the
conventional sequence alignments is O(N2)
---For the discrete correlation algorithm, the
complexity is reduced to O(N log2N) or O(N
log2N b2) b the length of the matched
subsequences Experiment Local
alignment for two 3000-entry DNA sequences
Using conventional dynamic programming
Computation time 87 sec.
Using the proposed
discrete correlation algorithm
Computation time
4.13 sec.
76
12. 3-D Accelerometer (?????)
????(?????????) ?????????
z-axis
y-axis
x-axis
???? x, y, z ??????????,????????
77
z-axis
y 0 z -9.8
y-axis
tilted by ?
z-axis
y -9.8sin ? z -9.8cos ?
y-axis
78
??????? ???? (???) ?? (??,???) ????,?
Parkinson ???? (????????,?????????)
79
13. ECG (???) Signal Analysis
?????????
?????
R
R
T
T
P
P
Q
Q
S
S
80
(a) The Original Signal (The First ECG Signal
in 9.bmp)
(b) Find the Baseline
(c) Subtracted by the Baseline
81
??????????
82
Q1 Telehealth (????)
Can we perform health examination by the ibon
machine in 7-11 or at home?
Q2 Wrist-type Photoplethysmographic (PPG) Signal
Analysis (??????)
83
????????
(1) ???,?? meeting ??,?????????????
??
(a) ????????????,????????????? meeting,
???????????????? meeting
(b) ????????,?????????????????,??? meeting
??
(c) ??????????? ????,??? meeting ?????
(d) ????????,????????????
(2) ????????,????????? CVGIP Take it
easy,????????,??????????,????? ??
(3) ??????,?????????? ??,??????????????????
84
(4) ?????????????,?????????????????
?,??????????? ?????????,???????????????,??????
(????),??????????????
(5) ????????????????
(6) ?? meeting ??????,?????
????????????????,???????
(7) ???? meeting ?????????,???? meeting
85
(8) ????,????????????(??????????,???
??????????????????????) ???,??????????
???,??????????????,????????????,
??????????
(9) ?????????????,?????????
(10) ??????,??? meeting ???? 2015?8???
86
??????,??????,???? ,???????????
?????????????,??????, ??????????????????????
87
???????????? ??????????
????????????,?????????? ????????
????????????????????