Daubechies Wavelets

1
Daubechies Wavelets

• Chapter 3.1 3.3
• 2008-12-17
• Woo-Cheol Kim

2
Daubechies WT

• Discovered by Ingrid Daubechies
• Professor, Princeton university
• Department of mathematics
• Pronunciation of Daubechies
• ??? in French
• ???? in German

3
3.1 The Daub4 Wavelets (1)

• Daubechies wavelets
• Many Daubechies transforms
• They are all very similar
• Daub4 wavelet transform
• The simplest Daubechies WT
• Defined in essentially the same way as the Haar
  WT
• Can be extended to multiple levels, like the Haar
  WT
• Difference between the Haar WT and Daub4 WT
• The scaling signals and wavelets

4
3.1 The Daub4 Wavelets (2)

• The scaling signals of the Daub4 WT
• The scaling numbers
• In chapter 5, we describe how these scaling
  numbers were obtained
• Using these scaling numbers, the 1-level Daub4
  scaling signals are defined

5
3.1 The Daub4 Wavelets (3)

• The scaling signals of the Daub4 WT
• The 1-level scaling signals
• Each scaling signal has a support of just 4
  time-units
• Wrap-around

cf) Natural basis
6
3.1 The Daub4 Wavelets (4)

• The second level Daub4 scaling signals
• Repeating the operations
• The natural basis of signals ?1-level Daub4
  scaling signals
• 1-level Daub4 scaling signals ? 2-level Daub4
  scaling signals
• Wrap-around
• Shift 4 time-units
• cf) the 1-level scaling signals 2 time-units

7
3.1 The Daub4 Wavelets (5)

• The second level scaling signals
• Support 10 time-units
• cf) support of the 1-level scaling signals 4
  time-units
• ex)

8
3.1 The Daub4 Wavelets (6)

• Properties of scaling signals
• 1-level scaling signal has energy 1
• Also, k-level scaling signal has energy 1
• 1-level trend value is average of 4
  values of f, multiplied by(proof is needed)
• 2-level trend value is average of 10
  values of f, multiplied by 2 (proof is needed)

9
3.1 The Daub4 Wavelets (7)

• The Daub4 wavelets
• The wavelet numbers
• The 1-level Daub4 wavelets
• Each wavelet signal has a support of 4 time-units

10
3.1 The Daub4 Wavelets (8)

• The Daub4 wavelets
• 1-level
• 2-level
• k-level
• Properties of wavelets
• 1-level wavelets have energy 1
• k-level wavelets have energy 1

11
3.1 The Daub4 Wavelets (9)

• Properties of wavelets (cont.)
• If the signal f is constant over the support of a
  Daub4 wavelet , a fluctuation value
  will be zero
• Ex)

(a1d1) (a1, a2, a3, a4, a5, a6, a7, a8 d1,
d2, d3, d4, d5, d6, d7, d8)
12
3.1 The Daub4 Wavelets (10)

• Properties of wavelets (cont.)
• Property I
• If a signal f is (approximately) linear over the
  support of a k-level Daub4 wavelet , then
  the k-level fluctuation value is
  (approximately) zero
• Ex)

13
3.1 The Daub4 Wavelets (11)

• Why is Property I so important?
• A large proportion of the signal consists of
  valuesthat are approximately linear over the
  support ofone of the Daub4 wavelets
• ex) Figure 3.1(c), (d)
• The signal values are approximately linear
  withinsmall squares

14
Haar WT VS Daub4 WT

• The Daub4 averaged signals A3 through A1 all
  appear to be equally close approximations of the
  original signal

15
3.1.1 Remarks on Small Fluctuation Values

• We showed by means of an example
• it applies to sampled signals when the analog
  signal has a continuous second derivative over
  the support of a Daub4 wavelets
• We assume that the signal f has values satisfying
  
• stands for a quantity that is a
  bounded multiple of
• is the constant step-size
• is generally much smaller than 1 and
  consequently is very tiny indeed
• cf) Haar WT

16
3.2 Conservation and Compaction of Energy

• Daubechies WT
• Conserves the energy of signals
• Redistributes this energy into a more compact
  form

• 2-level Haar transform
• 2-level Daub4 transform
• Cumulative energy profiles of (a)
• Cumulative energy profiles of (b)

The Daub4 transform achieves a more compact
redistribution of the energy of the signal
17
3.2.1 Justification of Conservation of Energy

• The matrix
• The rows of are the 1-level Daub4 scaling
  signals and wavelets
• They satisfy

18
Appendix. Orthogonal Matrix

• Orthogonal matrix
• In matrix theory, a real orthogonal matrix is a
  square matrix Q whose transpose is its inverse
• Orthonormal basis
• An orthonormal basis of an inner product space V,
  is a set of mutually orthogonal vectors of
  magnitude 1.
• If the rows of matrix Q form an orthonormal set
  of vectors, the Q is an orthogonal matrix.

19
3.2.1 Justification of Conservation of Energy

• is an orthogonal matrix
• The rows of form an orthonormal set of
  vectors
• proof)

20
3.2.2 How Wavelet and Scaling Numbers are Found

• We briefly outline how the Daub4 scaling numbers
  and wavelet numbers are determined
• The constraints that determine the Daub4 scaling
  and wavelet numbers
• The wavelet numbers are then determined by the
  equations
• We shall provide a more complete discussion in
  Chapter 5

21
3.3 Other Daubechies Wavelets

• We shall describe others Daubechies wavelets
• The DaubJ transforms for J6,8,,20
• The CoifI transforms for I6,12,18,24,30
• These wavelet transforms are all quite similar to
  the Daub4 transform
• There are also many more wavelet transforms
• Spline wavelet transforms
• Various types of biorthogonal wavelet transforms
• DaubJ transforms
• The most obvious difference between them is the
  length of the supports of their scaling signals
  and wavelets

22
3.3 Other Daubechies Wavelets

• The Daub6 transform
• The scaling numbers
• The wavelet numbers

• Scaling signals

23
3.3 Other Daubechies Wavelets

• The Daub6 scaling signals
• k-level scaling signal
• 1-level scaling signal has energy 1
• Also, k-level scaling signal has energy 1
• 1-level trend value is average of 6
  values of f, multiplied by
• 2-level trend value is average of 16
  values of f, multiplied by 2

24
3.3 Other Daubechies Wavelets

• The Daub6 wavelets
• k-level wavelet
• 1-level wavelets have energy 1
• k-level wavelets have energy 1
• Property II If a signal f is (approximately)
  linear or quadratic over the support of a k-level
  Daub6 wavelet , the k-level Daub6
  fluctuation value is (approximately)
  zero

25
3.3 Other Daubechies Wavelets

• Daub6 vs. Daub4
• Because of Property II, the Daub6 transform will
  often produce smaller size fluctuation values
  than those produced by the Daub4 transform

26
3.3 Other Daubechies Wavelets

• DaubJ transform J8,10,,20
• The scaling numbers satisfy
• The wavelet numbers defined by
• Property III
• If a signal f is (approximately) equal to a
  polynomial of degree less than J/2 over the
  support of a k-level DaubJ wavelet , the
  k-level DaubJ fluctuation value is
  (approximately) zero

27
3.3 Other Daubechies Wavelets

• One advantage of using a DaubJ wavelet with a
  larger value for J, say J20, is an improvement
  in the resulting MRA for smoother signals
• The Daub20 MRA is superior to both of these
  previous multiresolution analyses, especially for
  the lower resolution averaged signals

28
3.3 Other Daubechies Wavelets

• Daub20 is the best?
• We do not mean to suggest, however, that the
  Daub20 wavelets are always the best.
• For example, for signal 1, the Haar wavelets do
  the best job of compression and noise removal

29
3.3.1 Coiflets

• The CoifI wavelets
• These wavelets are designed to maintain an
  extremely close match between the trend values
  and the original signal values
• The Coif6 wavelets
• Scaling numbers
• Wavelet numbers

30
3.3.1 Coiflets

• The Coif6 wavelet
• 1-level scaling numbers

• 1-level wavelet numbers

31
3.3.1 Coiflets

• New property of Coif6
• If a signal consists of sample values of an
  analog signal, then a Coif6 transform produces a
  much closer match between trend subsignals and
  the original signal values than can be obtained
  with any of the DaubJ transforms
• ex)
• Maximum error, 0,0.25)
• 2-level Daub4
• 2-level Coif6
• Another interesting feature of CoifI scaling
  signals and wavelets is that their graphs are
  nearly symmetric

32
Chapter 3. Daubechies Wavelets3.4 Compression
of Audio Signals3. 5 Quantization, Entropy, and
Compression

• Embio Database Lab.
• Dec. 23, 2008
• Yunku, Yeo

33
Contents

• Basic Compression of Audio Signals
• Quantization and Significance Map
• Information Theory and Entropy
• Denoising Audio Signals

34
Compression of Audio Signals

• Recall the basic method of WT compression
• Setting equal to 0 all transform values lt
  Threshold
• Significant, non-zero values do survive
• Decompression
• Reconstruct the thresholded transform using the
  significance map and the significant transform
  values
• Perform inverse WT
• Produce an approximation of the original signal
• Compression works well when
• Very few, high-energy transform values capture
  most of the energy of signal

35
Compression of Audio Signals

• Signal 1
• None of the Daubechies transforms does better job
  compressing signal 1
• Haar and Walsh transforms have been used for many
  years as tools for compressing piecewise constant
  signals like signal 1

36
Compression of Audio Signals

• Signal 2
• 4096 212 points ? use 12-level Coif30 transform
• Use top 125 highest magnitude, T 0.00425
• 32 1 compression(4096/125 ? 32)
• We are ignoring issues such asquantization and
  compressionof the significance map

37
Quantization and Significance Map

• Quantize
• Analog audio signal ? digital audio signal
• Digital audio signal typically consists of
  integer values that specify volume levels.
• Digital audio signal of N length? 256 levels
  8N bits, 65536 levels 16N bits

38
Quantization and Significance Map

• Uniform scalar quantization
• Divides volume levels into a fixed number of
  uniform width subintervals, and rounds each
  volume level into the midpoint of the subinterval
  in which it lies
• 8 bpp (bits per point) or 16 bpp in a quantized
  audio signal

39
Quantization and Significance Map

• Uniform quantization with a dead-zone
• T threshold
• M maximum for all the magnitudes of transform
  values
• (-T, T) insignificant values ? not encoded
• -M, -T and T, M are divided into uniform
  width subintervals
• Round each transform valueinto the midpoint of
  thesubinterval containing it

40
Quantization and Significance Map

• Uniform quantization with a dead-zone (cont.)
• ex) Signal 2
• M 9.4, use T 9.4 / 27 ? 100 significant
  values? can be encoded using 8 bits (1 sign bit)
• As can be seen from Figure 3.8(b), the
  significant values of the transform lie in 0,
  0.25) ? The bits of value 1 in the significance
  map lie only within the first 256 bits
• Decompressed signal is indistinguishable?
  Perceptually lossless compression

41
Quantization, Entropy, and Compression

• Specific signal greasy grí?si, -zi
• Suppose this signal is quantizedusing an 8-bit
  scalar quantization
• Intensity values correspond to intensity level,k
  0 255 or -128 128

42
Quantization, Entropy, and Compression

• Use equal-length bit ? wasteful!!
• Encode the most commonly occurring intensity
  level (ex. 0 level) into shortest-length bit
• This idea is similar to Morse code whose
  procedure is made mathematically precise by
  fundamental results from the field known as
  information theory

43
Information Theory, Entropy

• Basic concept
• Suppose that pk are the relative frequencies of
  occurrence of the intensity levels k 0 255
• pk 0, p0 p1 p2 p255 1
• Lk length of the bit sequence that is used to
  encode the intensity level k
• Based on the Shannon Coding Theorem,

Entropy
44
Information Theory, Entropy

• Basic concept (cont.)
• Lk length of the bit sequence that is used to
  encode the intensity level k
• cf) Entropy ? ??? ???
• - ?????? 0? 1? ????? ???? ???? ??
• - ? ??? M? bit? ?? N?? ??? ???,
• - M N? ?? ???? 0? 1? ??? 5050 ??? ?

Entropy for the probabilities pk
45
Information Theory, Entropy

• Basic concept (cont.)
• a lossless encoding technique cannot achieve an
  average length less than the entropy(called
  entropy encoding)
• Huffman encoding entropy entropy 1
• Arithmetic coding asymptotically close to
  entropy
• Assume that well-chosen encoding entropy 0.5
• For greasy, entropy 5.43
• Impossible to make any set of bit sequences whose
  average length is less than 5.43

46
Information Theory, Entropy

• Basic concept (cont.)
• For greasy, entropy 5.43
• 16,384 points in graph
• Other encoding 16384 ? (5.430.5) ? 97,000
  bits? not a particularly effective compression,
  since it still represents 5.93 bpp versus 8 bpp
  for the original signal
• 14-level Coif30 transform 8-bit dead-zone
  quantization
• The entropy for the histogram in Figure 3.10(d)
  is 4.34
• 3,922 non-zero quantized transform values
• 3,922 ? 4.84 ? 19,000 bits significance map
  (max 16,384)

47
Information Theory, Entropy

• Another possibility of WT compression
• 4th trend entropy 6.19, 793 non-zero
  coefficients
• 793 ? 6.69 ? 5,305 bits
• Fluctuation
• Use only 6 bits
• entropy 3.18
• 2,892 non-zero coefficients
• 2,892 ? 3.68 ? 9,197 bits
• 5,3059,19714,502 lt 19,000

48
Information Theory, Entropy

• Another possibility of WT compression (cont.)
• One further possibility is to use8 bpp for the
  trend and 6 bppfor the four fluctuations
• And, separate entropies arecalculated for the
  trend and foreach of the four fluctuations
• Then, the estimated totalnumber of bits needed
  is 13,107

49
Chapter 3. Daubechies Wavelets3. 6 Denoising
Audio Signals

• Embio Database Lab.
• Dec. 26, 2008
• Yunku, Yeo

50
Contents

• Compression Denoising
• Choosing Threshold Value
• Removing Pop Noise and Background static

51
Compression Denoising

• Compression? denoising? ??? ??? ??
• Compression wavelet transform? few high-energy
  transform value? signal? energy? ????? capture? ?
  ?? ??? ??
• ?? ??? ???? ??? energy ??? ????
• ???? denoising method? ??
• Random noise? ??, signal?? ?? ???? ???
• Threshold ??? value? noise? ??, ??
• ?? ??? ?? wavelet transform? ??? ?? ?? signal?
  energy? ????? noise? ??? ? ??

52
Compression Denoising

• Haar transform? ??? denoising (random noise)

??? Haar transform? ????? ???? signal ??
Denoising ??? high energy? ?? transform
value? ?? ?? ?? ? Denoising ??? ?? signal? ?
?? RMS Error 0.057 ? 0.011
53
Compression Denoising

• Haar transform? ??? denoising (random noise)

???? Haar transform?? ?? ?? signal? energy?
100 ????? ?? transform value? ???
Denoising ???? ?? ?? transform value? ?????
??? ?? signal? ???? ?? RMS Error 0.057 ?
0.035 ? Daubechies transform? ??
54
Compression Denoising

• Daubechies transform? ??? denoising

?? ?? ??? signal? ??? 0, 025)???
transform value???? ?? signal? ????? ??
Daubechies wavelet? property - If a signal f
is (approximately) linear over the support
of a k-level Daub4 wavelet Wmk, then the
k-level fluctuation value f Wmk is
(approximately) zero RMS Error 0.057 ? 0.014
55
Choosing a Threshold Value
56
Choosing a Threshold Value

• Wavelet? ??? denoising? ??
• ?? ???? random noise ??? ????
• Noise signal? ?? prior knowledge ???
• Automatic?? threshold? ?? ? ??

57
Choosing a Threshold Value

• ? Gaussian random noise
• ??? ??? ??? ??
• Gaussian ??(????)? ??? random noise
• Intensity level? frequency? ??? histogram? ???
  ???? ??(bell-shape)? ??
• µ(mean) 0??, d(standard deviation)? ?? ?? ??

58
Choosing a Threshold Value

• ? Gaussian random noise
• Daubechies transform? ????? Gaussian noise? ???
  ???? ???
• Daubechies transform?matrix ?????
  orthogonality(???) ???
• µ, d? ?? ???? ???
• µ 0, d 0.505

59
Choosing a Threshold Value

• ? Gaussian random noise
• ???!!
• Daubechies transform ?
• d? ??? ??? ??????? noise? ???? ??? ??? ? ??
• X? T ??? ?? ?? ?? ? ???? ??
  (Gaussian probability density
  function)

60
Choosing a Threshold Value

• ? Gaussian random noise
• ?, ????? 4.5d ??? threshold? ????99 ???
  Gaussian noise? denoising? ? ??
• ???? d? ??? ??????
• Transform? signal ? noise? ?? ??? ???? ??
• ?????, first level fluctuation?? ??
• ?? signal?? ? first level fluctuation value? ??
  ?? ??
• ?? ??? ?? ??? ??? ??? threshold???? ? ??

61
Choosing a Threshold Value

• Thresholding? ?? ?? ?? transform value????? ??
  signal? ??? ? ????
• ?? ??? few high-energy value? ??? ? compress ????
  ???
• Daubechies transform? smooth? analog signal??
  sampling? ??? ?? ??? ???

62
Removing Pop Noise andBackground Static
63
Removing Random Pop noise

• Pop noise(outlier) Gaussian noise
• 3.14 Whistle static background ?? ?? ??
• (b) second fluctuation ??
• 0.46, 0.51 random noise? ????? ???? ??? ? ??
• ? ????? d? ?? 0.66086? T 4.5d 2.97387 ?
  2.98? Random noise ??

Pop noise!
64
Removing Random Pop noise

• Pop noise? ?? fluctuation ?? ?? ??? random
  noise?? ???? ??
• ?? ????? fluctuation?? outlier?? ??
• Outlier? ???? ??acceptance band ? ??
• Acceptance band ??? ??? 0
• Acceptance bands were obtainedby a visual
  inspection of thetransform

65
Removing Random Pop noise

• ??? 1
• 1, 2, 4 level? outlier? ??
• Transform? ??? ?? 5, 6 ??? level?? outlier? ??
• ??? 2
• Acceptance band? ???? ?? 0?? ??? ????, ???
  fluctuation ?? ???? ??? ?? ?? ???? ? ????

66
Chap. 3.7 8Biorthogonal Wavelets

• Embio Database Lab.
• Dec. 23, 2008
• Jaegyoon, Ahn

67
Contents

• Daub 5/3 System
• Daub 5/3 Inverse
• MRA for the Daub 5/3 System
• Daub 5/3 Transform, Multiple Levels
• Daub 5/3 Integer-to-Integer System
• Daub 9/7 System

68
Daub 5/3 System

• Haar wavelet, Daub4 wavelet orthogonal system
• Daub 5/3 wavelet simplest biorthogonal system
• Not energy preserving
• One set of basis signals is used for
  transformationSecond set of basis signals is
  used for the MRA expansions
• Lossless image compression

69
Daub 5/3 System

• Daub 5/3 transform
• Daub 5/3 scaling signals and wavelets

70
Daub 5/3 System

• Properties of scaling signals
• Trend values at any given level are often close
  matches of an analog signal (
  , and so on)

71
Daub 5/3 System

• Properties of wavelets
• If the signal f is constant over the support of a
  Daub 5/3 wavelet , a fluctuation value
  will be zero
• If a signal f is (approximately) linear over the
  support of a k-level Daub5/3 wavelet , then
  the k-level fluctuation value is
  (approximately) zero

72
Daub 5/3 Inverse

• Inverse of 1-level Daub 5/3 transform

73
MRA for the Daub 5/3 System

• Inverse Daub 5/3 transform ?

74
MRA for the Daub 5/3 System

75
MRA for the Daub 5/3 System

• Example

Haar system
Daub 5/3 system
76
MRA for the Daub 5/3 System

• Example

Daub 5/3 system ? Energy not preserved
Haar system ? Energy preserved
77
Daub 5/3 Transform, Multiple Levels

of Coif6
of Coif6
of Daub 5/3
of Daub 5/3
78
Daub 5/3 Integer-to-Integer System

• Lossless image compression

79
Daub 9/7 System

• Lossy image compression and denoising
• JPEG2000
• Scaling signal and wavelet numbers

80
Daub 9/7 System

• Properties of scaling signals
• Trend values at any given level are often close
  matches of an analog signal (
  , and so on)

81
Daub 9/7 System

• Properties of wavelets
• Signals values are closely approximated by
  either a constant sequence, a linear sequence, a
  quadratic sequence or a cubic sequence
• ?? ??? ??, Daub9/7 system? even symmetric? ???
  image? endpoint? ? approximate? ? ??. (Image?
  ???? ?? endpoint(??? ??? ??? ?)? ??.)

82
Daub 9/7 System

• Much smoother ? very useful in image compression
  and denoising
• Very close to energy preserving, ratio of
  to is about 1.02, only 2 difference

of Daub 5/3
of Daub 5/3
of Daub 9/7
of Daub 9/7