Chapter 5 Image Restoration

1
Chapter 5Image Restoration

• ???????? ?????
• ???(Chuan-Yu Chang ) ??
• Office ES 709
• TEL [phone] ext. 4337
• E-mail chuanyu_at_yuntech.edu.tw

2
Chapter 5 Image Restoration

• Image Degradation/Restoration Process
• The objective of restoration is to obtain an
  estimate of the original image.
• will be close to f(x,y).

Restoration????????????????,?????????????????
3
Image Degradation/Restoration Process

• The degraded image is given in the spatial domain
  by
• The degraded image is given in the frequency
  domain by

Degradation function
(5.1-1)
noise
(5.1-2)
4
Noise Models Some Important Probability Density
Functions

• The principal sources of noise
• Image acquisition
• transmission
• Gaussian noise (normal noise)
• Rayleigh noise
• Erlang (Gamma) noise

5
Noise Models

• Exponential noise
• Uniform noise
• Impulse (salt and pepper) noise

6
Some important probability density function
???????,????
7
Example 5.1Sample noisy images and their
histograms

• ???????,?????????????

8
Example 5.1 (cont.)Sample noisy images and their
histograms
9
Example 5.1 (cont.) Sample noisy images and
their histograms
10
Periodic Noise

• Periodic Noise
• Arises typically from electrical or
  electromechanical interference during image
  acquisition.
• It can be reduced via frequency domain filtering.
• Estimation of Noise Parameters
• Estimated by inspection of the Fourier spectrum
  of the image.
• Periodic noise tends to produce frequency spikes
  that often can be detected by visual analysis.
• From small patches of reasonably constant gray
  level.
• The heights of histogram are different but the
  shapes are similar.

11
Example
????sinusoidal noise???
???????
???spectrum
12
Fig 5.4(a-c)????????histogram
Histogram??????Fig4(d,e,k)???????????
13
Periodic Noise

• The simplest way to use the data from the image
  strips is for calculating the mean and variance
  of the gray levels.
• The shape of the histogram identifies the closest
  PDF match.

(5.2-15)
(5.2-16)
14
Restoration in the presence of noise only-spatial
filtering

• Degradation present in an image is noise
• The noise terms (h(x,y), N(u,v)) are unknown, so
  subtracting them from g(x,y)or G(u,v)is not a
  realistic option.
• In periodic noise,it is possible to estimate
  N(u,v) from the spectrum of G(u,v).

15

• Mean Filter
• Arithmetic mean filter
• Let Sxy represent the set of coordinates in a
  rectangular subimage windows of size mxn,
  centered at point (x,y).
• The arithmetic mean filtering process computes
  the average value of the corrupted image g(x,y)
  in the area defined by Sxy.
• This operation can be implemented using a
  convolution mask in which all coefficients have
  value 1/mn.
• Noise is reduced as a result of blurring

16
Mean Filter (cont.)

• Geometric mean filter
• Each restored pixel is given by the product of
  the pixels in the subimage window, raised to the
  power 1/mn.
• A geometric mean filter achieves smoothing
  comparable to the arithmetic mean filter, but it
  tends to lose less image detail in the process.

17
Example 5.2Illustration of mean filters
????0,???400?????????????
18
Restoration in the presence of noise only-spatial
filtering

• Harmonic mean filter
• Contra-harmonic mean filter

???salt noise, ??pepper noise???
? Qgt0 ???pepper noise, ? Qlt0 ???salt noise, ? Q0
????? ? Q-1?Harmonic mean
19
???0.1?salt???????
???0.1?pepper???????
Chapter 5 Image Restoration
The positive-order filter did a better job of
cleaning the background. In general, the
arithmetic and geometric mean filters are well
suited for random noise. The contraharmonic
filter is well suited for impulse noise
20
Results of selecting the wrong sign in
contra-harmonic filtering
The disadvantage of contraharmonic filter is that
it must be known whether the noise is dark or
light in order to select the proper sign for Q.
The result of choosing the wrong sign for Q can
be disastrous. ?contra-harmonic
filter??????????????
21
Order-Statistics Filters

• The response of the order-statistics filters is
  based on ordering (ranking) the pixels contained
  in the image area encompassed by the filter.
• Median filter
• Replaces the value of a pixel by the median of
  the gray levels in the neighborhood of that
  pixel.
• Medial filter provide excellent noise-reduction
  capabilities, with considerably less blurring
  than linear smoothing filters of similar size.
• Median filters are particularly effective in the
  presence of both bipolar and unipolar impulse
  noise.

(5.3-7)
22
Order-Statistics Filters (cont.)

• Max filter
• This filter is useful for finding the brightest
  points in an image.
• It reduces pepper noise
• Min filter
• This filter is useful for finding the darkest
  points in an image
• It reduces salt noise.

???pepper noise
(5.3-8)
(5.3-9)
???salt noise
23
Order-Statistics Filters (cont.)

• Midpoint filter
• This filter works best for randomly distributed
  noise, such as Gaussian or uniform noise.
• Alpha-trimmed mean filter
• We delete the d/2 lowest and the d/2 higest
  gray-level values of g(s,t) in the neighborhood
  Sxy.

(5.3-10)
(5.3-11)
???0.5d?????????,?????????? ?d0,??mean
filter ?d(mn-1)/2?,?median filter
24
Example 5.3Illustration of order-statistics
filters
Result of one pass with a median filter of size
3x3, several noise points are still visible.
Image corrupted by salt and pepper noise with
probabilities PaPb0.1
Result of processing (b) with median filter again
Result of processing (c) with median filter again
25
Example 5.3Illustration of order-statistics
filters
Result of filtering with a min filtering
Result of filtering with a max filtering
26
Example 5.3 Illustration of order-statistics
filters
Result of filtering with a arithmetic mean filter
Result of filtering with a geometric mean filter
Result of filtering with a median filter
Result of filtering with a alpha-trimmed mean
filter
27
Adaptive Filter

• Adaptive Filter
• The behavior changes based on statistical
  characteristics of the image inside the filter
  region defined by the m x n rectangular windows
  Sxy.
• The price paid for improved filtering power is an
  increase in filter complexity.
• Adaptive, local noise reduction filter
• The mean gives a measure of average gray level in
  the region.
• The variance gives a measure of average contrast
  in that region.
• The response of the filter at any point (x,y) on
  which the region is centered is to be based on
  four quantities
• g(x,y) the value of the noisy image.
• The variance of the noise corrupting f(x,y) to
  form g(x,y)
• mL, the local mean of the pixels in Sxy.
• The local variance of the pixels in Sxy.

28
Adaptive local noise reduction filter

• The behavior of the filter to be as follows
• If the variance of g(x,y) is zero, the filter
  should return simply the value of g(x,y).
• If the local variance is high relative to the
  variance of g(x,y) , the filter should return a
  value close to g(x,y).
• If the two variances are equal, return the
  arithmetic mean value of the pixels in Sxy.
• An adaptive expression for obtaining estimated
  f(x,y) based on these assumptions may be written
  as

(5.3-12)
29
Example 5.4 Illustration of adaptive, local
noise-reduction filtering
Arithmetic mean 77
Gaussian noise
geometic mean 77
Adaptive filter
30
Adaptive median filter

• Adaptive median filtering can handle impulse
  noise, it seeks to preserve detail while
  smoothing nonimpulse noise.
• The adaptive median filter changes the size of
  Sxy during filter operation, depending on certain
  conditions.
• Consider the following notation
• Zmin minimum gray level value in Sxy.
• Zmax maximum gray level value in Sxy.
• Zmed median of gray levels in Sxy.
• Zxy gray level at coordinates (x,y).
• Smax maximum allowed size of Sxy.

31
Adaptive median filter (cont.)

• The adaptive median filtering algorithm
• Level A A1zmed-zmin A2zmed-zmax if A1gt0
  and A2lt0, goto level B else increase the
  window size if window size ltSmax, repeat
  level A else output zxy
• Level B B1zxy-zmin B2zxy-zmax if B1gt0
  and B2 lt0, output zxy else output zmed

??zmed???impulse noise
32
Adaptive median filter (cont.)

• The objectives of the adaptive median filter
• Remove the slat-and-pepper noise
• Preserve detail while smoothing nonimpulse noise
• Reduce distortion
• The purpose of level A is to determine in the
  median filter output, zmed is an impulse or not.

33
Example 5.5 Illustration of adaptive median
filtering
Corrupted by salt-and pepper noise with
probabilities PaPb0.25
Result of adaptive median filtering with Smax7
Result of filtering with a 7x7 median filter
Preserved sharpness and detail
The noise was effectively removed, the filter
caused significant loss of detail in the image
34
Periodic Noise Reduction by Frequency Domain
Filtering

• Bandreject Filter
• Remove a band of frequencies about the origin of
  the Fourier transform.

(5.4-1)
Ideal Bandreject filter
N order Butterworth filter
(5.4-2)
Gaussian Bandreject filter
(5.4-3)
35
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)
36
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)
Image corrupted by sinusoidal noise
Butterworth bandreject filter of order 4
37
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)

• Bandpass Filters
• A bandpass filter performs the opposite operation
  of a bandreject filter.
• The transfer function Hbp(u,v) of a bandpass
  filter is obtained from a corresponding
  bandreject filter with transfer function Hbr(u,v)
  by

(5.4-4)
38
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)

• Bandpass filtering is quit useful in isolating
  the effect on an image of selected frequency
  bands.

???????????5.16(a)???????

• This image was generated by
• Using Eq(5.4-4) to obtain the bandpass filter.
• Taking the inverse transform of the
  bandpass-filtered transform

39
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)
????????

• Notch Filters
• Rejects frequencies in predefined neighborhoods
  about a center frequency.
• Due to the symmetry of the Fourier transform,
  notch filters must appear in symmetric pairs
  about the origin

(5.4-5)
(5.4-6)
(5.4-7)
40
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)

• order n Butterworth notch filter
• Gaussian notch reject filter
• These three filters become highpass filters if
  u0v00.

(5.4-8)
(5.4-9)
41
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)
Ideal notch
order 2 Butterworth notch filter
Gaussian notch filter
?u0v00,???????,?????????
42
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)

• Notch pass filters
• We can obtain notch pass filters that pass the
  frequencies contained in the notch areas.
• Exactly the opposite function as the notch reject
  filters.
• Notch pass filters become lowpass filters when
  u0v00.

(5.4-10)
43
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)
??????????????? (???????)
Spectrum image
Notch filter
????????
44
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)

• Optimal Notch filtering
• Clearly defined interference patterns are not
  common.
• Images obtained from electro-optical scanner are
  corrupted by coupling and amplification of
  low-level signals in the scanners electronic
  circuitry.
• The resulting images tend to contain significant,
  2D periodic structures superimposed on the scene
  data.

45
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)

• Image of the Martian terrain taken by the Mariner
  6 spacecraft.
• The interference pattern is hard to detect.
• The star-like components were caused by the
  interference, and several pairs of components are
  present.
• The interference components generally are not
  single-frequency bursts. They tend to have broad
  skirts that carry information about the
  interference pattern.

46
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)

• Optimal Notch filtering minimizes local variances
  of the restored estimate image.
• The procedure contains three steps
• Extract the principal frequency components of the
  interference pattern.
• Subtracting a variable, weighted portion of the
  pattern from corrupted image.

47
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)

• The first step is to extract the principal
  frequency component of the interference pattern
• Done by placing a notch pass filter, H(u,v) at
  the location of each spike.
• The Fourier transform of the interference noise
  pattern is given by the expressionwhere G(u,v)
  denotes the Fourier transform of the corrupted
  image.

48
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)

• Formation of H(u,v) requires considerable
  judgment about what is or is not an interference
  spike.
• The notch pass filter generally is constructed
  interactively by observing the spectrum of G(u,v)
  on a display.
• After a particular filter has been selected, the
  corresponding pattern in the spatial domain is
  obtained from the expression

49
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)

• Because the corrupted image is assumed to be
  formed by the addition of the uncorrupted image
  f(x,y) and the interference, if h(x,y) were know
  completely, subtracting the pattern from g(x,y)
  to obtain f(x,y) would be a simple matter.
• This filtering procedure usually yields only an
  approximation of the true pattern.
• The effect of components not present in the
  estimate of h(x,y) can be minimized instead by
  subtracting from g(x,y) a weighted portion of
  h(x,y) to obtain an estimate of f(x,y).
• The function w(x,y) is to be determined, which is
  called as weighting or modulation function.
• The objective of the procedure is to select this
  function so that the result is optimized in some
  meaningful way.

(5.4-13)
50
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)

• To select w(x,y) so that the variance of the
  estimate f(x,y) is minimized over a specified
  neighborhood of every point (x,y).
• Consider a neighborhood of size (2a1) by (2b1)
  about a point (x,y), the local variance can be
  estimated aswhere

(5.4-14)
(5.4-15)
51
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)

• Substituting Eq(5.4-13) into Eq(5.4-14) yield
• Assuming that w(x,y) remains essentially constant
  over the neighborhood gives the approximation
• This assumption also results in the
  expressionin the neighborhood.

(5.4-16)
(5.4-17)
(5.4-18)
52
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)

• With these approximations Eq5.4-160 becomes
• To minimize variance, we solvefor w(x,y)
• The result is

(5.4-19)
(5.4-20)
(5.4-21)
53
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)
?5-20(a)??????
54
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)
???????
N(u,v)??????
55
Periodic Noise Reduction by Frequency Domain
Filtering (cont.)
??????
56
Linear, Position-Invariant Degradations
Additivity If H is a linear operator, the
response to a sum of two inputs is equal to the
sum of the two response
(5.5-1)
(5.5-2)
(5.5-3)
57
Linear, Position-Invariant Degradations

• Homogeneity
• The response to a constant multiple of any input
  is equal to the response to that input multiplied
  by the same constant.

(5.5-4)
58
Linear, Position-Invariant Degradations

• Position (space) invariance
• The response at any point in the image depends
  only on the value of the input at that point, not
  on its position.
• f(x,y)???????????
• ??h(x,y)0,??Eq(5.5-6)??Eq(5.5-1)??
• ??H??????,????????

(5.5-5)
(5.5-6)
(5.5-7)
(5.5-8)
59
Linear, Position-Invariant Degradations (cont.)

• ???f(a,b)?x,y??,????Homogeneity????,H?????(impu
  lse response),h(x,a,y,b)??????(point spread
  function, PSF)
• ?Eq(5.5-10)??Eq(5.5-9)??

(5.5-9)
(5.5-10)
(5.5-11)
60
Linear, Position-Invariant Degradations (cont.)

• ?H?????,?Eq(5.5-5)???Eq(5.5-11)??????convolu
  tion integral(?Eq(4.2-30))
• ?????????, Eq(5.5-11)????
• ?H?????,?Eq(5.5-14)???

(5.5-12)
(5.5-13)
(5.5-14)
(5.5-15)
61
Linear, Position-Invariant Degradations (cont.)

• Summary
• ?????h(x,y)???,??????,??Eq(5.5-15)???
• A linear, spatially invariant degradation system
  with additive noise can be modeled in the
  spatially domain as the convolution of the
  degradation function with an image, followed by
  the addition of noise.

(5.5-16)
(5.5-17)
62
Estimating the Degradation Function

• There are three principal ways to estimate the
  degradation function for use in image
  restoration
• Observation
• Experimentation
• Mathematical modeling

63
Estimating the Degradation Function

• Estimation by image observation
• When a given degraded image without any knowledge
  about the degradation function H.
• To gather information from the image itself.
• Look at a small section of the image containing
  simple structures.
• Look for areas of strong signal content. Gs(u,v)
• Construct an unblurred image as the observed
  subimage. Fs(u,v)
• Assume that the effect of noise is negligible,
  thus the degradation function could be estimated
  by Hs(u,v)Gs(u,v)/Fs(u,v)
• To construct the function H(u,v) by turns out the
  Hs(u,v) to have the shape of Butterworth lowpass
  filter.

64
Estimating the Degradation Function

• Estimation by experimentation
• A linear, space-invariant system is described
  completely by its impulse response.
• A impulse is simulated by a bright dot of light
• ???????????????????H(u,v)

(5.6-2)
65
Estimating the Degradation Function
???
????
66
Estimating the Degradation Function

• Estimation by modeling
• Degradation model based on the physical
  characteristics of atmospheric turbulence

67
Estimating the Degradation Function
68
Remove the degradation of planar motion
(5.6-8)
(5.6-9)
(5.6-10)
(5.6-11)
69
Chapter 5 Image Restoration
????Fourier Transform??(5.6-11)?H(u,v)?,???Fourier
Transform???? ab0.1, T1
70
Inverse Filtering

• Direct inverse filtering
• ??Eq(5.1-2)???????
• ????????,??????????????,??N(u,v)??????????????
• ???????????????,?N(u,v)/H(u,v)?????F(u,v)

(5.7-1)
(5.7-2)
71
Cutoff H(u,v) a radius of 40
Chapter 5 Image Restoration
??G(u,v)/H(u,v)
Cutoff H(u,v) a radius of 85
Cutoff H(u,v) a radius of 70
72
Minimum Mean Square Error (Wiener) Filtering

• Incorporated both the degradation function and
  statistical characteristics of noise into the
  restoration process.
• The objective is to find an estimate f of the
  uncorrupted image f such that the mean square
  error between them is minimized.

(5.8-1)
?????,??? K???
(5.8-2)
(5.8-3)
73
Example 5.12
74
Example 5.13
75
Constrained Least Squares Filtering

• The difficulty of the Wiener filter
• The power spectra of the undegraded image and
  noise must be known
• A constant estimate of the ratio of the power
  spectra is not always a suitable solution.
• Constrained Least Squares Filtering
• Only the mean and variance of the noise are
  needed.

76
Constrained Least Squares Filtering

• ?Eq(5.5-16)????????

(5.9-1)
(5.9-2)
(5.9-3)
(5.9-4)
(5.9-5)
77
Chapter 5 Image Restoration
????r,?????????
78

• ???????g
• ??????g

79

• Step 1??g????
• Step 2??r2
• Step 3???Eq(5.9-8)???,?
  ???g??? ???g??????2,????g???E
  q(5.9-4)

80
Chapter 5 Image Restoration
81
Geometric Transformations

• Spatial Transformations

82
Chapter 5 Image Restoration

• Gray-level Interpolation
• Zero-order interpolation
• Cubic convolution interpolation
• Bilinear interpolation

83
Chapter 5 Image Restoration
84
Chapter 5 Image Restoration