Some Blind Deconvolution Techniques in Image Processing

1
Some Blind Deconvolution Techniques in Image
Processing

• Tony Chan
• Math Dept., UCLA

Joint work with Frederick Park and Andy M. Yip
Astronomical Data Analysis Software
Systems Conference Series 2004 Pasadena, CA,
October 24-27, 2004
2
Outline

• Part I
• Total Variation Blind Deconvolution
• Part II
• Simultaneous TV Image Inpainting and Blind
  Deconvolution
• Part III
• Automatic Parameter Selection for TV Blind
  Deconvolution

Caution Our work not developed specifically for
Astronomical images
3
Blind Deconvolution Problem
?


Observed image
Unknown true image
Unknown point spread function
Unknown noise
Goal Given uobs, recover both uorig and k
4
Typical PSFs
PSFs w/ sharp edges
PSFs w/ smooth transitions
5
Total Variation Regularization

• Deconvolution ill-posed need regularization
• Total variation Regularization

• The characteristic function of D with height h
  (jump)

• TV Length(?D)?h
• TV doesnt penalize jumps
• Co-area Formula

6
TV Blind Deconvolution Model
(C. and Wong (IEEE TIP, 1998))
Objective
Subject to

• ?1 determined by signal-to-noise ratio
• ?2 parameterizes a family of solutions,
  corresponds to different spread of the
  reconstructed PSF

• Alternating Minimization Algorithm
• Globally convergent with H1 regularization.

7
Blind v.s. non-Blind Deconvolution
Clean image

• Observed Image noise-free

non-Blind
True PSF

• An out-of-focus blur is recovered automatically
• Recovered blind deconvolution images almost as
  good as non-blind
• Edges well-recovered in image and PSF

8
Blind v.s. non-Blind Deconvolution High Noise
Clean image
Blind

• Observed Image SNR5 dB

non-Blind
True PSF
?1 2?10?5, ?2 1.5?10?5

• An out-of-focus blur is recovered automatically
• Even in the presence of high noise level,
  recovered images from blind deconvolution are
  almost as good as those recovered with the exact
  PSF

9
Controlling Focal-Length

• Recovered Images are 1-parameter family w.r.t. ?2

Recovered Blurring Functions
(?1 2?10?6)

?2
0
The parameter ?2 controls the focal-length
10
Generalizations to Multi-Channel Images

• Inter-Channel Blur Model
• Color image (Katsaggelos et al, SPIE 1994)

k1 within channel blur
k2 between channel blur
m-channel TV-norm (Color-TV) (C. Blomgren, IEEE
TIP 98)
11
Examples of Multi-Channel Blind Deconvolution
(C. and Wong (SPIE, 1997))
Original image
Out-of-focus blurred
blind non-blind
Gaussian blurred
blind non-blind

• Blind is as good as non-blind
• Gaussian blur is harder to recover
  (zero-crossings in frequency domain)

12
TV Blind Deconvolution Patented!
13
Outline

• Part I
• Total Variation Blind Deconvolution
• Part II
• Simultaneous TV Image Inpainting and Blind
  Deconvolution
• Part III
• Automatic Parameter Selection for TV Blind
  Deconvolution

14
TV Inpainting Model(C. Shen SIAP 2001)

• Scratch Removal

Graffiti Removal
15
Images Degraded by Blurring and Missing Regions

• Blur
• Calibration errors of devices
• Atmospheric turbulence
• Motion of objects/camera

• Missing regions
• Scratches
• Occlusion
• Defects in films/sensors

16
Problems with Inpaint then Deblur

• Inpaint first ? reduce plausible solutions
• Should pick the solution using more information

17
Problems with Deblur then Inpaint
Original
Occluded
Support of PSF
Dirichlet
Neumann
Inpainting

• Different BCs correspond to different image
  intensities in inpaint region.
• Most local BCs do not respect global geometric
  structures

18
The Joint Model

• Do --- the region where the image is observed
• Di --- the region to be inpainted

• A natural combination of TV deblur TV inpaint
• No BCs needed for inpaint regions
• 2 parameters (can incorporate automatic parameter
  selection techniques)

19
Simulation Results (1)

• The vertical strip is completed
• Not completed
• Use higher order inpainting methods
• E.g. Eulers elastica, curvature driven diffusion

20
Simulation Results (2)
Observed
Restored
Original
Inpaint then deblur (many ringings)
Deblur then inpaint (many artifacts)
21
Boundary Conditions for Regular Deblurring
Original image domain and artificial boundary
outside the scene
22
(No Transcript)
23
Outline

• Part I
• Total Variation Blind Deconvolution
• Part II
• Simultaneous TV Image Inpainting and Blind
  Deconvolution
• Part III
• Automatic Parameter Selection for TV Blind
  Deconvolution
• (Ongoing Research)

24
Automatic Blind Deblurring (ongoing research)
observed image
Clean image
SNR 15 dB
Problem Find ?2 automatically to recover best u
k

• Recovered images 1-parameter family wrt ?2
• Consider external info like sharpness to choose
  optimal ?2

25
Motivation for Sharpness Support
u
Support of

• Sharpest image has large gradients
• Preference for gradients with small support

26
Proposed Sharpness Evaluator
u
Support of

• F(u) small gt sharp image with small support
• F(u)0 for piecewise constant images
• F(u) penalizes smeared edges

27
Planets Example
Rel. errors in u (blue) and k (red) v.s. ?2
?10.02 (optimal)
Optimal Restored Image
Auto-focused Image
Proposed Objective v.s. ?2
(minimizer of sharpness func.)
(minimizer of rel. error in u)
The minimum of the sharpness function agrees with
that of the rel. errors of u and k
28
Satellite Example
Rel. errors in u (blue) and k (red) v.s. ?2
?10.3 (optimal)
Optimal Restored Image
Auto-focused Image
Proposed Objective v.s. ?2
(minimizer of sharpness func.)
(minimizer of rel. error in u)
The minimum of the sharpness function agrees with
that of the rel. errors of u and k
29
Potential Applications to Astronomical Imaging

• TV Blind Deconvolution
• TV/Sharp edges useful?
• Auto-focus appropriate objective function?
• How to incorporate a priori domain knowledge?
• TV Blind Deconvolution Inpainting
• Other noise models e.g. salt-and-pepper noise

30
References

1. C. and C. K. Wong, Total Variation Blind
   Deconvolution, IEEE Transactions on Image
   Processing, 7(3)370-375, 1998.
2. C. and C. K. Wong, Multichannel Image
   Deconvolution by Total Variation Regularization,
   Proc. to the SPIE Symposium on Advanced Signal
   Processing Algorithms, Architectures, and
   Implementations, vol. 3162, San Diego, CA, July
   1997, Ed. F. Luk.
3. C. and C. K. Wong, Convergence of the Alternating
   Minimization Algorithm for Blind Deconvolution,
   UCLA Mathematics Department CAM Report 99-19.
4. R. H. Chan, C. and C. K. Wong, Cosine Transform
   Based Preconditioners for Total Variation
   Deblurring, IEEE Trans. Image Proc., 8 (1999),
   pp. 1472-1478
5. C., A. Yip and F. Park, Simultaneous Total
   Variation Image Inpainting and Blind
   Deconvolution, UCLA Mathematics Department CAM
   Report 04-45.