Image Denoising using Locally Learned Dictionaries

1
Image Denoising using Locally Learned Dictionaries

• Priyam Chatterjee
• Peyman Milanfar
• Dept. of Electrical Engineering
• University of California, Santa Cruz

Computational Imaging VII 20 Jan, 2009
2
Overview

• Data Model
• Kernel Regression for Denoising
• Denoising with Locally Learned Dictionaries
  (K-LLD)
• Results
• Conclusions

3
Data Model

• Pointwise data model
• Patchwise model

Zero-mean I.I.D. noise
Observation
Locally smooth function to be estimated
denoted as
4
Steering Kernel Regression (SKR)
Data-dependentweights

• Optimization problem
• Solution Nonlinear filters

Polynomial basis
5
SKR Weights

• Weights based on pixel self-similarity in a
  local patch
• Covariance matrix takes into account
  orientation and strength of edges

Gradient Covariance
6
Steering Kernel

• Decompose the matrix into three components
• Scaling parameter, rotation matrix, and
  elongation matrix.

Elongate
Rotate
Scale
7
SKR Weights
Noisy
Note how the weights adapt to the underlying
image structure
H. Takeda, S. Farsiu, and P. Milanfar, Kernel
Regression for Image Processing and
Reconstruction, IEEE Trans. on Image
Processing, vol. 16, no. 2, pp. 349-366, February
2007.
8
Now we extend it ..

• is fixed order, everywhere -- not depending
  on underlying image structure
• Lower orders fit flat regions, higher order for
  texture and fine details
• Global dictionary does not adapt to local image
  characteristics
• Dictionary atoms should capture underlying local
  image structure

9
Denoising with Locally Learned Dictionaries
(K-LLD)

• Identify dictionary which best captures
  underlying geometric structure
• Similar structures will have similar dictionary,
  similar weights
• Cluster image based on geometric similarity
  (K-Means on the SKR weights)
• Learn dictionary and order of regression for each
  cluster

10
K-LLD Algorithm Outline
Noisy Image
11
K-LLD Algorithm Outline
12
K-LLD Algorithm Outline
13
Clustering Stage

• Objective Cluster image based on geometric
  similarity of underlying data
• Feature Selection
• What features capture data geometry ?
• Distance Metric
• What metric captures distance between features ?
• Clustering Algorithm
• What algorithm segments the image best ?

14
K-Means for Clustering

• Features normalized steering wts
• Distance Metric L2
• Initialization Randomly initialize cluster
  centers
• Run K-Means multiple times and select result that
  minimizes within-cluster distance

15
Noisy
Noise-free
Clustering the noise-free image
Clustering the noisy image
16
K-LLD Algorithm Outline
17
Dictionary Selection

• Represent each patch in the kth cluster
• Variable Proj.

Mean patch of k-th cluster
Enforce orthonormality
Solved by PCA
18
Dictionary Selection

• PCA in each cluster to form a dictionary
• Describe data without fitting noise
• Number of atoms based on cluster geometry
• Clusters with flat regions need fewer atoms,
  finer details need more

Singular values
patch size
No. of atoms
constant
19
Example House Image
Cluster
Atom 1
Atom 2
Atom 3
Dictionary atoms for AWGN of std. dev. 15
20
Example Noise-free case
Cluster
Atom 1
Atom 2
Atom 3
Atom 4
Few of the atoms in the dictionaries for
different clusters
21
Example Noise-free clustering
Cluster
Atom 1
Brick facade
Atom 2
Atom 3
Atom 4
Few of the atoms in the dictionaries for
different clusters
22
Data Representation

• Data represented as
• For point-wise estimator, local weights should be
  considered
• Cluster boundaries not necessarily described well
  by dictionary
• Protects against errors in clustering

Unknown, to be estimated
23
Algorithm Outline
24
Coefficient Calculation

• Kernel Regression
• Weighted least squares solution
• Final estimate

25
Algorithm Outline
26
Iteration

• Re-learn weights (features) from denoised image
• Perform clustering of updated image using new
  features
• Learn dictionary from updated image
• Kernel regression on input noisy image
• Preserves edges and finer structures

27
Algorithm Outline
28
K-LLD Algorithm Outline
Noisy Image
Iterate
Original Noisy Image
29
Performance Gain
Performance gain by iterating
30
Results AWG noise (std dev 25)
BM3D, MSE 88.82 SSIM 0.841
Original Parrot Image
Noisy Image
K-LLD, MSE 96.95 SSIM 0.825
K-SVD, MSE 101.54 SSIM 0.826
SKR, MSE 99.96 SSIM 0.826
31
More Results
MSE
SSIM
32
Results Real noise color
ISKR
K-LLD
BM3D
33
Color Results
BM3D
Original Image
ISKR, Order 2
K-LLD
34
Conclusions

• Data adaptive method having multiple degrees of
  freedom
• Denoising through a local data representation of
  images
• Can be extended to have adaptive patch size based
  on geometry of data in each cluster

35
Thank you
P. Chatterjee and P. Milanfar, Clustering-based
Denoising with Locally Learned Dictionaries,
Accepted for publication in IEEE Trans. Image
Processing Available at http//www.ee.ucsc.e
du/milanfar
36
Iterative Scheme

• Why iterate ?
• Weights true to underlying structure in presence
  of lesser noise
• Better weights means better clustering
• Dictionary captures underlying data better when
  learned on less noisy image