Bayesian Image Super-resolution, Continued

1
Bayesian Image Super-resolution, Continued
Lyndsey C. Pickup, David P. Capel, Stephen J.
Roberts and Andrew Zisserman, Robotics Research
Group, University of Oxford
Introduction
Key Idea Integrate over the unknown registrations
Synthetic Data Example
The image registrations can be viewed as
nuisance parameters to be integrated out of the
super-resolution problem in a Bayesian setting.
Goal Multi-frame super-resolution aims to
produce a high-resolution image from a set of
low-resolution images by recovering or inventing
plausible high-frequency information. With more
than one low-resolution image, high-frequency
information can be recovered by making use of
sub-pixel image displacements. The problem is
made more difficult because the sub-pixel
displacements -- as well as lighting changes and
image blur parameters -- must be recovered from
this noisy set of low-resolution input images.
This alternative accept registration
uncertainty, and integrate over the resulting
distribution to find super-resolution image
directly from inputs.
Most methods, including Tipping Bishops
Bayesian Image Super-resolution work, fix the
registration first, then compute the
high-resolution image.
First and last of the 16 input images from the
synthetic dataset.
Generative model for super-resolution
Assume small perturbations or corrections d in
the set of registration parameters
Take a Taylor series expansion of the error in
terms d
Now find log(p(xy(k))) by integrating over d
Results on real data
The goal is to reconstruct the high-resolution
image x (right) from a set of noise inputs y(k)
generated according to the model above.
10 input images, each with 4 registration
parameters (two geometric, two photometric).
Images were registered using a simple iterative
intensity-based method, typically accurate to
within 0.1 pixels. Outputs are at a zoom factor
of 4.
where
H is a block diagonal sparse matrix, so for many
covariance assumptions, S is correspondingly
sparse also. Typically C can be assumed diagonal,
and L can be solved using gradient descent
optimization methods.
Registration-Integrating (detail)
Maximum A Posteriori Solution
Huber-MAP regularizes the smooth areas well,
though gives slightly over-crisp edges. Tipping
Bishop typical Gaussian over-smoothing, though
weakening the prior would risk over-fitting to
the noise. Registration-integrating shows
letters clearly but also gives good
regularization on the constant white background
areas.
The regular Huber-MAP objective function for
finding x is where G(x) is the set of image
gradients at each pixel, and ? is the Huber
function, defined
Huber-MAP (detail)
Advantages of Registration-Integrating
There are a number of advantages to integrating
over the unknown registrations rather than the
high-resolution image in Bayesian image
super-resolution
Tipping Bishop (detail), extended to include
photometric model.
Integrating over registrations 1a Can use good
image priors such as the Huber prior, which give
much better results than Gaussian priors. 1b A
Gaussian prior for the registration offsets is
realistic. 2 The matrix in the log-determinant
term of the objective function is small
(4K). 3 Requires little more memory than the
standard Huber-MAP algorithm, so can handle large
images.
Integrating over high-resolution image 1 Must
have a Gaussian prior in order for the integral
to be tractable. This is not a good model for
natural image statistics and tends to blur out
the image edges. 2 The matrix in the
log-determinant term of the objective function is
large because the matrices tend to be NxN. 3
The full image-integrating approach can only be
run on very small input images (e.g. 9x9 pixels).
Extra terms as an image prior
The two terms of L involving S can be viewed as
an extra prior over the high-resolution image,
favouring images which are not acutely sensitive
to tiny perturbations in the registrations. Such
a perturbation is enough to change the noise
pattern on the ML estimate drastically. Two
vectors d1 and d2 are drawn from N(0,I0.042) and
used to corrupt the registration on a synthetic
dataset.