ImageBased Rendering

1
Image-Based Rendering

• Produce a new image from real images.
• Combining images
• Interpolation
• More exotic methods

2
Why Image-Based Rendering?

• Whats the most realistic image? A photograph.
• But photographs lack flexibility.
• Cant change viewpoint.
• Cant change lighting.

3
The need for correspondence

• Image-based rendering is mostly combining images
  to get a new image.
• Correspondences needed to sensibly combine
  images.
• If viewpoint has changed this can be hard.
• If not, its trivial.

4
How to get correspondences

• By hand
• Works if few correspondences needed
• By matching intensities
• This is really ½ of computer vision.

5
Matching

• Simplest SSD with windows.
• Windows needed because pixels not informative
  enough.
• Compare windows

• Search for windows that match well

6
Mosaics

• Take multiple images and construct one big image.
• Represented as image, cylinder or sphere.
• Allows panning and zooming.
• Simplest kind of motion.

7

• Fixed focal point.
• Correspondence needed to align images.
• Image rectification

8
(Images in paper by Szeliski and Shum, linked to
on web page)
9
(Images in paper by Szeliski and Shum, linked to
on web page)
10
(Images in paper by Szeliski and Shum, linked to
on web page)
11
Other mosaicing issues

• Pixel interpolation needed.
• Mosaicing can provide more information at each
  pixel.
• Higher resolution images possible.
• Higher dynamic range.

12
Morphing

• What happens if you interpolate images?
• Need corresponding points.

13
Morphing

• Corresponding points needed.
• Often done by hand.
• Interpolate each point.
• Position
• and intensity.
• Also use interpolation for more correspondences.

14
Linear Interpolationof Position
15
Other Interpolation

• Also interpolate intensities.
• Interpolate to find other point correspondes.

16
(Images from Marc Levoys slides
http//graphics.stanford.edu/papers/light/)
Light Field Rendering (Levoy and Hanrahan paper
and slides)
17
(Images from Marc Levoys slides
http//graphics.stanford.edu/papers/light/)
Light Field Rendering (Levoy and Hanrahan paper
and slides)
18
(Images from Marc Levoys slides
http//graphics.stanford.edu/papers/light/)
Light Field Rendering (Levoy and Hanrahan paper
and slides)
19
(Images from Marc Levoys slides
http//graphics.stanford.edu/papers/light/)
Light Field Rendering (Levoy and Hanrahan paper
and slides)
20
(Images from Marc Levoys slides
http//graphics.stanford.edu/papers/light/)
Light Field Rendering (Levoy and Hanrahan paper
and slides)
21
Interpolation

• We have possibly non-uniform samples of a 4D
  space.
• Must interpolate to fill in.
• Worry about aliasing

22
(Images from Marc Levoys slides
http//graphics.stanford.edu/papers/light/)
Light Field Rendering (Levoy and Hanrahan paper
and slides)
23
Linear basis for lighting
lZ
lY
lX
Surface normal (X,Y,Z) albedo
l Directional source (Lx,Ly,Lz) I
l(Lx,Ly,Lz)(X,Y,Z) LxlX LylY Lz l Z Take
Max of this and 0
24
Using Linear Basis for Rendering

• Render three images
• Take linear combinations.
• Why cant we do this with three real images?

25
Reflectance smooths lighting
26
Basis from diffuse lighting
l
lZ
lY
lX
lXZ
lYZ
lXY
27

• Note, this can also be done with 9 real images,
  because this is a basis that contains real images
• In 3D, real images arent in the 3D space, we
  have to take the max with 0 to get real images.

28
Non-Photorealistic Rendering

• Take a photo and turn it into a different kind of
  image.

29
De Carlo and Santella
Video
30
Image Analogies
(Pictures from image analogies paper linked to on
class web page).
Given A, A and B, generate B A bit like Efros
and Leung texture synthesis.