Rendering%20with%20Concentric%20Mosaics

1
Rendering with Concentric Mosaics

• Heung Yeung Shum
• and
• Li Wei He

Presentation By Jonathan A. Bockelman
2
Agenda

1. A general description of concentric mosaics
2. Rendering concentric mosaics
3. Capturing concentric mosaics
4. Some examples
5. Issues that still need to be resolved and future
   plans
6. A brief demo

3
Rendering Made Easy... Sort of

• Problems with traditional rendering schemes
• The appeal of image-based modeling and rendering
• The plenoptic function

4
History of Plenoptic Functions
5
What is a Concentric Mosaic?

• A manifold mosaic
• A 3D plenoptic radius, rotation angle, and
  vertical elevation
• A 3D image built from a series of 360 slit images

6
Rendering a Novel View

• Any point within the outermost circle can be the
  viewpoint
• Rays tangent to the camera paths are used
• Bilinear interpolation between neighboring
  mosaics can also be used

7
The Problem of Non-Planar Rays

• Rays off the plane need to be approximated
• Objects assumed to have an infinite depth
• Vertical distortion is created

8
The Need for Depth Correction

• Depth correction can fix the vertical distortion
• 3 types of depth correction exist

9
Full Perspective Correction

• Individual corrections are made for each pixel
• Exact depths of objects are necessary
• Hole-filling problems are a complication
• Excellent results are seen in synthetic scenes

10
Weak Perspective Correction

• Corrections are made for each vertical line
• Estimated depths are calculated
• Vertical distortions can occur

11
Constant Depth Approximation

• A constant depth is used
• Users can control the assumed depth
• Vertical distortions are produced if the wrong
  depth is given

12
Consequences of a 3D function

• Vertical parallax is not captured
• Much smaller data sets are required
• Users can move in a circular region

13
Synthetic Mosaics

• 3D Studio Max can be used
• Images are cut into slits
• Depth values for each pixel can be found
• Sampling is a bit tricky

14
How NOT to Do Real World Scenes

• A series of single-slit cameras on a rotating
  beam
• A single camera that can slide along a beam

15
The Lone Camera

• A single off-centered camera sits on a rotary
  table
• Regular images are taken
• Multiple concentric mosaics can be recreated from
  one image

16
Ideal Solution

• A single camera can produce distortion
• A few tangential cameras along a beam can correct
  the problem

17
How the Pros Do It

• An single ordinary digital video camera is used
  with a rotary table
• The camera faces radially outward
• 1351 frames are captured in 90 seconds
• The system is incredibly simple and efficient

18
The Lobby Scene
3 concentric mosaics from a lobby scene
19
Occlusion
20
Horizontal Parallax
21
Lighting and Glare
22
Constant Depth Correction Revisited

• Aspect ratios are maintained at the chosen depth
• Objects at other depths are distorted

23
Point vs. Bilinear Sampling

• Point sampling is twice as fast, but image
  quality is lower
• Bilinear sampling is slower, but images are much
  smoother

24
Compression

• Since adjacent frames are very similar, a
  majority of the data can be compressed.
• Vector quantization and entropy coding allow the
  415Mb original video to be shrunk to 16Mb.
• MPEG4 compression can reduce the data size to
  640k, but blocky artifacts are created.

25
Why Use Concentric Mosaics?

• Quick and easy image capture
• Parallax and specular highlights are preserved
• Much smaller data sets than Lumigraphs
• No messy geometry and lighting
• User interaction is automatically incorporated

26
Future Endeavors

• Correcting vertical distortion
• Increasing the region of motion
• Improving compression ratios

27
One Last Example
28
Demo
29
Mathematical Madness