IBM / IBR

1
IBM / IBR

• The study of image-based modeling
• and rendering is the study of
• sampled representations of geometry.

2
Shortcutting thevision/graphics pipeline
real world
vision pipeline
image-based rendering
geometry
graphics pipeline
views
(from M. Cohen)
3
Apple QuickTime VRChen, Siggraph 95

• outward-looking
• panoramic views taken at regularly spaced
  points
• inward-looking
• views taken at points on the surface of a sphere

4
View interpolationfrom a single view

• 1. Render object
• 2. Convert Z-buffer to range image
• 3. Re-render from new viewpoint
• 4. Use depths to resolve overlaps
• Q. How to fill in holes?

5
View interpolationfrom multiple views

• 1. Render object from multiple viewpoints
• 2. Convert Z-buffers to range images
• 3. Re-render from new viewpoint
• 4. Use depths to resolve overlaps
• 5. Use multiple views to fill in holes

6
Post-rendering 3D warpingMark et al., I3D97

• render at low frame rate
• interpolate to real-time frame rate
• interpolate observer viewpoint using B-Spline
• convert reference images to polygon meshes
• warp meshes to interpolated viewpoint
• composite by Z-buffer comparison and conditional
  write

7
Results

• rendered at 5 fps, interpolated to 30 fps
• live system requires reliable motion prediction
• tradeoff between accuracy and latency
• fails on specular objects

8
Light field renderingLevoy Hanrahan, Siggraph
96

• must stay outside convex hull of the object
• like rebinning in computed tomography

9
A light field is an array of images
10
Stanford multi-camera array(Horowitz, Levoy,
Hanrahan)

• cheap imagers cheap optics fast networking
  plentiful computation high-performance
  imagingusing an array of low-cost cameras

11
Applications for the array

• key issue is the spacing andarrangement of the
  cameras

12
Cameras tightly packedhigh-X imaging

• high-resolution
• by abutting the cameras fields of view
• high speed
• by staggering their triggering times
• high dynamic range
• mosaic of shutter speeds, apertures, density
  filters
• high depth of field
• mosaic of differently focused lenses

13
Cameras widely spaceda video-rate light field
camera

• compression of (video) light fields
• key is estimation of rough geometry
• autostereoscopic display of light fields
• lens arrays, bumpy mirrored sheets
• shape from light fields
• Do more images help?