Interactive Rendering using the Render Cache

1
Interactive Rendering using the Render Cache

• Bruce Walter, George Drettakis
• iMAGIS-GRAVIR/IMAG-INRIA
• Steven Parker
• University of Utah
• iMAGIS is a joint project of CNRS/IMAG/INRIA and
  UJF

2
Motivation

• Goal Interactive rendering

Ray tracing
Path tracing
3
Motivation

• High-quality renderers
• Pixel based
• Ray tracing, path tracing, etc.
• Wide range of lighting effects
• Reflection, refraction, global illumination, etc.
• Too slow for interactive use
• Many seconds per image
• Often used only for final images
• Alternate renderers used for interactive editing

4
Motivation

• Interactive rendering
• Rapid feedback is paramount
• High accuracy is less important
• Fast consistent framerate
• Eg, gt 5 fps
• Could use faster renderer
• Eg, hardware accelerated scan conversion
• Use same renderer
• Need to bridge framerate gap

5
Visual Feedback Loop

• Standard visual feedback loop
• Entirely synchronous
• Framerate is limited by the renderer

6
Visual Feedback Loop

• Modified visual feedback loop

Asynchronous interface
7
Goals

• Independent display process
• Works with many (pixel-based) renders
• Exploit frame to frame coherence
• Reproject pixels from previous frames
• Fast consistent framerate
• Use simple, fast methods
• Concentrate rendering effort
• Prioritize pixels need for recomputation

8
Video
9
Previous Work

• Approximate or progressive approaches
• Progressive radiosity or ray tracing
• Frameless rendering
• Reprojection or warping
• Image based rendering (IBR)
• Ray tracing acceleration

10
Previous Work

• Parallel processing
• Multiprocessors or distributed clusters
• Intelligent display processes
• Post-rendering warp
• Holodeck system

11
Algorithm Overview
Display process
renderer
project
Render cache
depth cull
image
interpolate
renderer
sampling
12
Image Estimation

• Projection
• Project cached points onto current image
• Camera transform provided by application
• Z-buffer
• In case multiple points map to a pixel

13
Image Estimation

• Problem visual artifacts

Original view
New view
14
Image Estimation

• Depth cull heuristic
• Problem occluded points may be visible
• Z-buffering only works within a pixel
• Find pixels with locally inconsistent depths
• Likely to be from different occluding surfaces

Raw projection
After depth cull
15
Image Estimation

• Interpolation / smoothing
• Problem small gaps in point data
• Interpolate pixel colors
• Compute locally-weighted average colors
• Currently uses 3x3 neighborhoods

Raw projection
After interpolation
After depth cull
16
Image Estimation

• Results after each stage

Projection
Depth cull
Interpolation
17
Image Estimation

• Problem visual artifacts
• Simple filter can remove many artifacts
• Need new points from renderer
• Previously invisible areas
• Color changes due to non-diffuse shading
• Eg,specular highlights
• Changes due to user editing
• Changes in lighting, geometry, materials

18
Sampling

• Generate priority image
• Based on pixels need for re-rendering
• Priority given to pixels with older points
• Render cache stores an age with each point
• Empty pixels priority based on local density
• Highest priority given to regions without points

19
Sampling

• Choose pixels for rendering
• Sampling must be sparse
• Relatively few pixels are rendered per frame
• Chosen using error-diffusion dither
• Concentrates pixels in high priority regions
• Maintains good spatial distribution
• Requested pixels sent to renderer(s)
• Results returned at some later frame

20
Sampling
Displayed image
Priority image
Requested pixels
21
Optimizations

• Further prioritizing sampling
• Identify points that are likely to be outdated
• Color change heuristic
• Renderer supplied hints
• Prematurely age these points
• Forces sooner resampling of these points

22
Optimizations

• Moving objects
• Application can supply object transforms
• Applied to points in the render cache
• Improves tracking of moving objects
• Points also aged to encourage resampling

23
Video
24
Results

• Timing 70.5 ms or 14 fps
• 256x256 image, display process only
• 195 Mhz R10K processor

25
Conclusions

• Greatly improved interactivity
• Eg, ray tracing, path tracing
• Efficient reuse of rendered pixels
• Using reprojection and simple filters
• Prioritized sparse sampling
• Efficently uses limited rendering budget
• Independent automatic display process
• Can be used with many different renderers

26
Future Work

• Larger images
• Cost scales linearly with of pixels
• Higher render ratios
• Currently work well out to about 164
• Anti-aliasing

27
The End
28
Overview

• Cache rendered results
• Stored as colored points in 3D
• Estimate current image
• Project points onto current image plane
• Filter to reduce artifacts
• Prioritize future rendering
• Identify problem pixels
• Sparse sampling for limited render budget