Breaking the Frame

1
Breaking the Frame

• David Luebke
• University of Virginia

2
Frameless Rendering
Technique Bishop et al. 1994 Implementation
Video Parker et al. 1999 Codec huffYUV
dll inf
3
Overview What Were Doing

• Spatio-temporally adaptive frameless sampling
• Prioritize sampling towards regions of greater
  change
• Spatial change edges
• Temporal change motion
• Reconstruction of resulting samples
• A deep buffer stores samples in time space
• Reconstruct image at front edge of time apply
  filter kernel with varying width in space and
  time

4
Temporally Adaptive Reconstruction

• Static scenes/regions
• Old samples useful, use them to sharpen/antialias
  
• Temporal width should dominate
• Dynamic scenes/regions
• New samples useful, old samples stale
• Emphasize new samples even if image is less sharp
• Spatial width should dominate

5
Video Preview
Traditional frameless
Adaptive frameless
Adaptive Frameless Rendering Dayal et al.,
EGSR05
6
Summary

• Better than traditional frameless rendering
• Better than traditional framed rendering!
• Frameless ungridded temporal sampling? lower
  latency
• Samples when and where needed ? better images at
  low sampling rates
• Antialiases static regions by incorporating old
  samples? lower error even than 10x sampling rate
• Still in simulation

7
Discussion Asynchronous Graphics

• What if reconstruction was part of display?
• Imagine display as systolic array of pixels
• Input stream of samples, not sequence of images
• Enables asynchronous parallel graphics
• Parallel graphics frameworks share common
  constraint must ultimately combine all results
  into a single frame
• Breaking the frame also breaks underlying
  assumption and constraint in parallel graphics!
• See SIGGRAPH Panel The Ultimate Display
• Punchline Refreshing every pixel every time
  Bad Idea

8
The End

• Acknowledgements
• OpenRT Interactive Raytracing Project
• BART ray tracing benchmark
• Stanford 3D Scanning Repository
• National Science Foundation awards 0092973,
  0093172, and 0112937

9
System overview
Sampler
Reconstructor
Sampler
Reconstructor
tiling, view,
tiling, view,
gradients
gradients
Adaptive
Adaptive
Controller
Controller
Filter Bank
Filter Bank
gradients
variation,
gradients
image
locations
variation,
image
locations
samples
samples
samples
samples
Deep
Ray
Deep
Ray
samples
samples
Deep
Deep
Buffer
Tracer
Buffer
Tracer
Buffer
Buffer
10
Temporally Adaptive Reconstruction
static scene
dynamic scene
11
Comparison Traditional Frameless
static scene
dynamic scene
12
Discussion Coherence

• What about coherence?
• Frameless rendering implicitly gives up spatial
  coherence, which is big win for fast ray tracers
• Partially ameliorate with tiled structure,
  gradient rays
• Might need to organize random samples around
  memory
• But we gain temporal coherence!
• Fewer samples neednt resample everywhere every
  frame
• Can we design a parallel architecture around this
  temporal coherence?

13
Comparison Render Cache

• Probably most closely related approach
• Sampling based on (framed) priority image
• Biased toward old undersampled regions
• Killing off old samples also biases towards age
• Semantic hints age some samples quicker (e.g.
  specular surfaces)
• Temporal response by aging samples if new one
  detects variance
• Error diffusion dither to place samples within
  image
• Image-space reconstruction via (non-adaptive)
  filtering
• 7x7 prefilter followed by 3x3 Gaussian
• Depth culling helps with occlusions
• See Walter et al 1999

14
Evaluation Mostly Dynamic
15
Evaluation Mostly Static