GDC 2005

1
Mixed Resolution Rendering
Jeremy Shopf AMD Game Computing Applications
2
Outline

• Motivation
• Uniform Upsampling
• Adaptive Upsampling
• Demo

3
Offscreen Particles
Nguyen04

• Used in many shipping titles
• Method
• Render scene without particles
• Downsample Depth Buffer
• Render particles to low-res color buffer, using
  downsampled depth buffer for depth testing
• Composite low-res particles
• Fix up gaps caused by low-res depth testing

4
Offscreen Particles
High-Res Version (20 fps_at_1280x960)
5
Offscreen Particles
2x Downsampled Offscreen Version (32 fps_at_1280x960)
6
Offscreen Particles
Fix-up pixels (rest are low-res)
7
Offscreen Particles

• Tradeoff
• Save on Fill Rate
• - Lose resolution

8
Mixed Resolution Rendering

• For relatively low resolution effects, mixed
  resolution rendering results in barely
  perceivable loss in quality
• Dramatic performance improvement
• Scales linearly minus small overhead for
  upsampling
• Allows scaling of effects to maintain framerate

9
Bilinear Interpolation

• Used in Offscreen Particles
• Efficient and simple
• - Doesnt respect depth discontinuities
• Requires separate discontinuity detection pass

10
Bilateral Interpolation

• Based on bilateral filter
• Filters in two dimensions
• Spatial Distance in image space
• Frequency Depth distance

11
Bilateral Upsampling
Sloan07

• Modify bilinear weights by normal and depth
  discontinuities
• Weight each coarse sample by
• - Bilinear weight
• - Normal similarity weight
• - Depth similarity weight

12
Bilateral Upsampling
Hi-Res Texels
Low-Res Texels
13
Bilateral Upsampling
Computing bilinear weights
0
1
float4 vBilinearWeights4
// 0 1 2 3dddddd
float4( 9/16, 3/16, 3/16, 1/16 ), // 0
float4( 3/16, 9/16, 1/16, 3/16 ), // 1
float4( 3/16, 1/16, 9/16, 3/16 ), // 2 float4(
1/16, 3/16, 3/16, 9/16 ) // 3
0
1
3
2
2
3
Hi-Res Texels
Low-Res Texels
14
Bilateral Upsampling
Computing normal weights
0
1
float3 vNormalsCoarse4
0
1
float vNormalHiRes ...
for(int i0ilt4i) vNormalWeightsi
dot( vNormalsCoarsei,
vNormalHiRes )
vNormalWeightsi pow(vNormalWeightsi , 32
)
3
2
2
3
Hi-Res Texels
Low-Res Texels
15
Bilateral Upsampling
Computing depth weights
0
1
0
float fDepthsCoarse4
1
float fDepthHiRes
for(int i0ilt4i) float fDepthDiff
fDepthHiRes fDepthsCoarsei
vDepthWeightsi 1.0/( EPSILON abs(fDepthDiff
))
3
2
2
3
Hi-Res Texels
Low-Res Texels
16
Bilateral Upsampling
Putting it all together for(int nSample0
nSamplelt4 nSample) float fWeight
vNormalWeightsnSample
vDepthWeightsnSample
vBilinearWeightsnTexelnSample
fTotalWeight fWeight vUpsampledShading
vShadingCoarsenSample vUpsampledShading /
fTotalWeight
0
1
0
1
3
2
2
3
Hi-Res Texels
Low-Res Texels
17
Degenerate Case

• No valid coarse samples
• Easy to detect this situation
• Happens infrequently
• Options
• Pick one coarse sample
• Average coarse samples
• Set stencil and do an additional pass to
  calculate these at full res

18
Illustration of Degenerate Case
19
Bilateral Upsampling Results

• Crease Shading

Diffuse only
20
Bilateral Upsampling Results

• Crease Shading

High-Res Shading (67 fps_at_1024x1024)
21
Bilateral Upsampling Results

• Crease Shading

Low-Res Shading (166 fps_at_1024x1024)
22
Bilateral Upsampling Results

• Sphere AO

No AO
23
Bilateral Upsampling Results

• Sphere AO

High-Res AO (62 fps_at_1280x960)
24
Bilateral Upsampling Results

• Sphere AO

Low-Res AO (135 fps_at_1280x960)
25
Bilateral Upsampling

• Small amount of overhead
• Cheap to compute weights
• - Uniform sampling
• Undersampling small features
• Oversampling low resolution regions

26
Pull-push Upsampling
Mitchell87,Grossman97

• Used to reconstruct images from sparse samples
• Allows adaptive sampling

27
Pull-push Upsampling

• Two phase algorithm fills in unknown values
  hierarchically
• Pull Bottom-up averaging of valid values

28
Pull-push Upsampling

• Two phase algorithm fills in unknown values
  hierarchically
• Push Top-down interpolation of higher levels

29
Soft Shadow Example

• Real-time Soft Shadow Mapping by Backprojection
  Guennebaud06,07
• Use a min/max pyramid of the shadow depth buffer
  to accelerate computation by culling unimportant
  shadowmap texels

30
Pull-push Upsampling

• Select pixels to be shaded based on some
  importance function

Estimated Penumbra Width
lt
Günnebaud06
Skip Computation
31
Screen Resolution Soft Shadows
32
Adaptive Sampling
33
Screen Resolution Soft Shadows
26 fps
34
Adaptive Sampling
58 fps
35
GPU Implementation

• Shading sparse points under-utilizes GPU
• Need to pack points to be shaded
• Unpack before pull-push interpolation

36
GPU Implementation

• Shading sparse points under-utilizes GPU
• Need to pack points to be shaded
• Unpack before push-pull interpolation

Select Points to be Shaded
37
GPU Implementation

• Shading sparse pixels under-utilizes the GPU
• Need to pack points to be shaded
• Unpack before push-pull interpolation

Pack Selected Points
38
GPU Implementation

• Shading sparse points under-utilizes GPU
• Need to pack points to be shaded
• Unpack before push-pull interpolation

Shade Packed Points
39
GPU Implementation

• Shading sparse points under-utilizes GPU
• Need to pack points to be shaded
• Unpack before push-pull interpolation

Unpack Shaded Points
40
Packing/Unpacking Implementation

• Render point primitive for each pixel
• Evaluate selection function
• Streamout point (with pixel coordinate) if
  selection function passes
• Shade each point (using DrawAuto()) by fetching
  normal and depth using pixel coordinate vertex
  attribute
• Unpack DrawAuto() point buffer again, use pixel
  coordinate as position

41
Discussion

• Selection function must be cheap to compute and
  the shading function expensive
• Shading in packed texture may be limiting
• Proxy-based shading (e.g. Sphere AO) incompatible
• Discontinuities have to be handled at full
  resolution

42
MultiRes Upsampling
NicholsWyman09

• Adaptive upsampling that respects discontinuities
• Compute shading at several different resolutions
• Upsample and combine hierarchically
• Use bilinear interpolation in regions with no
  discontinuities

43
Multiresolution Indirect Illumination

• Distribute Virtual Point Lights in scene via
  shadow map
• Compute direct illumination from VPLs
• 400 VPLs 400 lighting calculations per pixel

44
Detecting Discontinuities

• Construct hierarchical Min/Max depth and normal
  buffers of scene from viewer
• Like a Mip Map, but stores min and max rather
  than average for all resolutions
• If Max-Min gt some threshold, there is a
  discontinuity at this resolution

45
Choosing a resolution

• For each resolution (lowest to highest)
• For each texel
• If there is no discontinuity at this min/max
  level
• Splat
• Else
• Refine splat to higher resolution

46
Upsampling

• For each resolution (2nd lowest to highest)
• Upsample previous levels
• If all four coarse samples are available, use
  bilinear interpolation, otherwise point sample
• Combine with splat result of current res

47
Reducing artifacts

• Interpolation doesn't work well when texels are
  adjacent to texels of a non-adjacent level
• When determining discontinuity, examine
  neighboring min/max texels also
• Widens discontinuity kernel
• Should be possible to "fix up" this situation
• Refine texels adjacent to much higher res texels

48
Reducing artifacts
49
Demo
22 fps full res 57 fps multi res 400 VPLs per
pixel 1024x1024 resolution
50
Conclusions

• Mixed resolution rendering can save you perf with
  a modicum of overhead
• Uniform Upsampling
• Lightweight and scales well
• Adaptive Upsampling
• Adaptive but with more overhead

51
More Details

• Read my blog, level of detail
• http//www.jshopf.com/blog
• "Mixed Resolution Rendering"
• Contact me
• jshopf_at_gmail.com
• Thank you Greg Nichols and Chris Wyman,
• GCAG

52
References

• Peter-Pike Sloan, Naga K. Govindaraju, Derek
  Nowrouzezahrai, John Snyder. "Image-Based Proxy
  Accumulation for Real-Time Soft Global
  Illumination". Pacific Graphics 2007.
• Greg Nichols and Chris Wyman. "Multiresolution
  Splatting for Indirect Illumination." ACM
  Symposium on Interactive 3D Graphics and Games,
  83-90. 2009.
• D.P. Mitchell. "Generating Antialiased Images at
  Low Sampling Densities". In Proceedings of
  SIGGRAPH 1987, pages 65-72
• H. Nguyen. "Fire In the Vulcan Demo." GPU Gems.
• J.P. Grossman. "Point Sample Rendering". In 9th
  Eurographics Workshop on Rendering 1998. Pages
  181-192.
• Gael Guennebaud, Loic Barthe, Mathias Paulin.
  "High-Quality Adaptive Soft Shadow Mapping."
  Eurographics 2007.
• Gael Guennebaud, Loic Barthe, Mathias Paulin.
  "Real-time Soft Shadow Mapping by
  Backprojection." Eurographics Symposium on
  Rendering 2006.