Rendering With Coherent Layers

1
Rendering With Coherent Layers
SIGGRAPH 97

• Jed Lengyel John Snyder
• Microsoft Research

2
Traditional Pipeline

• Renders 3D scene to an image.

3
Traditional Pipeline

4
Traditional Pipeline

• Problems
• Does not exploit temporal coherence
• Spatial and temporal resolutions are global
• Limited integration of 2D elements in 3D

5
Traditional Pipeline

6
Traditional Pipeline

• How can we improve this?

7
Layered Pipeline

• Add a 2D image warp.

8
Layered Pipeline

• A sprite is a warped image with alpha.

Sprite
9
Layered Pipeline

• Each layer produces a sprite.

Layer
10
Layered Pipeline

• The scene is factored into layers.

11
Flyby Film
12
Spatial Resolution
13
Previous Work

• Image-caching Shade96, Schaufler96
• Composition architectures (with z per pixel)
  Duff85, Molnar92, Regan94, Mark97
• Image-based rendering Chen93, Chen95,
  McMillan95
• Regulation Funkhouser93, Maciel95
• Shading factorization Cook84, Hanrahan90,
  Segal92, Dorsey95, Guenter95, Meier96

14
2D Image Transform

• T is 3D transform to screen coordinates.

15
2D Image Transform

• Extra degrees of freedom

16
2D Image Transform

• Composition maps to same screen point.

17
2D Image Transform

• Transform A maps geometry tightly.

18
2D Image Transform

• Transform B is looser than A.

19
2D Image Transform

• Spatial resolution

20
2D Image Transform

• Temporal resolution

21
Factoring Geometry

22
Factoring Geometry

• Good

23
Factoring Geometry

• Good

24
Factoring Geometry

• Good

25
Factoring Geometry

• Good

26
Factoring Geometry

• Poor

27
Factoring Geometry

• Poor

28
Factoring Geometry

• Poor

29
Factoring Geometry

• Poor

30
Factoring Geometry

• Poor

31
Depth Sorting of Layers

• Depth sorting in software is effective.
• Relatively small number of primitives
• Exploits temporal coherence

32
Factoring Shading

33
Multipass Shading
Segal92
34
Multipass Shading
35
Factoring Shading

36
Warp Choice

• Accuracy of image interpolation through time
• Efficient hardware implementation
• Ease of computing the warp parameters

37
Characteristic Points

• Track 3D motion projected to 2D

38
Warp Calculation

• Characteristic points are matched

39
Warp Calculation

W P0 P1
40
Warp Calculation

W P0 P1
41
Warps Considered

• Pure translation
• Translation with isotropic scale
• Translation with independent scale in x and y
• Affine
• Perspective

42
Warp Comparison

43
Image Interpolation

• Warping intermediate images is effective
• Triple- or quadruple-framing
• Warp calculated with a small set of points
• Decouples 3D rendering from display

44
Image Interpolation

45
Regulation

• Set independent layer quality parameters

46
Regulator

• Maximize fidelity, balance resources

47
Fiducials

• Measure fidelity of approximation
• Geometric
• Photometric
• Sampling
• Visibility

48
Geometric Fiducial

• Compares warped and current points

49
Photometric Fiducial

• Samples lighting at characteristic points

50
Photometric Fiducial 2

• Measures change in light position

51
Sampling Fiducial

• Measures distortion of image samples

52
Visibility Fiducial

• Counts back-to-front transitions

Frame 0
Frame 1
53
Barnyard Example
54
Conclusion

• Layers are good for real-time 3D.
• Effective using affine image warp
• Exploit temporal coherence
• Factor shading
• Allow precise targeting of resources
• 3x-10x improvement