Photorealistic Rendering of Rain Streaks

1
Photorealistic Rendering of Rain Streaks
Kshitiz Garg Shree K. Nayar
Department of Computer Science Columbia
University
SIGGRAPH Conference July 2006, Boston,
USA Sponsors NSF, ONR
2
A Rain Scene from The Matrix Revolutions
3
A Rain Scene from The Matrix Revolutions
Frame 1
Frame 2
Frame 3
4
Rendering Streaks using Simple Brightness Model
Commercial Software
Heuristic-based
Starik and Werman 03
ATI ToyShop Demo
N. Tataturschuk et al. 05
Hybrid-Spectral Method
Langer et al. 04
5
Rendering Streaks using Hand Drawn Textures
Hand Drawn Textures
Frame from The Matrix Revolutions
Original Image Courtesy Andy Lomas
Original Image Courtesy Warner Bros.
6
Study of Visual Appearance of Rain Streaks

• Rain Streak Appearance Model
• Rain Streak Texture Database
• Image-Based Rain Rendering Algorithm

7
Oscillations in a Rain Drop
Beard and Kubesh 1989 Andsager et al. 1992
Turbulent Flow of air
Turbulent Flow of air
A Falling Rain Drop
8
Oscillations in a Rain Drop
Beard and Kubesh 1989 Andsager et al. 1992
Oblate-Prolate Mode
Transverse Mode
Oscillations in a Rain Drop
9
Appearance of a Rain Drop
Specular Reflection
N
Refraction
Internal Reflection
10
Appearance of a Non-Oscillating Rain Drop
N
Non-Oscillating Rain Drop
Rain Streak
Instant Time Shots
Falling Drop
11
Appearance of an Oscillating Rain Drop
N
Oscillating Rain Drop
Rain Streak
Instant Time Shots
Falling Drop
12
Rain Streak Appearance Model
Beard and Kubesh 1989 Andsager et al. 1992
Shape of Rain Drop
?
?
Oblate-Prolate Mode
Transverse Mode
Fall Direction
fdrop
(00 3600)
Not Rotationally Symmetric
13
Measurement of Rain Streak Appearances
Experimental Setup
Lighting and Viewing Configuration
Drop generator
Height 15 m
qlight (500, 900, 1300)
flight (100 , 300,, 1700)
qview (700, 900, 1100)
y,ydrop
Oscillation Parameters
qlight
qview
(A2,0 A3,1 fdrop) (10 images per
light view)
x
fdrop
flight
xdrop
-z
14
Estimating the Oscillation Amplitudes
qview
90
70
qlight
130
50
50
130
90
50
130
90
flight
130
10
70
30
10
150
30
10
110
50
170
30
170
90
110
50
130
30
Measured
Estimation Result
Oscillation Amplitudes
15
Comparison of Actual vs. Rendered Rain Streaks
qview
90
70
qlight
130
50
50
130
90
50
130
90
flight
130
10
70
30
10
150
30
10
110
50
170
30
170
90
110
50
130
30
Measured
Rendered
16
Precomputed Database of Streak Textures
Factors Affecting the Appearance
Render Offline (Database)
Lighting and Viewing Direction
Oscillations Parameters
Render Online (Algorithm)
Database contains 6300 HDR Rain Streak Textures
(400 MB)
( Publicly available Please send email to
rainstreaks_at_cs.columbia.edu )
17
Rain Rendering Pipeline
Render Novel Streak Textures
User Input
Rain Dynamics
Streak texture (Source 1)
Streak texture (Source n)

Streak texture (Ambient)
Scene Image/Video
Scene Image/Video With Rain
Rain Streak Database
18
Rendered Rain with Varying Light Direction
-150o
150o
Light Source Direction
19
Constant Brightness Streak Model
20
Comparison with Constant Brightness Model
Streak Appearance Model
Constant Brightness Model
-150o
150o
Light Source Direction
21
Night Scene with Isotropic Source
Coarse Depth Map
Image of the Scene
22
Night Scene with Isotropic Source
Rendered Rain
23
Night Scene with Isotropic Source
Defocus Effects
24
Scene with Colored and Anisotropic Sources
Coarse Depth Map
Scene with Multiple Sources
25
Scene with Colored and Anisotropic Sources
Rendered Rain
26
Scene with Colored and Anisotropic Sources
Streak Appearance Model
Constant Brightness Model
27
Scene with Changing Sky Illumination
Input Scene Video
28
Scene with Changing Sky Illumination
Rendered Rain
29
A Scene with Moving Light Sources
Input Scene Video
30
A Scene with Moving Light Sources
Rendered Video
31
Future Extensions

• Real-time Implementation of Rendering Algorithm
• Incorporating the Effects of Wind and Turbulence

32
Summary
Rain Streak Texture Database
Rain Streak Appearance Model
( Publicly available rainstreaks_at_cs.columbia.edu
)
Image-Based Rain Rendering Algorithm
33
Complex Appearances of Rain Streaks
Real Images of Rain Streaks