Nonphotorealistic rendering, and future cameras

1
Nonphotorealistic rendering, and future cameras

• Computational Photography, 6.882
• Bill Freeman
• Fredo Durand
• May 11, 2006

2
Organization of NPR methods

• Automated methods
• 2-d processing
• 3-d processing
• Interactive methods
• 2-d processing
• 3-d processing

3
Computer generated watercolor
http//www.cs.utah.edu/npr/papers/npr_course_Sig99
.pdf
4
http//citeseer.ist.psu.edu/cache/papers/cs/12745/
httpzSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cp
zSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis9
7computergenerated.pdf
5
http//citeseer.ist.psu.edu/cache/papers/cs/12745/
httpzSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cp
zSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis9
7computergenerated.pdf
6
http//citeseer.ist.psu.edu/cache/papers/cs/12745/
httpzSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cp
zSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis9
7computergenerated.pdf
7
http//citeseer.ist.psu.edu/cache/papers/cs/12745/
httpzSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cp
zSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis9
7computergenerated.pdf
8
http//citeseer.ist.psu.edu/cache/papers/cs/12745/
httpzSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cp
zSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis9
7computergenerated.pdf
9
http//citeseer.ist.psu.edu/cache/papers/cs/12745/
httpzSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cp
zSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis9
7computergenerated.pdf
10
http//citeseer.ist.psu.edu/cache/papers/cs/12745/
httpzSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cp
zSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis9
7computergenerated.pdf
11

• Interactive application. But too slow to let you
  paint in real-time.

http//citeseer.ist.psu.edu/cache/papers/cs/12745/
httpzSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cp
zSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis9
7computergenerated.pdf
12
Offline application
User inputs
http//citeseer.ist.psu.edu/cache/papers/cs/12745/
httpzSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cp
zSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis9
7computergenerated.pdf
13
http//citeseer.ist.psu.edu/cache/papers/cs/12745/
httpzSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cp
zSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis9
7computergenerated.pdf
14
http//citeseer.ist.psu.edu/cache/papers/cs/12745/
httpzSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cp
zSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis9
7computergenerated.pdf
15

• Steps in the rendering
• (user-supplied region mask not shown)

16
Resulting watercolor
http//citeseer.ist.psu.edu/cache/papers/cs/12745/
httpzSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cp
zSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis9
7computergenerated.pdf
17
Source image for a 3d animation
18
(No Transcript)
19
(No Transcript)
20
Pen and ink illustration, exploiting 3d geometry
http//www.cs.utah.edu/npr/papers/npr_course_Sig99
.pdf
21
(No Transcript)
22
http//www.cs.utah.edu/npr/papers/npr_course_Sig99
.pdf
23

• The artist approved of this one

http//www.cs.utah.edu/npr/papers/npr_course_Sig99
.pdf
24
Future cameras

• Computational Photography, 6.882
• Bill Freeman
• Fredo Durand
• May 11, 2006

25
(No Transcript)
26
(No Transcript)
27
What can be improved about current cameras?
(your list first)

• Dynamic range
• Blurred photos
• Post-shot controllable depth of field
• Post-shot editable lighting, positions, etc.
• Size of camera

28
What crazy other things?

• The previous list is all mostly with reference to
  the functionality of a film camera. Surely
  unexpected camera capabilities and uses, only
  possible with digital media, will come with
  future cameras.

29
Some possible future directions

• Assorted pixels
• Foveon imager
• Coded shutter flutter
• Light field camera
• Gradient camera

30
Some possible future directions

• Assorted pixels
• Foveon imager
• Coded shutter flutter
• Light field camera
• Gradient camera

31
Color pixel mosaic
http//www1.cs.columbia.edu/CAVE/publications/pdfs
/Narasimhan_PAMI05.pdf
32
Intensity attenuation mosaic
http//www1.cs.columbia.edu/CAVE/publications/pdfs
/Narasimhan_PAMI05.pdf
33
Color and intensity mosaic
http//www1.cs.columbia.edu/CAVE/publications/pdfs
/Narasimhan_PAMI05.pdf
34
Color and polarization mosaic
http//www1.cs.columbia.edu/CAVE/publications/pdfs
/Narasimhan_PAMI05.pdf
35
Temporal sensitivity modulation
http//www1.cs.columbia.edu/CAVE/publications/pdfs
/Narasimhan_PAMI05.pdf
36
Shree Nayar, Columbia University
37
http//www1.cs.columbia.edu/CAVE/publications/pdfs
/Narasimhan_PAMI05.pdf
38
http//www1.cs.columbia.edu/CAVE/publications/pdfs
/Narasimhan_PAMI05.pdf
39
http//www1.cs.columbia.edu/CAVE/publications/pdfs
/Narasimhan_PAMI05.pdf
40
http//www1.cs.columbia.edu/CAVE/publications/pdfs
/Narasimhan_PAMI05.pdf
41
http//www1.cs.columbia.edu/CAVE/publications/pdfs
/Narasimhan_PAMI05.pdf
42
http//www1.cs.columbia.edu/CAVE/publications/pdfs
/Narasimhan_PAMI05.pdf
43
Original (12 bits)
http//www1.cs.columbia.edu/CAVE/publications/pdfs
/Narasimhan_PAMI05.pdf
44
As sampled (8 bits)
http//www1.cs.columbia.edu/CAVE/publications/pdfs
/Narasimhan_PAMI05.pdf
45
Cubic spline interpolation to 12 bits
http//www1.cs.columbia.edu/CAVE/publications/pdfs
/Narasimhan_PAMI05.pdf
46
Linear regression interpolation to 12 bits
http//www1.cs.columbia.edu/CAVE/publications/pdfs
/Narasimhan_PAMI05.pdf
47
Some possible future directions

• Assorted pixels
• Foveon imager
• Coded shutter flutter
• Light field camera
• Gradient camera

48
http//www.foveon.com/
49
http//www.foveon.com/files/CIC13_Hubel_Final.pdf
50
http//www.foveon.com/files/CIC13_Hubel_Final.pdf
51
http//www.foveon.com/
52
http//www.foveon.com/
53
http//www.foveon.com/
54
http//www.foveon.com/
55
http//www.foveon.com/
56
Foveon features

• Use the optical properties of silicon itself to
  separate colors.
• Different wavelengths get absorbed at different
  depths of the siliconblue, then green, then red.
• More efficient at capturing lightdont discard
  2/3 of the spectrum at each pixel.
• Variable pixel size, depending on photo mode or
  video mode.
• 2002 destined to become the standard in image
  sensors for electronic cameras., said Carver
  Mead, Foveons founder. Status now?

57
http//www.foveon.com/
58
(No Transcript)
59
Some possible future directions

• Assorted pixels
• Foveon imager
• Coded shutter flutter
• Light field camera
• Gradient camera

60
(No Transcript)
61
(No Transcript)
62
(No Transcript)
63
(No Transcript)
64
Blurring convolution kernel
Fourier transforms into sinc
Fourier transforms into a function with no zeros
65
Some possible future directions

• Assorted pixels
• Foveon imager
• Coded shutter flutter
• Light field camera
• Gradient camera

66
http//graphics.stanford.edu/papers/lfcamera/lfcam
era-150dpi.pdf
67
http//graphics.stanford.edu/papers/lfcamera/lfcam
era-150dpi.pdf
68
http//graphics.stanford.edu/papers/lfcamera/lfcam
era-150dpi.pdf
69
http//graphics.stanford.edu/papers/lfcamera/lfcam
era-150dpi.pdf
70
http//graphics.stanford.edu/papers/lfcamera/lfcam
era-150dpi.pdf
71
http//graphics.stanford.edu/papers/lfcamera/lfcam
era-150dpi.pdf
72
http//graphics.stanford.edu/papers/lfcamera/lfcam
era-150dpi.pdf
73
http//graphics.stanford.edu/papers/lfcamera/lfcam
era-150dpi.pdf
74
http//graphics.stanford.edu/papers/lfcamera/lfcam
era-150dpi.pdf
75
Some possible future directions

• Assorted pixels
• Foveon imager
• Coded shutter flutter
• Light field camera
• Gradient camera

76
(No Transcript)
77
(No Transcript)
78
(No Transcript)
79
(No Transcript)
80
end