A%20Theory%20of%20Multiplexed%20%20Illumination

1
A Theory of Multiplexed Illumination
Yoav Y. Schechner

• Technion, Israel

Shree Nayar, Peter Belhumeur
Columbia University
ICCV Conference October 2003, Nice, France
Acknowledgments Taub Foundation, BSF, NSF,
Harish Peri
2
Shape Recovery
BRDF
Material Recognition
Human Vision
Rendering
Object / Face Recognition
3
Image-Based Rendering
based on
Schechner et. al. Multiplexed Illumination
4
Our Method
Schechner, Nayar, Belhumeur Multiplexed
Illumination
5
Image-Based Rendering / Recognition


Schechner et. al. Multiplexed Illumination
6
Illumination Direction Resolution
Schechner et. al. Multiplexed Illumination
7
Trade-Off
Schechner et. al. Multiplexed Illumination
8
Trade-Off
Schechner, Nayar, Belhumeur Multiplexed
Illumination
9
Schechner, Nayar, Belhumeur Multiplexed
Illumination
10
(No Transcript)
11
Multiplexed Illumination
i intensity under source k a acquired
measurement
k
12
Standard Illumination
13
Multiplexed Illumination
14
Multiplexed Illumination
Demultiplexed Images
15
Theory of Multiplexed Illumination
i intensities under individual sources a
acquired measurements
For each pixel
W
a i
16
Optimal Multiplexing Codes
Schechner, Nayar, Belhumeur Multiplexed
Illumination
17
Optimal Multiplexing Codes
Schechner, Nayar, Belhumeur Multiplexed
Illumination
18
Easy to Invert
Schechner, Nayar, Belhumeur Multiplexed
Illumination
19
Fixed acquisition time T
SNR
SNR
multiplex
single
Schechner, Nayar, Belhumeur Multiplexed
Illumination
20
Fixed acquisition time T
SNR
SNR
multiplex
single
Fixed high quality
SNR
multiplex
demultiplexed
standard
T
T

multiplex
single
standard
Schechner, Nayar, Belhumeur Multiplexed
Illumination
21
Fixed acquisition time T
SNR
SNR
multiplex
single
Fixed high quality
SNR
multiplex
demultiplexed
standard
T
T

multiplex
single
Fixed acquisition time
T
single
demultiplexed
Fixed high quality
SNR
multiplex
22
Setup Scalable,
Simple, Flexible
projector
Schechner, Nayar, Belhumeur Multiplexed
Illumination
23
Illumination Patterns
Multiplexed Illumination
Single-Source Illumination
Schechner, Nayar, Belhumeur Multiplexed
Illumination
24
Setup
Schechner, Nayar, Belhumeur Multiplexed
Illumination
25
Anistropic illumination
Schechner, Nayar, Belhumeur Multiplexed
Illumination
26
Adaptive directional resolution
Schechner, Nayar, Belhumeur Multiplexed
Illumination
27
The Quadtree algorithm
Schechner, Nayar, Belhumeur Multiplexed
Illumination
28
The Quadtree algorithm
Schechner, Nayar, Belhumeur Multiplexed
Illumination
29
Raw Images (Diffuse Objects)
Multiplexed images
Single-source images
Schechner, Nayar, Belhumeur Multiplexed
Illumination
30
Single-Source Images
Schechner, Nayar, Belhumeur Multiplexed
Illumination
31
Demultiplexed (Decoded) Images
Schechner, Nayar, Belhumeur Multiplexed
Illumination
32
Specular Objects
Schechner, Nayar, Belhumeur Multiplexed
Illumination
33
Specular Objects
Schechner, Nayar, Belhumeur Multiplexed
Illumination
34
Schechner, Nayar, Belhumeur Multiplexed
Illumination
35
Quantitative verification
noise std gray-levels
measurement samples
36
High Definition Specularities
Raw Images
Schechner, Nayar, Belhumeur Multiplexed
Illumination
37
Color
Schechner, Nayar, Belhumeur Multiplexed
Illumination
38
Diffuse Object
Schechner, Nayar, Belhumeur Multiplexed
Illumination
39
Camera Noise Model - Revisited
Schechner, Nayar, Belhumeur Multiplexed
Illumination
40
Photon (shot) Noise
Detector (pixel)
Electrons
Photon
either
50 quantum efficiency
Schechner, Nayar, Belhumeur Multiplexed
Illumination
41
Photon (shot) Noise
Electrons
either
50 quantum efficiency
Schechner, Nayar, Belhumeur Multiplexed
Illumination
42
Camera Noise Model - Revisited
Point-Grey Dragonfly, Gain 3.5 dB, 30fps
Schechner, Nayar, Belhumeur Multiplexed
Illumination
43
Schechner, Nayar, Belhumeur Multiplexed
Illumination
44
Conclusions
Multiplexing Higher quality
Very simple, yet flexible setup
Yoav Schechner Shree Nayar Peter Belhumeur