Practical Scene Illuminant Estimation via Flash/No-Flash Pairs

1
Practical Scene Illuminant Estimation via
Flash/No-Flash Pairs
Cheng Lu and Mark S. Drew Simon Fraser
University clu, mark_at_cs.sfu.ca
2
(No Transcript)
3
This paper Estimate Ambient Illuminant,
using Flash/No-flash Pairs
4
Whats the point? Can estimate scene (ambient)
illuminant without knowing

• Flash SPD
• Camera sensors
• Surface reflectance

5
Why estimate the illuminant? White balance, plus
many computer vision applications intrinsic
images without illumination.
Whats good about this method?

• Simple
• Fast

6
The set-up
2 images , one under ambient lighting,
another under flash.
Under Ambient Image A.
Under Both Image B.
7
The Key Pure-Flash Image

• The ambient light from A is also in B.
• Therefore if we subtract the two, we have
• F the pure-flash image.

Under Flash Image F
8
Image F the scene as imaged under Flash light
only.
Incidentally, note that there are now extra
shadows, from the flash (since its offset from
the lens).
9
1. Lambertian surface
RGB
Shading normal ? effective light-direction
Illuminant
Surface
Sensors
10
2. Narrow-band sensors
is exactly a single-spike sensor
so then
11
3. Planckian light
(in Wiens approximation)
But, can violate 1., 2., 3. and still succeed.
Gives
12

• Now take Logs, to pull apart multiplications

Camera-dept vector
Camera-dept vector
Intensity and shading
Surface
Color-temperature of light
where
13

• Wed like to remove intensity/shading term

So form geometric-mean chromaticity
In logs
Surface
Camera-dept vector
where
Color-temperature of light
14

• The point

As temp (light color) changes, move along
straight line.

• But, we have A and F images
• ?Subract them, and use same chromaticity trick
• ? Only illumination is left!

15
Log-difference Geometric-Mean Chromaticity
?

• So log-log delivers inverse-temperature
  difference
• Calibrate for 1/TA-1/TF,
• then in new scene obtain TA!

16
What does this
look like?
Moved to 2D color-matching functions in
geo-mean chromaticity. (9 Planckians, Macbeth
ColorChecker, spike sensors, xenon flash SPD)
17
Sony DXC930 sensors, DaylightsF2, actual xenon
flash SPD

• How to proceed
• Sharpen
• Find closest cluster

Reference locus
18
Effect of sharpening
Kodak DCS420
Poor clusters
?
Better clusters
ing
19
Test can we determine the illuminant?
102 illuminants, Sony camera, Macbeth patches
102 illuminants, Sony camera, Munsell patches
Estimate illum. from Munsell to Macbeth ?
Nearly 100 correctly identified.
20
Application White Balance
Image under CWF CWFXenon
No flash ??
4 calibration illuminants, HP camera, Macbeth
chart (each cluster has 24 dots)
With flash ??

• Sharpen
• Sample image at 24 locations
• evenly over image
• Same (daylight) color balance
• for training and for testing

21
Overlaps best with CWF, so use white patch of
Macbeth under CWF for white balance
Our color-balance Much closer.
Auto balance Wrong.
Fluor balance Correct.
22
Thanks! To Natural Sciences and Engineering
Research Council of Canada