Low-cost Photometric Calibration for Interactive Relighting

1
Low-cost Photometric Calibration for Interactive
Relighting

• Céline Loscos and George Drettakis
• iMAGIS-GRAVIR/IMAG-INRIA
• Computer Sciences Department - University College
  of London

2
Context augmented reality

• Mix real and virtual worlds
• Applications
• entertainment (virtual studio)
• cinema (special effects)
• medical
• etc.
• Richer virtual experience
• real world references

3
Common illumination
Breen et al. 96
4
Motivation

• Applications in augmented reality
• need of interactive systems
• simulation of realistic illumination needs time
• Find the right balance between
• the capture process speed
• the response speed of the system
• the quality of the lighting simulation
• convincing results

5
Overview

• State of the art
• relighting and remodelling for several known
  lighting conditions Loscos et al. 99
• Photometric calibration
• Conclusion

6
Realistic lighting simulation

• Global illumination
• direct (from light sources) indirect light
  (inter-reflections)
• Lighting simulation
• Input
• scene geometry reflectance and emittance
  properties of surfaces
• Output
• lit scene
• Reflectance describes the portion of light
  reflected
• Classical methods ray casting or radiosity

7
Inverse illumination

• Goal
• find radiometric properties (reflectance, light
  source exitance)
• real scene known independently of the original
  lighting conditions allows relighting
• Inverse illumination Sato et al. 97, Yu et al.
  99, etc
• input
• lit scene
• output
• reflectance estimation

8
Our relighting method Loscos et al. 1999

• Interactive relighting of real scenes
• Realistic common illumination
• consistency of lighting between real and virtual
• Simple capture process
• few photos
• low-cost equipment

9
Assumptions

• Relighting from a single viewpoint
• Diffuse scene
• Direct lighting ray tracing
• Indirect lighting hierarchical radiosity

10
Input data reflectance estimate

• Radiance images from a single viewpoint
• a single light source per image

Different lighting conditions
11
Reflectance estimate pixel per pixel

• For each radiance image
• Indirect approximated by an ambient term

reflectance radiosity / (direct light
indirect light )
Original photograph
Estimated reflectance
12
Merged reflectance
reflectance
confidence
Merged reflectance
x
avg.
x
13
Results of the method Loscos et al. 1999

• Video

14
Limitations in the reflectance estimate

• Colours transformed by the camera
• loss of information saturation, etc.
• Inaccuracy of the reflectance estimate

reflectance
Reflectance
pixels
15
Solution High-Dynamic Range images

• Radiance images Debevec et al. 97
• Input
• several pictures from the same point of view at
  different shutter speeds
• RGB values within integer range 0-255
• Output
• cameras response function
• high-dynamic range of colours
• Remark need to control the shutter speed

16
Adaptation low-cost HDR images

• New solution for a semi-automatic digital camera
  Kodac DC260
• No direct control of the shutter speed
• Use of the EV parameter provided by the camera

17
Adaptation low-cost HDR images

• 9 EV values -2..2 9 different exposure times
• EV 0 automatically chosen shutter speed
• Use of the conversion typically used in
  photography
• Fix to an arbitrary value ( EV 0)
• Results in
• better range of colours and less saturation

18
Limitations in the reflectance estimate

• Problems
• several lighting conditions
• exposure time automatically selected by the
  camera
• inconsistent radiance values
• Make radiance images consistent
• based on radiosity equation
• least squares solution

19
Make radiance images consistent

• Algorithm
• choose a reference radiance image
• compute a reference reflectance for the reference
  image (only for directly lit areas)
• compute an error factor for each radiance image
• apply this factor to get a consistent image

20
Limitations in the reflectance estimate

• Incorrect illumination estimate
• incorrect estimate in shadow areas

21
Iterative algorithm for reflectance estimate

• For each pixel
• convergence of reflectance values

22
Calibration results
Reflectance
RGB
Initial radiance
After iterations
23
Calibration results
Reflectance for a scanline (RGB)
reflectance
pixels
24
Calibration results
Reflectance for a scanline (initial radiance)
reflectance
pixels
25
Calibration results
Reflectance for a scanline (after iterations)
reflectance
pixels
26
Calibration results
Reflectance (single exposure time)
RGB
Initial radiance
After iterations
27
Improvements due to calibration
Reflectance (single exposure time)
RGB
Initial radiance
After iterations
28
Conclusion

• Photometric calibration
• improvement of the reflectance estimate quality
• respects the restrictions to the low-cost
  computation and equipment price

29
Future work

• Improve the final display
• apply the response function of the camera
• apply a tone mapping
• Simplify the capture process
• General perspectives
• specular effects
• moving viewpoint
• outdoor scenes
• toward real time