Recovering High Dynamic Range Radiance Maps from Photographs

1
Recovering High Dynamic Range Radiance Maps from
Photographs

• Paul Debevec and Jitendra Malik

2
Background on HDR (not from paper)

• We usually work in the low dynamic range
• RGB from 0 to 1 displayable on typical monitors
  (perfect white 1)
• Brightness of the sun is not 1
• High dynamic range values are much greater
  (1,000X-1,000,000X)
• Light in the environment is made up of HDR
  radiances

3
Background (cont.)

• Need high-end HDR monitors to display
• Clamp values above 1 to display on LDR devices
• Lose LOTS of information this way
• Can also use a tone mapping operator to map to
  displayable values
• Caustics in path tracer

4
Background (cont.)

• NVidia summarizes HDR rendering in three points
• Bright things can be really bright
• Dark things can be really dark
• And details can be seen in both

5
LDR vs. HDR
Without HDR
With HDR
6
Problems with Photographs

• Photograph pixel values are rarely true
  measurements of relative scene radiance
• Usually an unknown, nonlinear mapping from
  radiance to pixel values
• Mapping hard to know beforehand usually the
  composition of several nonlinear mappings in the
  photographic process (both analog and digital)
• The process loses a lot of information in the
  high dynamic range

7
Sources of Nonlinearities

• Film scanning
• Development, scanning and digitization processes
  introduce nonlinearities
• Digital cameras
• Use charge coupled device arrays (CCDs) to store
  the radiance
• Cameras convert 12-bit CCD output to 8-bit values
  for storage
• Both processes map all values above saturation
  point to same maximum value

8
Photographic Pipeline
9
Difficulties in Photography

• Limited dynamic range photographer must choose
  the range of radiance values and determine the
  proper exposure time
• Shiny surfaces, sunlit scenes, scenes with
  artificial light sources all need different
  exposure times
• Difficult to capture the actual scene radiance
  with one exposure time

10
The Solution

• Take a series of photographs at different
  exposures to cover the full dynamic range
• Use the photographs to determine the response
  function of the photographic process
• Using the response function, combine all the
  images into a composite radiance map

11
Film Response

• X exposure
• E irradiance
• ?t exposure time
• X E?t
• Response function summarized by characteristic
  curve graph of optical density D against log(X)

12
Reciprocity

• Assumption upon which the characteristic curve
  (response function) is defined
• Only product of E?t is important
• If E is halved and ?t is doubled, E?t (the
  exposure X) is unaffected
• Resulting optical density D is unaffected
• Reciprocity breaks down at extreme values of ?t

13
Response Recovery

• Z final pixel value
• Response function f(X)Z
• Goal is to find response function f
• To do so, we will use
• f -1(Z) X
• X E?t
• E X / ?t

14
Response Recovery (cont.)

• Zij final value of pixel i at ?tj (photograph
  j)
• Z f(X)
• Zij f(Ei?tj)
• f -1(Zij) Ei?tj
• ln f -1(Zij) ln Ei ln ?tj
• Define g ln f -1
• g(Zij) ln Ei ln ?tj

15
Lots of math
16
More math

• From the response function, construct the HDR
  radiance map, using
• ln Ei g(Zij) - ln ?tj

17
Obtaining the Radiance Map

• Must use algorithm for each color channel
• Grayscale
• Color RGB

18
Grayscale 11 Exposures
19
Grayscale Response Function
20
Grayscale Reconstructed Map
1907.1
46.2
15116.0
1.0
18.0
21
Color 16 Exposures
22
Color Response Functions
23
Photograph 2s exposure
24
Relative radiance values
25
Final radiance map
26
Applications

• Image-based modeling and rendering
• Image processing
• Image compositing
• Research tool

27
Motion blur

• Synthetically blurred digital image

28
Motion blur (cont.)
Actual photograph from moving camera
Synthetically blurred radiance map