Title: High-performance imaging using dense arrays of cameras
1Light field microscopy
Marc Levoy, Ren Ng, Andrew Adams Matthew
Footer, Mark Horowitz
Stanford Computer Graphics Laboratory
2Executive summary
- captures the 4D light field inside a microscope
- yields perspective flyarounds and focal stacks
from a single snapshot, but at lower spatial
resolution - focal stack ? deconvolution microscopy ? volume
data
3Devices for recording light fields
(using geometrical optics)
- handheld camera Buehler 2001
- camera gantry Stanford 2002
- array of cameras Wilburn 2005
- plenoptic camera Ng 2005
- light field microscope (this paper)
4Light fields at micron scales
- wave optics must be considered
- diffraction limits the spatial angular
resolution - most objects are no longer opaque
- each pixel is a line integral through the object
- of attenuation
- or emission
- can reconstruct 3D structure from these integrals
- tomography
- 3D deconvolution
5Conventional versus plenoptic camera
6Conventional versus plenoptic camera
7 8Digital refocusing
S
- refocusing summing windows extracted from
several microlenses
9Example of digital refocusing
10Refocusing portraits
11Macrophotography
12Digitally moving the observer
S
S
- moving the observer moving the window we
extract from the microlenses
13Example of moving the observer
14Moving backward and forward
15A light field microscope (LFM)
eyepiece
intermediate image plane
objective
specimen
16A light field microscope (LFM)
- 40x / 0.95NA objective
- ?
- 0.26µ spot on specimen 40x 10.4µ on sensor
- ?
- 2400 spots over 25mm field
-
- 1252-micron microlenses
- ?
- 200 200 microlenses with12 12 spots per
microlens
sensor
eyepiece
intermediate image plane
objective
specimen
? reduced lateral resolution on specimen
0.26µ 12 spots 3.1µ
17A light field microscope (LFM)
sensor
18Example light field micrograph
- orange fluorescent crayon
- mercury-arc source blue dichroic filter
- 16x / 0.5NA (dry) objective
- f/20 microlens array
- 65mm f/2.8 macro lens at 11
- Canon 20D digital camera
200µ
ordinary microscope
light field microscope
19The geometry of the light fieldin a microscope
- microscopes make orthographic views
- translating the stage in X or Y provides no
parallax on the specimen - out-of-plane features dont shift position when
they come into focus
objective lenses are telecentric
20Panning and focusing
panning sequence
focal stack
21Mouse embryo lung(16x / 0.5NA water immersion)
200µ
pan
focal stack
light field
22Axial resolution(a.k.a. depth of field)
- wave term geometrical optics term
- ordinary microscope (16x/0.4NA (dry), e 0)
- with microlens array (e 125µ)
- stopped down to one pixel per microlens
? number of slices in focal stack 12
233D reconstruction
- confocal scanning Minsky 1957
- shape-from-focus Nayar 1990
- deconvolution microscopy Agard 1984
- 4D light field ? digital refocusing ?3D focal
stack ? deconvolution microscopy ?3D volume
data
243D deconvolution
McNally 1999
focus stack of a point in 3-space is the 3D PSF
of that imaging system
- object PSF ? focus stack
- ? object ? PSF ? ? focus stack
- ? focus stack ? ? PSF ? ? object
- spectrum contains zeros, due to missing rays
- imaging noise is amplified by division by zeros
- reduce by regularization, e.g. smoothing
25Silkworm mouth(40x / 1.3NA oil immersion)
100µ
slice of focal stack
slice of volume
volume rendering
26Insect legs(16x / 0.4NA dry)
200µ
273D reconstruction (revisited)
- 4D light field ? digital refocusing ?3D focal
stack ? deconvolution microscopy ?3D volume
data - 4D light field ? tomographic reconstruction
?3D volume data
28Implications of this equivalence
- light fields of minimally scattering volumes
contain only 3D worth of information, not 4D - the extra dimension serves to reduce noise, but
could be re-purposed?
29Conclusions
- captures 3D structure of microscopic objects in a
single snapshot, and at a single instant in time
Calcium fluorescent imaging of zebrafish larvae
optic tectum during changing visual stimula
30Conclusions
- captures 3D structure of microscopic objects in a
single snapshot, and at a single instant in time - but...
- sacrifices spatial resolution to obtain control
over viewpoint and focus - 3D reconstruction fails if specimen is too thick
or too opaque
31Future work
- extending the field of view by correcting
digitally for objective aberrations
32Future work
- extending the field of view by correcting
digitally for objective aberrations - microlenses in the illumination path
- ? an imaging microscope scatterometer
angular dependence of reflection from single
squid iridophore
33http//graphics.stanford.edu/projects/lfmicroscope