Stereopsis

1
CNS186 (S. Shimojo)
Stereopsis 1)Psychophysics and
2)Physiology
2
Part 1. Psychophysics of
Stereopsis 1. Principles -Distance and depth
Make a clear distinction between absolute
distance (between observer and object)
relative distance, or depth (between or within
object(s)) -Depth cues and common
representation Taxonomy static vs. dynamic,
monocular vs. binocular, physiological
vs. pictorial, etc. Generic representation of
depth based on various cues? cf. Transfer of
depth contrast (simultaneous/sequential
Graham, M. Rogers, B. 1982) cf.
Stimulus convergence and response convergence
(A. Yonas)
3
-Horopter (Vieth-Muellers circle) defined
by fixation points for a constant vergence
angle. This determines zero
disparity. -Binocular disparity lateral
displacement of retinal images between the
eyes, caused by depth in the real
world. -Binocular vision (fusion vs. rivalry),
determined primarily by correlation between
the two eyes inputs. cf. Helmholtzs
stereogram(opposite contrast between
the half-images) -Scaling by distance Given
a disparity, depth is correlated with distance2.
4
Theoretical and empirical horopter
Binocular disparity
5
Classical stereogram
Panums fusinal area
6
2. Classical stereogram classical
theories -Stereoscope (Wheatstones
Brewsters) -Stereoacuity as a hyperacuity (both
in order of several min. in the normal
adult) -Pseudoscopic perception, or integration
of cues -Contour-based theories (eg. K. Ogle)
3. Julesz Random-dot stereogram stereo as a
module -Purely cyclopean (cf. Fodor
modularity) -Early, and bottom-up -Ambiguity
solving and the ghosts(--gt4) -Binocular
correlation (random-dot correlogram) -Hysteresis -
Flexibility -Filling-in and surface formation
(2-5 dot density is sufficient)
7
Theoretical Impact of Julesz Ran-dot Stereogram
8
(No Transcript)
9
Construction of a Ran-dot Stereogram
10
Autostereogram
Wallpaper effect
11
4. Constraints The stereo mechanism implements
constraints to solve the correspondence problem.
THE computational belief. -Uniqueness D.
Weinshalls special case of violation. -Order
(shallow slope) J. Foleys nail illusion.
D. Jones J. Maliks computational
model. -Smoothness G. Mitchson S. McKees
spatially repeated stereogram. However, do not
forget that the correspondence solving is not
the final goal. --gt Depth assignment, and
consistent 3D scene interpretation is the final.
(eg. illusory motion in stereogram, vertical
disparity)
12
Illusory Motion with Stereogram
13
Why is vertical binocular disparity relevant?
B. Rogers and his collaborators have shown that
vertical disparity matters to stereo
perception.
14

• 5. DaVinci Stereopsis (K. Nakayama S. Shimojo,
  1989)
• -Is interocular correspondence really necessary
  for stereo?
• No. (cf. slit view)
• -Depth perception and illusory occluding edge by
• unpaired stimulus (Da Vinci stereo).
• A new concept of Binocular stereopsis, based on
  the eye-of-
• origin information, not necessarily on
  simultaneous binocular
• inputs.

15
Da Vinci Stereopsis
(Nakayama Shimojo, 89)
16
Da Vinci Stereopsis
Stereo matching data
Constraint zones
17
6. Stereo as a grouping/segregation cue

• Intrinsic/extrinsic edges, and the aperture
  problem
• (Shimojo et al., 89)
• Grouping/segregation in behind surface
• (Nakayama, et al., 89)
• Ecological constraints, and escape from rivalry
• (Shimojo Nakayama, 90)

18
Occluding edge intrinsic Occluded edges
extrinsic Only the intrinsic edges matter for
motion ambiguity solving in the aperture!
(Shimojo et al., 89)
19
Stereo as a grouping/segregation factor
Depending of depth/occlusion, a single, or three
black disks are seen.
(Nakayama, et al., 89)
20
Occlusion and Binocular Rivalry
Ecologically invalid unpaired zone undergoes
rivalry. Valid zones, however, escape from it.
(Shimojo Nakayama, 90)
21
7. Development of stereopsis and the eye of
origin -Is 3D perception innate? Yes, and
No. -Pre-stereoptic binocular vision and the
onset of ocular- dominance segregation ?
constrains the onset of stereopsis at about
4-months of age in the normal human infant.
Onset of stereopsis and fusion/rivalry in
infants (Birch, et al., 85)
Pre-stereoptioc binocular fusion (Shimojo, et
al., 86)
22
8. Adaptation and plasticity of
stereopsis -Stereo adaptation (cf. Graham
Rogers, mentioned above). -Adaptation to
exaggeration or reduction of disparity
occurs in order of minutes or hours (Brown,
Wallach, etc.) -Adaptation to the reversal
of disparity? (S. Shimojo Y. Nakajima,
1980) -Paradox about Anisometropia the patients
do not suffer artificial tilt in depth
(which is expected from the optical
distortion) until they are tested with
stereoscope.
9. Evolution of stereopsis - In ancient nocturnal
mammals? Replacement of motion parallax? (J.
Pettgrews speculation) - Anatomically different
projections from the two eyes in birds -gt an
example of convergence of evolution.
23
Part 2. Neurophysiology of Stereopsis
1. Different recording techniques. -
Intracellular recording - Extracellular
recording - Imaging -Intracellular
recording Record inside the cell using glass
electrodes. This technique allows the
experimenter to record sub-threshold responses
and to understand mechanisms behind a cells
response characteristics. -Whole-cell
intracellular recording A modern form of
intracellular recording. This technique allows
the experimenter to introduce chemicals into the
cell and manipulate the cells milieu. This is a
very powerful technique that maintains greater
recording stability and greater control over the
cells chemical and electrical properties.
24
-Extracellular recording As the name suggests,
this technique entails the recording (metal)
electrode staying close to but outside the cell.
Using this technique, one may record from
- Anesthetized, paralyzed animals.
Stable recordings. Better
single-unit isolation. - Awake monkeys.
Recordings from the same animal
for several years. More
realistic recordings.
25
2. Other modern techniques in electrophysiology.
-The techniques described so far have one thing
in common only one neuron (or a few at most)
can be recorded simultaneously. Other techniques
have been developed to record from a population
of neurons. These include - Multiple
electrode extracellular recording. -
Optical recordings with voltage sensitive dyes.
- Optical recording of intrinsic signals
- fMRI - Chronic extracellular
recording.
26
3. Physiology of stereopsis. -Horizontal
disparity (HD) is the key to stereopsis.
Manipulating HD cues and recording from neurons
in cortex. -Vertical disparity (VD) is not a
direct cue (although it contribute in some
specific fashions cf. induced depth effect).
Small VDs often ruin stereopsis. -Why V1? First
area where signals from both eyes converge. cf.
monocular and binocular neurons, ocular dominance
segregation.
27
Projections from the eyes to LGN and V1
28
(No Transcript)
29
Response of area 17 neurons to classical and
ran-dot stereograms
30
-Yet, many areas have disparity-tuned neurons.
-Pettigrew and colleagues recorded from cats
(70s). Postulated fine disparity
tuning. -Poggio and colleagues recorded
from V1, V2 and V3a in awake monkeys
(80s). -MT neurons tuned for motion
direction and disparity as well
(Andersen). -Some other areas contain
disparity-selective neurons, but they
behave more towards perception rather than the
retinal, local disparity (I. Fujita
and his colleagues, in 00s). -Classification of
neurons based on response characteristics.
- Zero disparity tuning neurons a) excitatory
(TE)
b) inhibitory (TI) - Tuned
near (TN crossed disparities). - Tuned
far (TF uncrossed disparities).

31
Classses of Disparity Tuning
32
Idealized Tuning Functions
33
-Responses to solid bar stereograms and to
dynamic random-dot stereograms are in general
similar. -Mechanism for stereopsis. Receptive
field disparities between the receptive fields
for the two eyes of the same cell might account
for disparity tuning. RF displacements for the
two eyes.
What is important about G. Poggios work? ?Idea
of coarse coding. Broad tuning of neurons for
disparity. Gives idea of population coding. Our
depth acuity is better than this. Neurons in
different channels (parallel pathways?) must
cooperate to give us better depth
acuity. Phase-selective neurons? (De Angeles,
Freeman, and Ohzawa) Unanswered question Why
have excitatory and inhibitory tunings?
34
4. Interocular suppression and rivalry. When
there is no correlation at any disparity, there
is no depth. So, as an extreme case, when the
two eyes see gratings of orthogonal
orientations, there is no depth. This is
binocular rivalry. -Relation to stereo
exclusive, or related? (60s-70s) -Logothetis et
al. recordings showed neurons in V1, V2, MT, V4
and IT all showed rivalry-tuned responses. The
percentages kept increasing as they went further
downstream. In lower cortical areas, majority of
cells responded irrespective of whether the
monkey reported the percept. -Tong and Engel,
on the other hand, recently demonstrated in their
fMRI study that the V1 area corresponding to the
blind spot in one eye shows activity modulation
in correlation with percept. (Still
controversial.)
35
-The issue is what is rivalrous - the stimulus or
the eye. Test swap the stimuli between the two
eyes result the same stimulus continues to
dominate. Hence, the rivalry is between
alternative stimulus representations. (Still
controvercial.) -Why is studying rivalry
important? Because this leads into a study of
consciousness. Note that the suppressed stimulus
hits the retina, yet the observer is not
conscious of it.
36
5. Development of binocular vision. There is
behavioral and physiological evidence that normal
humans have average onset of both binocular
fusion(rivalry) stereo. at 4 month of age (Birch
et al., Held et al., Shimojo et al., in
80s). -What constraints this onset? Presumably,
maturation and segregation of monocular neurons
in V1. -Some ocular abnormality (such as
strabismus) can be detected by such early
binocularity tests, and is quite informative to
sort out genetic, and early environmental
factors.
37
(No Transcript)
38
References (Physiology) G. Poggio
(1994) Stereoscopic processing in monkey visual
cortex a review. In T. Papathomas, C. Chubb, A.
Gorea, and E. Kowler (Eds.) Early Vision and
Beyond, pp 43-53. Cambridge, MA The MIT
Press. J.D. Pettigrew, T. Nikara, and P.O.
Bishop (1968) Binocular interaction on single
units in cat striate cortex Simultaneous
stimulation by single moving slit with receptive
fields in correspondence. Exp. Brain Res. 6,
391-410. D. Leopold and N. K. Logothetis (1996)
Activity changes in early visual cortex reflect
monkeys percepts during binocular rivalry.
Nature 379, 549-553. Uka T, Tanaka H, Yoshiyama
K, Kato M, Fujita I. (2000) Disparity
selectivity of neurons in monkey inferior
temporal cortex. Journal of Neurophysiology, 84,
120-132.