Psychophysics

1
Psychophysics
How well can your patients see?
2
Chapter 4 Psychophysics The science of
measuring perception
3
Three problems faced when trying to assess how
well a patient sees. 1. They are unreliable
(within subject variability) 2. They are all
different (between subject variability) 3.
Cannot believe what they say (criterion
problem) In order to be an effective clinician
you MUST be able to solve all 3 problems.
4
Theory
Since problems 1 2 are statistical (due to
variability), we first review some simple
statistics.
1. Central Tendency (mean, median, mode)
Normal or Gaussian distribution
Number of eyes or patients (frequency)
Mean Median Mode
Score/value
5
1. Central Tendency (mean, median, mode)
Mode Most frequently occurring
score/value Median Half of eyes/patients
(sample) score above half below. Divides sample
into two equally sized groups. Mean (x) Sum
of scores above sum of scores below. Divides
sample based on scores (sensitive to outliers).
6
Asymmetric distributions mean, median and mode
are generally different
median
mode
mean
Normal (symmetric)
mean median mode
Skewed
Score/value
Notice that mean is affected most by asymmetry
and outliers, and thus when we do not want
outliers to skew your measure of central
tendency, typically median or mode are used, e.g.
media household income statistic is not distorted
by Mr. Gates.
7
Mode can often be quite variable because it is
based only upon one point in the distribution,
whereas median and mean are more stable because
they are based upon the entire sample.
mean median
mode
Number of eyes or patients (frequency)
Score/value
8
Two measures of variability
Variability
SD sqrt(VAR)
frequency
Score/value
9
Impact of variability on estimate of mean
Larger sample, better estimate of mean
SE Standard Error of Mean (standard deviation
of sample means) SE SD/sqrt(n)
frequency
Population distribution
Distribution of Sample means
SE of mean
Score/value
10
Problem 1 Within Subject Variability
Distribution of Visual Acuities from a single
patient. On any single trial VA might be
anywhere between 20/13 and 20/20.
Assign patient a single VA, use one measure of
central tendency.
How can we estimate the median or mean VA?
Improve estimate by increasing sample size
(number of trials).
frequency
Additional problem There is no direct method to
estimate VA on single trial. All we can know is
that a stimulus size is gtVA or ltVA depending on
whether it is seen or not.
20/15
20/13
20/20
Visual Acuity
11
Letter this size will be gtVA on 80 of trials and
ltVA on 20 of trials.
Estimating threshold VA
trials
Distribution of VAs
E
80
20
20/13
20/15
20/20
Visual Acuity
12
Estimating threshold VA
median
Distribution of VAs
trials
E
E
Cumulative distribution
E
80
trials in which Patient reads letter
Psychometric Function
50
20
20/13
20/15
20/20
Letter Size
13
Letter size that patient can read 50 of trials
is an estimate of median VA. Because of within
subject variability, need to measure enough
trials to get good estimate of median VA.
14
Can estimate median accurately with very few
samples
Estimating threshold VA
median
If within subject variance is very small
trials
E
E
E
trials in which Patient reads letter
80
Very steep psychometric function
50
20
20/13
20/15
20/20
Letter Size
15
In order to get a good estimate of the median VA
more accurate than the range of VAs, or sizes
over which performance is lt100 and gt0,
multiple trials/presentations must be used.
100
trials in which Patient reads letter
Psychometric Function
50
0
20/13
20/15
20/20
Letter Size
Possible psychometric functions for n1 trials
16
Problem 2 Between Subject variability
Experimentally determined distribution of median
VAs from samples of normal and diseased eyes.
Number eyes (frequency)
Disease
Normals
x
x
Score
In this examples, it is clear that the disease
produces a shift in the sample mean score (e.g.
VA). Thus, we conclude that disease affects
vision.
17
Using performance score for diagnosis
Disease
Normals
Number eyes (frequency)
x
x
Size
80
patients with VA gt
Cumulative distribution
20
Size
If your individual patient has this VA, is she
normal or does she have disease?
18
Interpretation of patient scores
normal
disease
Definitely has disease
Definitely normal
Could be normal or have disease
It is an uncertain world, and often have to make
diagnosis with uncertain data. If have to decide
(N or D), what is strategy for minimizing error
or cost of errors.
19
Two types of error False Alarm (false positive)
and Miss (false negative). Must choose a
criterion score.
normal
disease
Total error minimized by choosing criterion at
cross point in 2 distributions
low criterion
C
high criterion
Misses (have disease but diagnose as normals)
False Alarms (normals misdiagnosed as having
disease)
error
Criterion Letter Size
20/30
20/40
20/20
20
Normal
Disease
Scientific language
Correct rejections
Hits
FA
Miss
score
Clinical language
sensitivity
specificity
criterion
21
Dx
SD
Normal
Disease
Correct rejections
Hits
FA
Miss
score
The Hits and FA rate are uniquely determined by
the overlap of the distributions, or the ratio of
the Dmean and SD. This ratio is referred to as
d or detectability where dDx/SD or the
difference between the means in Z-score units
(SDs). Thus, if do experiment and record Hits
and FA, then know d.
22
Sensitivity misses 100 of disease
distribution Specificity FA 100 of normal
distribution.
Typical format for displaying these results
Decision/Diagnosis
Decision Matrix
Disease
Normal
Pmiss
Phit
Disease
Stimulus or Disease
PFA
Pcorr-reject
Normal
Theory of Signal Detection (TSD)
23
Normal
Disease
score
d
ROC curve
Low criterion
Phit (sensitivity)
High criterion
PFA (1-specificity)
24
1. Select tests with high sensitivity AND
specificity. 2. Employ multiple tests (see
below).
Two strategies for minimizing errors, and thus
costs.
Score on test 1
For example Glaucoma Test 1 IOP Test 2 C/D
ratio
Case 1 test 1 has no diagnostic value
glaucoma
normals
Score on test 2
Case 3 both test 1 and 2 have diagnostic value,
and the error rate is lower when using both
scores to diagnose (less overlap in
distributions).
t1
t1
Case 2 test 2 has no diagnostic value
t2
t2
25
Problem 3 Criterion
Whenever a patient is asked can you see or read
the stimulus? or how good does it look?, they
have to see the stimulus AND make a cognitive
decision Does the sensory signal exceed my
internal criterion for saying that I can see or
read. This internal criterion can vary
dramatically between individuals. For example,
young male pilots (low criterion) and older
patients with some disability (high criterion).
As optometrists, you must evaluate the visual
system, and you do not want your patients
internal criterion biasing your measurements.
Must develop criterion-free methods.
26
Forced Choice Methods Criterion Free
Question to subject Which side of the screen
is the grating? Not Can you see the grating?
Subject never has to decide if they can see the
stimulus, and often with forced choice methods,
people perform perfectly with stimuli that they
claim they cannot see.
27
Forced Choice Methods create their own
problems Patients can guess the correct answer
even though they cannot see the stimulus. A
correct answer due to guessing is
indistinguishable from a correct answer due to
seeing. Solution 1 Make probability of
correct guess very small. Performance due to
guessing Chance performance 1/n where there
are n alternatives.
2 AFC (Preferential Looking)
26 AFC (letter acuity)
4 AFC (tumbling E)
28
Solution 2 make subjects successfully get
multiple trials correct. To pass a line on a
VA chart requires that multiple letters be
correctly identified. The probability of getting
5 out of 5 correct by chance on a 26 AFC method
is (1/n)r, where r number of repeats and
nnumber of alternatives. Clearly, for a letter
chart, the probability that a line can be read by
chance is basically zero. (1/26)5 0.038 5
0.00000008 VSM (very small number)
29
Two alternative vs. two interval forced choice
2AFC both options are presented at the same
time (is stimulus on R or L of screen?, e.g.
preferential looking)
2IFC the n alternatives are presented at
different times in a sequence (which interval
contained the stimulus? E.g. subjective
refraction, which is better lens 1 or 2?)
t1
t2
30
Psychophysical methods for measuring
thresholds Threshold is the minimum stimulus
intensity, size, color, motion, etc. that is just
visible, resolvable, detectable, etc.
Fechner developed 3 methods 1. Method of
adjustment 2. Method of constant
stimuli 3. Method of limits More recent
methods 4. Adaptive methods (staircases) 5. TSD
(d)
Supra-threshold psychophysics
6. Magnitude estimation
31
Subject adjusts stimulus intensity, size,
contrast etc. with a dial, or some other device.
1. Method of adjustment
Threshold
Stop adjusting when stimulus is just visible.
32
2. Method of constant stimuli
A predetermined series of stimulus magnitudes are
presented, either in random or non-random order,
and proportion seen or correctly identified is
recorded.
E
Psychometric Function
E
E
E
E
trials in which Patient reads letter
50
20/13
20/15
20/20
33
3. Method of limits
Stimulus intensity is controlled by experimenter
(clinician), and either increased from invisible
(Ascending MoL) or decreased from highly visible
(Descending MoL). On A trials, subjects report
when the stimulus 1st becomes visible, on D
trials they report when it becomes invisible.
Mean transition intensity threshold
34
Experimenter adjusts stimulus intensity, size etc
in steps down from easily visible stimulus until
subject says invisible, cannot see, then
experimenter increases intensity in steps until
subjects says see, then reduce signal and
continue stepping up and down. Stimuli presented
adapt to the subjects performance.
4. Staircase Method
Adaptive Method





Threshold mean reversal intensities ().
35
5. TSD yes/no methods
The key difference with this method blank
trials are mixed in with signal trials.
Visibility is determined by the difference in
number of trials that say see with signal and
number say see with no signal.
Decision
Decision Matrix
If know Phit and PFA, then know d (Dmean/SD)
See
Do not see
Pmiss
Signal noise
Phit
Stimulus
PFA
Pcorr-reject
Noise
36
6. Magnitude Estimation
This method is rarely used, but when used it
provides information about the sensory response
to supra-threshold stimuli, and requires the
subject to assign a number (e.g. 0 to 10) to the
perceptual magnitude they experience. Note this
approach is often used by clinicians to quantify
pathology. E.g. cataract classification scale.
A cortical cataract (4 point scale)
cII
control
cI
cIII
cIV
B Nuclear Sclerosis (3 point scale)
cII
control
cI
cIII
37
Psychophysics and Optometry part A (patient as
subject, clinical test as stimulus)
1. Snellen Acuity Forced Choice (n26),
Psychometric Function, Method of Constant
Stimuli 1-10 presentations 2. Tangent
Screen Method of limits Ascending (dynamic
fields) Descending (blind spot) 3. Pelli
Robson Chart Forced Choice (n26) Method of
Constant Stimuli (3 presentations) Criterion
correct (67)
38
4. Vis Tech Chart Forced Choice (n3) Method
of Constant Stimuli 1 presentation 5. Ishi
Hara Color Plates Forced Choice (n?) 1
presentation 6. Goldman-Weekers Dark
Adaptometer Ascending Method of Limits 7.
Van Graefe's phoria test Method of adjustment
matching 8. Humphry Perimeter Yes/no
staircase 9. Duochrome Test method of
adjustment matching 10. Randot Stereo test 3AFC
39
Psychophysics and Optometry part B (clinician
as subject, patient as stimulus)
1. Cover test Magnitude estimation 2.
Alternate cover test with prism nulling Method
of Adjustment 3. Subjective Refraction Method
of Limits (max plus min minus) 4.
Visuoscope Magnitude Estimation 5. Pupil
size method of adjustment matching
(pupil-ruler)n1 trial
40
6. Retinoscopy method of adjustment
nulling 7. Opthalmoscopy Disc
cupping magnitude estimation 8. Cataract
Grading magnitude estimation 9.
Keratometry method of adjustment alignment
10. Goldman Tonometer method of adjustment
alignment
41
Psychophysics and Optometry
Rationale It is as important to measure VA
accurately irrespective of the actual VA.
Designing a letter chart introduction to logMAR.
Strategy Keep constant (1) number of
letters/line, (2) step size between lines, (3)
spacing between letters
Goal Make VA chart as accurate at the top as
at the bottom.
Step size 0.1 log units (26 change in size),
line spacing 1 letter height of line above,
spacing 1 letter width (constant crowding)
42
Use TSD to determine value of tests for Glaucoma
Review of Sensitivity and specificity of a test.
Specificity proportion of those without disease
that pass test.
Sensitivity proportion of those with disease
that fail test.
Sensitivity
Specificity
Test score
Criterion score
pass
fail
Limitation Although sensitivity gives
proportion of those with disease that are
diagnosed as having disease, it does not give
proportion of those diagnosed who have the
disease. This latter statistic turns out to be
very important for most ophthalmic diseases.
43
Accuracy of screening test for rare diseases.
When the disease is rare, if specificity is not
very high, there will be a large number of
normals misdiagnosed as having the disease.
criterion
normals
Positive Predictability Rate Proportion of those
failing the test who actually have the disease.
Disease group
In this diagram the sensitivity and specificity
are both high (about 90), but as the normals
become more prevalent, the number of false
positives increases, and the Positive
predictability Rate declines from about 0.9 to
0.5 to 0.1 as the relative prevalence of disease
declines from 50, to 10 to 1.
That is, in a screening environment, where
disease prevalence might be 1, only 1 out of 10
positives actually has the disease even though
sensitivity and specificity are high (90).
44
Is there a good screening test for Glaucoma?
ROC Curves
Normals
Glaucoma
IOP
20
Phit (sensitivity)
23
Chance Performance (test is useless at detecting
disease)
24
28
Cup/Disk
PFA (1-specificity)
IOP
C/D
45
The cost-benefit analysis when setting criteria.
If costs of FA and Misses are the same, just
minimize total error by setting criterion at
intersection of distributions (for equally
likely conditions). However, if a miss cost 100X
more than a FA (e.g. miss a disease and the
patient goes blind, treat a FA and the patient is
inconvenienced), then use a lower criterion
(fewer misses, but more FA). To balance the
costs associated with both types of error, want
PFACFA PMISS CMISS
Criterion when CFA lt CMISS
Criterion when CFA CMISS
Strategy adjust criterion not to minimize
errors, but minimize cost.
normal
disease