Tor D' Wager

1
Experimental Design

• Tor D. Wager
• Columbia University

2
Types of designs Blocked and event-related

• Block design Similar events are grouped

• Event-related design Events are mixed

3
Types of designs Mixed block/ER
4
Birds eye view

• Goal
• Induce human subject to do or experience the
  psychological states youre studying
• Effectively detect brain signals related to those
  psychological states
• Design method
• You control what to present and when
• Two kinds of considerations Psychological and
  statistical

5
Psychological considerations

• Does the task induce the subject to think the way
  you want?
• Stimulus predictability
• Time on task
• Participant strategy
• Temporal precision of psychological manipulations
• Unintended psychological activity

6
Psychological considerations

• Does the task induce the subject to think the way
  you want?
• Stimulus predictability
• Time on task
• Participant strategy
• Temporal precision of psychological manipulations
• Unintended psychological activity

7
Predictability

• Predictability influences psychological state

• Example Go-no go task
• press fast, but withhold if X

N
P
R
S
X
X
X
X
N
P
X
X
R
Q
T
X

• The predictability of the no-go stimulus
  determines how hard it is to not respond

8
Psychological considerations

• Does the task induce the subject to think the way
  you want?
• Stimulus predictability
• Time on task
• Participant strategy
• Temporal precision of psychological manipulations
• Unintended psychological activity

9
Time on task

• You can only image what subjects are doing, so
  they should be doing what you want as much of the
  time as possible.
• Example Famous face recognition
• Fast, perhaps automatic process

10
Time on task

• You can only image what subjects are doing, so
  they should be doing what you want as much of the
  time as possible.
• Example Famous face recognition
• Fast, perhaps automatic process

11
Time on task

• You can only image what subjects are doing, so
  they should be doing what you want as much of the
  time as possible.
• Example Famous face recognition
• Fast, perhaps automatic process

Sequence
Time
5 seconds
12
Time on task
Recog 250 ms
What was he in?
Get back on task
13
Psychological considerations

• Does the task induce the subject to think the way
  you want?
• Stimulus predictability
• Time on task
• Participant strategy
• Temporal precision of psychological manipulations
• Unintended psychological activity

14
Participant Strategy

• Different stim. configurations afford different
  strategies
• Example Stroop task

• Compatible green yellow red blue
• Incompatible red blue green yellow

Predictable blue red green red yellow blue
yellow red
Unpredictable red blue red blue red green blue
yellow yellow
If compatible stimuli are predictable, then task
is different Predictable compatible stimuli
induce READING Unpredictable compatible stimuli
induce COLOR NAMING
15
Psychological considerations

• Does the task induce the subject to think the way
  you want?
• Stimulus predictability
• Time on task
• Participant strategy
• Temporal precision of psychological manipulations
• Unintended psychological activity

16
Precision of psychological manipulation

• What you expect from subjects should fit with
  what subjects can do.
• Example Imaging Emotions
• Compare Recall sad memories vs. recall happy
  memories
• A typical fMRI block design will not work
  Subjects cannot switch back and forth among
  emotions
• Feeling states take longer to achieve
• And they dont go away when you want to turn them
  off

17
Psychological considerations

• Does the task induce the subject to think the way
  you want?
• Stimulus predictability
• Time on task
• Participant strategy
• Temporal precision of psychological manipulations
• Unintended psychological activity

18
Unintended psychological activity

• Subjects brains are responding as theyre doing
  things you didnt tell them to do.
• Example Spatial attention shifting

Compare Switch vs. Stay Sequence Stay - Stay -
Switch - Stay - Stay - Switch - Stay
Predicted sequence
But subjects shift attention to the fixation
cross spontaneously!
Likely true sequence
Your analysis is based on the predicted sequence,
but brain activity follows the true sequence!
19
Birds eye view

• Goal
• Induce human subject to do or experience the
  psychological states youre studying
• Effectively detect brain signals related to those
  psychological states
• Design method
• You control what to present and when
• Two kinds of considerations Psychological and
  statistical

20
Statistical considerations

• Maximize task-induced changes
• Equal numbers of stimuli in each condition (if
  possible)
• Minimize correlations among predictors of
  interest
• Think about analysis needs and inference
• Want robustness? Want design/analysis simplicity?
  Try a block design
• Intd in parts of trials, trial-to-trial
  relationships with performance, or specific
  events? An event-related design may be good

21
Maximize variance of predictors

• Maximize variation along x-axis of the plots
  below (Rescaling doesnt count)

• Principles
• Keep equal numbers in high and low predicted
  groups
• Concentrate on extremes

22
Minimize covariance among predictors

• Avoid confounds and partial confounds

23
HRFs vary across regions
Checkerboard, n 10
Thermal pain, n 23

• HRF shape depends on
• vasculature
• time course of neural activity

Stimulus On
Aversive picture, n 30
Aversive anticipation
See Schacter et al. Aguirre et al.
24
HRF mismatch in blocked and ER designs

• What happens when the true HRF does not match the
  assumed one?
• Simple case mis-specification of onsets

25
HRF mismatch in blocked and ER designs
26
Principles for event-related designs

• Include a rest condition or jittered
  inter-trial intervals (ITIs) only if you care
  about comparing activity to rest or recovering
  HRF shapes for all trial types
• Avoid ultra-rapid designs if possible (4 sec
  ITIs) Signal nonlinearity (Miezin et al., 2005
  Wager et al. 2005 Birn Huettel)
• If youre interested in separating events that
  always occur in a sequence, consider catch
  trials (Ollinger, 2001)

27
Trial spacing and jitter
For A - B About 5 s, on average, between reps of
the same event, no rest
1 rest events41 rest events81 rest events
Efficiency of contrast 1 -1
For A B About 16-20 s, on average, between
events of same type (i.e., use jitter)
2
4
6
10
14
Wager Nichols, 2003
ISI in sec
Efficiency with nonlinear saturation
28
fMRI data nonlinear at short ISIs

• Meizin et al. (2000) 10 nonlinear saturation at
  5 s ISI

• Series of 1,2,5,6,10 or 11 events. Each event
  125 ms flashing checkerboard, 1s ISI. Series
  followed by 30s rest
• Note actual vs. predicted relative magnitude

Wager et al., 2005
29
Nonlinearity in BOLD signal
30
Catch trials
Delay
Probes
Memory Set
Remember IDLE
Yes/No? VARY

AFFIRM
Yes/No? SEVER
CYNIC
TOPIC
Yes/No? OPAL
ABHOR
VARY
Yes/No? OBEY
500 ms
OBEY
WISDOM
1500 ms
CRISIS
8000 ms
3500-5500 ms
6000 ms
Time
31
Catch trials
Memory Set
Trial end signal
Memory Set
Remember IDLE

AFFIRM
CYNIC
Remember IDLE
TOPIC
AFFIRM
ABHOR
CYNIC
VARY
TOPIC
OBEY
ABHOR
WISDOM
VARY
CRISIS
OBEY
WISDOM
CRISIS
3500-5500 ms
Time
32
Example Anticipatory engagement of cognitive
control

• ER fMRI, N16
• Ps respond to position or meaning (W) of words
• (up, down, left, right)
• Cues are informative (P/W) or not (N)
• 50 catch trials to separate task-set preparation
  from response selection
• Monetary payoff schedule designed to ensure that
  people are motivated to actively prepare during
  cue period

Response UP
Informative trials (P or W)
6 s
Response UP
Non-informative trials (N/P or N/W)
6 s
Catch trials (P or W)
(no response)
Stern et al., 2007
33
Cueing effects on performance
34
Expectancy activates the superior attention
network
Stern et al., 2007
35
Extra stuff

• Download the Genetic Algorithm toolbox at
• http//www.columbia.edu/cu/psychology/tor/

36
Interpretability

• Block designs do not inform about whether
  activity is related to specific psychological
  events
• Case study Face recognition compare
  Famous-Nonfamous

Recog 250 ms
What was he in?
Get back on task
37
Basis sets
Time (s)
38
Design efficiency in fMRI

• Formula is more complex, principle is the same
• Factor in contrasts, filtering and
  autocorrelation
• Define filtering matrix K, autocorrelation
  matrix V
• Matrix whose rows contain a set of contrasts C
• filtered design matrix Z
• Z- pseudoinverse of Z inv(ZZ)Z

• Not equal to power, but can be converted to power
  given effect sizes

See Friston et al., 2000 Zarahn, 2001
39
Pros and cons of blocking

• High power, if parameters chosen correctly
• Simple to implement
• Relatively robust to changes in HRF shape
• - Predictable events may change task strategy and
  activity patterns
• - Cannot infer activity related to specific
  psychological events
• - Power limited if Ss are not doing cognitive
  operation of interest throughout blocks

40
High-pass filtering
Frequency domain
fMRI Noise Time domain
Unfiltered
Filtered