The Subjective Experience of Feelings Past

1
Group analysis and mixed effects
2
Overview

• Mixed effects motivation
• Evaluating mixed effects methods
• Case Studies
• Summary statistic approach (HF)
• SnPM
• SPM2
• Conclusions

3
Overview

• Mixed effects motivation
• Evaluating mixed effects methods
• Case Studies
• Summary statistic approach (HF)
• SnPM
• SPM5
• Conclusions

4
Motivation
Functional MRI experiments are often repeated for
Several runs in the same session.
(i)
Several sessions on the same subject.
(ii)
(iii)
Several subjects drawn from a population.
5
Motivation
Conducting multi subject/session experiments may

• increase the sensitivity of the overall
  experiment (More data is available).

• allow for generalization of your conclusions
  about activation effects to a whole population of
  subjects.
• We very rarely make conclusions about individuals
  in science! Individual results may not be
  representative of others, and thus not useful in
  a broader scientific context.

6
Lexicon

• Hierarchical Models
• Mixed Effects Models
• Random Effects (RFX) Models in imaging lit.
• ... all the same
• ... model multiple sources of variation
  (Variance Components)
• in contrast to fixed effects models, with no
  components

7
Fixed and random effects

• Fixed effect
• Always the same, from experiment to experiment
  levels are not draws from a random variable
• Sex (M/F)
• Drug type (Prozac)

• Random effect
• Levels are randomly sampled from a population
• Subject
• Day, in longitudinal design
• Word, in experiments with verbal materials

• If effect is treated as fixed, error terms in
  model do not include variability across levels
• Cannot generalize to unobserved levels
• e.g., if subject is fixed, cannot generalize to
  new subjects

8
Subject as a fixed effect (wrong)
Animation over replications of one subjects
experiment

• Only variation is measurement error
• True response magnitude is fixed
• Significance based on estimated response relative
  to measurement error variance

9
Fixed Effects Implemented

• Grand GLM approach
• Model all subjects at once
• Assumptions (all likely violated)
• No individual differences
• Data points within each subject are independent
• Error variance is the same for all subjects

Contrast (1 1 1)
s1
s2
s3
10
Imaging Then and Now

• Early fMRI and PET studies tended to use the
  Super subject approach, ignoring that different
  images came from different subjects.
• Fixed effects analysis, b/c subject treated as
  fixed.
• Implications? Cannot generalize to population.
  I do not agree with attempts to argue that this
  is scientifically valid.

11
Why Subjects are Not Fixed Effects

• Imagine I make 1000 observations each on two
  subjects.
• I run a GLM, ignoring subject ID.
• Thus, I pretend this is the same as having made
  one observation each on 2000 people. I find a
  significant effect. (Sound fishy?)
• If I had really tested 2000 people, a significant
  effect would tell me something about whether a
  previously unobserved person is likely to show
  the effect on average.
• With only 2 people, my ability to say something
  about people in general is virtually nil. But
  if I dont tell the GLM that observations are
  nested within subjects, my stats are
  anticonservative.

12
Fixed vs. Random effects Bottom Line

• If I treat subject as a fixed effect, the error
  term reflects only scan-to-scan variability, and
  the degrees of freedom are determined by the
  number of observations (scans).
• If I treat subject as a random effect, the error
  term reflects the variability across subjects,
  which includes two parts
• Error due to scan-to-scan variability
• Error due to subject-to-subject variability
• and degrees of freedom are determined by the
  number of subjects.

13
Mixed-effects analysis
Assume the signal strength varies across sessions
and subjects.
There are two sources of variation
Measurement error
(i)
Response magnitude - Each subject/session has a
random magnitude.
(ii)
The population mean is fixed.
14
Subject as a random effect
Animation over different subjects

• Two sources of variation
• Measurement error (scan-to-scan var.)
• Individual differences (subj.-to-subj. var.)

15
Random/Mixed Effects
Animation over different subjects

• Response magnitude is random
• Each subject has a different magnitude

16
Comments
Mixed-effects models take into consideration the
fact that the individual subjects are sampled
from the population and thus are random
quantities with associated variances.
Mixed-effects models make fewer assumptions about
the data and the results are valid for
the population from which the subjects are drawn.
However, they also tend be more conservative.
17
Fixed vs.RandomEffects in fMRI
Distribution of each subjects estimated effect
?2FFX
Subj. 1
Subj. 2

• Fixed Effects
• Intra-subject variation suggests all these
  subjects different from zero
• Random Effects
• Intersubject variation suggests population not
  very different from zero

Subj. 3
Subj. 4
Subj. 5
Subj. 6
0
?2RFX
Distribution of population effect
18
Overview

• Mixed effects motivation
• Mixed effects issues
• Evaluating three approaches
• Summary statistic approach (HF)
• SnPM
• SPM5
• Conclusions

19
Assessing RFX ModelsIssues to Consider

• Assumptions Limitations
• What must I assume?
• When can I use it
• Efficiency Power
• How sensitive is it?
• Validity Robustness
• Can I trust the P-values?
• Are the standard errors correct?
• If assumptions off, things still OK?

20
Issues Assumptions

• Distributional Assumptions
• Gaussian? Nonparametric?
• Homogeneous Variance
• Over subjects?
• Over conditions?
• Independence
• Across subjects?
• Across conditions/repeated measures
• Note
• Nonsphericity (Heterogeneous Var) or
  (Dependence)

21
Issues Soft AssumptionsRegularization

• Regularization
• Weakened homogeneity assumption
• Usually variance/autocorrelation regularized over
  space
• Examples
• fmristat - local pooling (smoothing) of
  (?2RFX)/(?2FFX)
• SnPM - local pooling (smoothing) of ?2RFX
• FSL3 - Bayesian (noninformative) prior on ?2RFX

22
Issues Efficiency Power

• Efficiency 1/(Estimator Variance)
• Goes up with n
• Power Chance of detecting effect
• Goes up with n
• Also goes up with degrees of freedom (DF)
• DF accounts for uncertainty in estimate of ?2RFX
• Usually DF and n yoked, e.g. DF n-p

23
Issues Validity

• Are P-values accurate?
• I reject my null when P lt 0.05Is my risk of
  false positives controlled at 5?
• Exact control
• FPR a
• Valid control (possibly conservative)
• FPR ? a
• Problems when
• Standard Errors inaccurate
• Degrees of freedom inaccurate

24
Four RFX Approaches in fMRI

• Holmes Friston (HF)
• Summary Statistic approach (contrasts only)
• Holmes Friston (HBM 1998). Generalisability,
  Random Effects Population Inference. NI, 7(4
  (2/3))S754, 1999.
• Holmes et al. (SnPM)
• Permutation inference on summary statistics
• Nichols Holmes (2001). Nonparametric
  Permutation Tests for Functional Neuroimaging A
  Primer with Examples. HBM, 151-25.
• Holmes, Blair, Watson Ford (1996).
  Nonparametric Analysis of Statistic Images from
  Functional Mapping Experiments. JCBFM, 167-22.
• Friston et al. (SPM5)
• Empirical Bayesian approach
• Friston et al. Classical and Bayesian inference
  in neuroimagining theory. NI 16(2)465-483,
  2002
• Friston et al. Classical and Bayesian inference
  in neuroimaging variance component estimation
  in fMRI. NI 16(2)484-512, 2002.
• Beckmann et al. Woolrich et al. (FSL3)
• Summary Statistics (contrast estimates and
  variance)
• Beckmann, Jenkinson Smith. General Multilevel
  linear modeling for group analysis in fMRI. NI
  20(2)1052-1063 (2003)
• Woolrich, Behrens et al. Multilevel linear
  modeling for fMRI group analysis using Bayesian
  inference. NI 211732-1747 (2004)

25
Overview

• Mixed effects motivation
• Mixed effects issues
• Evaluating three approaches
• Summary statistic approach (HF)
• SnPM
• SPM5
• Conclusions

26
Summary Statistics (contrasts only)
Second level
First level
Data Design Matrix Contrast Images
SPM(t)
One-sample t-test _at_ 2nd level
27
Case Studies HFAssumptions

• Distribution
• Normality
• Independent subjects
• Homogeneous Variance
• Intrasubject variance homogeneous
• ?2FFX same for all subjects
• Balanced designs
• Efficiency of design to detect effect same for
  all Ss

28
Case Studies HF Efficiency

• If assumptions true
• Optimal, fully efficient
• If ?2FFX differs between subjects
• Reduced efficiency
• Here, optimal requires down-weighting the 3
  highly variable subjects

0
29
Case Studies HF Validity

• If assumptions true
• Exact P-values
• If ?2FFX differs btw subj.
• Standard errors OK
• Est. of ?2RFX unbiased
• DF not OK
• Here, 3 Ss dominate
• DF lt 5 6-1
• Violation of equality of variance
• Solution would require, e.g., weighted least
  squares at 2nd level based on variance estimates

0
?2RFX
30
Case Studies HFRobustness

• Heterogeneity of ?2FFX across subjects...
• How bad is bad?
• Dramatic imbalance (rough rules
  of thumb only!)
• Some subjects missing 1/2 or more sessions
• Measured covariates of interest having
  dramatically different efficiency
• E.g. Split event related predictor by
  correct/incorrect
• One subj 5 trials correct, other subj 80 trials
  correct
• Dramatic heteroscedasticity
• A bad subject, e.g. head movement, spike
  artifacts

31
Overview

• Mixed effects motivation
• Mixed effects issues
• Evaluating three approaches
• Summary statistic approach (HF)
• SnPM
• SPM5
• Conclusions

32
Case Study SPM5

• 1 effect per subject
• Uses Holmes Friston approach
• gt1 effect per subject
• SPM2 and higher Variance basis function approach
  used...
• The good news You can test contrasts using
  multiple basis functions at the 2nd level
• The bad news
• SPM Pools variance estimates across brain, bad if
  variance is not homogenous
• Validity of inference depends on accurate
  estimation of DF!
• Likely typical results somewhere between fixed
  and random effects.

33
SPM5 Repeated-Measures ANOVA
y X ? e N ? 1 N ? p p ? 1
N ? 1
F-contrast for diffs among conditions

• 12 subjects,4 conditions
• Use F-test to find differences btw conditions
• Standard Assumptions
• Identical distribution
• Independence
• If all 48 images were independent, would have 48
  - 4 44 degrees of freedom.
• But theyre not because they come from the same
  subjects!

2nd level design matrix
X
Condition 1
Condition 4
34
Sphericity IID Case
Image of spherical Cov matrix
sphericity
Scans

• Violations of sphericity
• Unequal variances
• Autocorrelation
• Correlation due to repeated measures (same
  subjects)

Scans
35
2nd level Non-sphericity
Error covariance
Errors are independent but not identical
Errors are not independent and not identical
36
SPM5 Repeated measures ANOVAcovariance matrix
y X ? e N ? 1 N ? p p ? 1
N ? 1
Error covariance

• 12 subjects, 4 conditions
• Measurements btw subjects uncorrelated
• Measurements w/in subjects correlated

N
N
Errors can now have different variances and
there can be correlations
Allows for nonsphericity
37
Mixed-effects (theoretical only in SPM5?)
Step 1 first level
Summary statistics
Step 2 Group
EM approach
w/i subjects part
btwn part
Friston et al. (2004) Mixed effects and fMRI
studies, Neuroimage
38
Some practical points (from Will Penny)
RFX If not using multi-dimensional contrasts at
2nd level (F-tests), use a series of 1-sample
t-tests at the 2nd level.
Use mixed-effects model only, if seriously in
doubt about validity of summary statistics
approach.
If using variance components at 2nd level
Always check your variance components (explore
design).
39
When to take multiple images to 2nd level

• When you have to
• When comparing conditions modeled with multiple
  basis functions

40
Case Study SPM5

• Assumptions Limitations
• assumed to
  globallyhomogeneous
• lks only estimated from voxels with large F
• Most realistically, Cor(e) spatially
  heterogeneous
• Intrasubject variance assumed homogeneous

41
Case Study SPM5

• Efficiency Power
• If assumptions true, fully efficient
• Validity Robustness
• P-values could be wrong (over or under) if local
  Cor(e) very different from globally assumed
• Stronger assumptions than Holmes Friston

42
Robust regression at 2nd level

• Potential way to deal with heteroscedastic
  variances
• High-variance observations tend to dominate if
  their observed values are extreme
• When assumptions cannot be checked at each voxel,
  automatic procedures for weighting based on
  outlier status advantageous

43
Group analysis with basis functions at 1st
levelDealing with mutiple parameter estimates
per event type
44
Example Famous vs. non-famous faces

• We want to use HRF temporal derivatives to
  model each event type
• Interested in A B difference (famous non)

• So we have 6 images, 3 for each subject, that
  capture differences between event types A and B!

45
Review Design matrix with basis sets
46
Issues with using multiple parameters to estimate
responses for each trial type

• Cant summarize the response magnitude with a
  single number
• How to combine them?
• Use just the main one?
• Enter multiple contrast images (one for each
  basis function) in 2nd level group analysis in
  SPM
• 3) Re-parameterize fitted responses in terms of
  magnitude, delay, etc. (e.g., Calhoun 2003,
  Lindquist Wager, 2007, 2008/in press)

47
Issues with using multiple parameters to estimate
responses for each trial type

• Cant summarize the response magnitude with a
  single number
• How to combine them?
• Use just the main one? No, because fitted
  response is a combination of parameters
• Enter multiple contrast images (one for each
  basis function) in 2nd level group analysis in
  SPM. Requires additional assumptions about
  homogeneity of nonsphericity, degrees of freedom
• 3) Re-parameterize fitted responses in terms of
  magnitude, delay, etc. I like this.

48
Proving that you cant just ignore some basis
functions An example with HRFderivatives

• Does the canonical (blue) function always capture
  the main response?
• Not necessarily! The response magnitude is a
  combo of ALL THREE basis functions.
• Can you guess the height of the canonical
  parameter estimate in the Fitted HRF below?

Fitted
49
Proving that you cant just ignore some basis
functions An example with HRFderivatives

• Does the canonical (blue) function always capture
  the main response?
• Not necessarily! The response magnitude is a
  combo of ALL THREE basis functions.
• Can you guess the height of the canonical
  parameter estimate in the Fitted HRF below?

Fitted

0
10
3
a1
a3
a2
50
Proving that you cant just ignore some basis
functions An example with HRFderivatives
Data
Fitted
1
2
3

• The graph above looks like it shows an A B
  difference (blue red), right? Does that mean a1
  gt b1, a2 gt b2, a3 gt b3?
• Consider trial A - B, or a1 a2 a3 - b1 b2 b3
• Difference between amplitudes of fitted
  responses 0.84
• Difference between canonical HRF betas 0.43
• Amplitude of the difference does NOT equal
  difference between canonical param. estimates
  (a1 - b1)

51
Solutions Re-parameterizing
52
Example Emotional resilience

• Waugh et al., in press

53
Overview

• Mixed effects motivation
• Evaluating mixed effects methods
• Case Studies
• Summary statistic approach (HF)
• SnPM
• SPM5
• Conclusions

54
Conclusions

• Random Effects crucial for pop. inference
• SPM2 / SPM5
• Most RFX analyses proceed just like SPM99
• But nonsphericity modeling can handle multiple
  contrasts per subject
• FSL
• Accounts for different intrasubject var.
• More valid, may be more sensitive
• Usually will be similar to SPM results

55
End of this section.