Mediation and moderation of treatment effects Andrew Pickles

1
Methods of explanatory analysis for psychological
treatment trials workshop

• SESSION 2
• Mediation and moderation of treatment effects
  Andrew Pickles

Funded by MRC Methodology Grant G0600555 MHRN
Methodology Research Group
2
Moderators Mediators

• Moderator is a variable that modifies the form
  or strength of the relation between an
  independent and a dependent variable.
• Mediator is a variable that is intermediate
  in the causal sequence relating an independent
  variable to a dependent variable.

3
Moderators in RCTs

• Moderators are baseline characteristics that
  influence the effect of treatment, or the effect
  of treatment allocation (on intermediate or
  final outcomes).
• They are pre-randomisation effect- modifiers.
• Examples sex, age, previous history of
  mental illness, insight, treatment centre,
  therapist characteristics, genes etc.

4
Typical local example
Figure 2. SF36 scores by abuse categories at
baseline and follow-up (treated patients only)
Creed et al., Psychosomatic Medicine 67490499
(2005)
5
Testing for Moderation

• A moderator variable is typically a baseline
  variable (e.g. not-abused, abused)
• Makes treatment effect greater in one group than
  another (moderator may or may not have an
  additional direct effect on outcome). It is a
  source of treatment effect heterogeneity
• A classic error is to claim moderation when
  treatment effect is significant effect in one
  group and not significant in another. Is simply a
  recipe for increasing Type I (false positive)
  error rate

6
Interaction Synergy

• Need to show significant interaction with
  treatment on outcome
• But on what scale?
• Can find that interaction significant on one
  scale but is not significant if outcome variable
  is transformed. Choice of scale requires both
  statistical and clinical considerations.
• If outcome binary then usual test is for
  interaction on the log-odds scale
• Some argue that main effects on log-odds scale
  already suggests synergy
• e.g. if the base outcome rate is low and the
  treatment and moderator each increase outcome by
  100 then the two together increase the outcome
  rate not by 200 but by 300 even without an
  interaction

7
The SoCRATES Trial

• SoCRATES was a multi-centre RCT designed to
  evaluate the effects of cognitive behaviour
  therapy (CBT) and supportive counselling (SC) on
  the outcomes of an early episode of
  schizophrenia.
• Participants were allocated to one of three
  conditions
• Analysed as two conditions
• Control condition Treatment as Usual (TAU),
• Treatment condition TAU plus psychological,
  either CBT TAU or SC TAU.

8
SoCRATES (contd.)

• 3 treatment centres Liverpool, Manchester and
  Nottinghamshire. Other baseline covariates
  include logarithm of untreated psychosis and
  years of education.
• Outcome (a psychotic symptoms score) was obtained
  using the Positive and Negative Syndromes
  Schedule (PANSS).
• From an ITT analyses of 18 month follow-up data,
  both psychological treatment groups had a
  superior outcome in terms of symptoms (as
  measured using the PANSS) compared to the control
  group.

9
SoCRATES (contd.)

• Post-randomization variables that have a
  potential explanatory role in exploring the
  therapeutic effects include the total number of
  sessions of therapy actually attended and the
  quality or strength of the therapeutic alliance.
• Therapeutic alliance was measured at the 4th
  session of therapy, early in the time-course of
  the intervention, but not too early to assess the
  development of the relationship between therapist
  and patient. We use a patient rating of alliance
  based on the CALPAS (California Therapeutic
  Alliance Scale).
• Total CALPAS scores (ranging from 0, indicating
  low alliance, to 7, indicating high alliance)
  were used in some of the analyses reported below,
  but we also use a binary alliance variable (1 if
  CALPAS score 5, otherwise 0).

.
10
SoCRATES - Summary Statistics
Lewis et al, BJP (2002) Tarrier et al, BJP
(2004) Dunn Bentall, Stats in Medicine (2007).
11
Socrates positive symptomsbasic analysis

• xi regress enpstot psubtota rgrp
• Source SS df MS
  Number of obs 225
• -------------------------------------------
  F( 2, 222) 14.80
• Model 792.779676 2 396.389838
  Prob gt F 0.0000
• Residual 5945.22032 222 26.7802717
  R-squared 0.1177
• -------------------------------------------
  Adj R-squared 0.1097
• Total 6738 224 30.0803571
  Root MSE 5.175
• --------------------------------------------------
  ----------------------------
• enpstot Coef. Std. Err. t
  Pgtt 95 Conf. Interval
• -------------------------------------------------
  ----------------------------
• psubtota .34999 .0785569 4.46
  0.000 .1951774 .5048026
• rgrp -2.240193 .7425587 -3.02
  0.003 -3.703559 -.7768275
• _cons 6.986856 1.954491 3.57
  0.000 3.135127 10.83859
• --------------------------------------------------
  ----------------------------

12
Socrates positive symptomsincluding main
effects of centre

• xi regress enpstot psubtota i.centre rgrp
• --------------------------------------------------
  ----------------------------
• enpstot Coef. Std. Err. t
  Pgtt 95 Conf. Interval
• -------------------------------------------------
  ----------------------------
• psubtota .1710413 .0847491 2.02
  0.045 .0040172 .3380653
• _Icentre_2 1.679312 .8588526 1.96
  0.052 -.0133193 3.371944
• _Icentre_3 -2.857869 .823287 -3.47
  0.001 -4.480408 -1.235331
• rgrp -2.158757 .7039854 -3.07
  0.002 -3.546176 -.7713389
• _cons 11.42025 2.038804 5.60
  0.000 7.402161 15.43833
• --------------------------------------------------
  ----------------------------
• testparm _Icen
• ( 1) _Icentre_2 0
• ( 2) _Icentre_3 0
• F( 2, 220) 13.56
• Prob gt F 0.0000

13
Socrates positive symptomstreatment effect by
centre

• xixi bysort centre regress enpstot psubtota
  rgrp
• -gt centre 1
• --------------------------------------------------
  ----------------------------
• enpstot Coef. Std. Err. t
  Pgtt 95 Conf. Interval
• -------------------------------------------------
  ----------------------------
• psubtota .1686252 .1880974 0.90
  0.373 -.2066189 .5438693
• rgrp -3.439661 1.348812 -2.55
  0.013 -6.130467 -.7488547
• _cons 12.34583 4.501713 2.74
  0.008 3.365161 21.3265
• --------------------------------------------------
  ----------------------------
• -gt centre 2
• --------------------------------------------------
  ----------------------------
• enpstot Coef. Std. Err. t
  Pgtt 95 Conf. Interval
• -------------------------------------------------
  ----------------------------
• psubtota .0768268 .1548842 0.50
  0.621 -.2314624 .385116
• rgrp -1.785964 1.448293 -1.23
  0.221 -4.668719 1.096791
• _cons 15.31862 4.007697 3.82
  0.000 7.341504 23.29575
• --------------------------------------------------
  ----------------------------
• -gt centre 3

14
Socrates positive symptomsmoderation/heterogenei
ty?

• xi regress enpstot psubtota i.centrergrp
• --------------------------------------------------
  ----------------------------
• enpstot Coef. Std. Err. t
  Pgtt 95 Conf. Interval
• -------------------------------------------------
  ----------------------------
• psubtota .1685492 .0866722 1.94
  0.053 -.0022736 .339372
• _Icentre_2 .6983508 1.398679 0.50
  0.618 -2.058313 3.455015
• _Icentre_3 -4.532945 1.481842 -3.06
  0.002 -7.453517 -1.612374
• rgrp -3.439764 1.245653 -2.76
  0.006 -5.894829 -.9846987
• _IcenXrgrp_2 1.458311 1.720016 0.85
  0.397 -1.931679 4.848301
• _IcenXrgrp_3 2.418837 1.779205 1.36
  0.175 -1.087808 5.925483
• _cons 12.3476 2.257408 5.47
  0.000 7.898459 16.79674
• --------------------------------------------------
  ----------------------------
• testparm _IcenX
• ( 1) _IcenXrgrp_2 0
• ( 2) _IcenXrgrp_3 0

15
Mediators in Randomised Clinical Trials (RCTs)

• Mediators are intermediate outcomes on the causal
  pathway between allocation to or receipt of
  treatment and final outcome.
• By definition, in an RCT, they are measured after
  randomisation.
• Treatment effect may be fully or partially
  explained by a given mediator. Possible for a
  given mediator to serve the role of surrogate
  outcome.
• Possibility of multiple mediators (multiple
  pathways) and interactions between mediators.

16
Post-randomisation effect modifiers

• Intermediate outcomes that influence either (a)
  the effects of treatment/treatment allocation on
  other intermediate outcomes (mediators) or (b)
  the effects of the other intermediate outcomes on
  the final outcome.
• Candidates amount of treatment (sessions
  attended), treatment fidelity, therapeutic
  alliance.
• Distinction between these variables and mediators
  not obvious.

17
Examples

• Compliance with allocated treatment
• Does the participant turn up for any therapy?
• How many sessions does she attend?
• Fidelity of therapy
• How close is the therapy to that described in
  the treatment manual? Is it a cognitive-behavioura
  l intervention, for example, or merely emotional
  support?
• Quality of the therapeutic relationship
• What is the strength of the therapeutic alliance?

18
Examples (cont.)

• What is the concomitant medication?
• Does psychotherapy improve compliance with
  medication which, in turn, leads to better
  outcome? What is the direct effect of
  psychotherapy?
• What is the concomitant substance abuse?
• Does psychotherapy reduce cannabis use, which in
  turn leads to improvements in psychotic symptoms?
• What are the participants beliefs?
• Does psychotherapy change attributions
  (beliefs), which, in turn, lead to better
  outcome? How much of the treatment effect is
  explained by changes in attributions?

19
The Mediation Industry

• Baron RM Kenny DA (1986). The
  moderator-mediator variable distinction in social
  psychological research conceptual, strategic,
  and statistical considerations. Journal of
  Personality and Social Psychology 51, 1173-1182.
• As of 16th September 2009 12,292 citations!
• Assumptions are very rarely stated, let alone
  their validity discussed.
• One suspects that the majority of investigators
  are oblivious of the implications.

20
A Naïve Look at mediation the BK framework
Randomised to Psych treatment Independent X
c
a
Psychotic Symptoms Dependent Y
Number of sessions Mediator M
e3
e2
b
Regression eqns used to assess mediation Yd1cX
e1 Yd2cXbMe2 Md3aXe3 total effectc
mediated effect ab or (c-c) (in simple linear
models these should be equal if
estimated on same
sample)
21
Testing for Mediation

• Estimate of mediated effect
• Confidence interval /- 1.96seab
• Estimate of seab sqrt( seb2
  sea2)
• Bootstrap resampling better (allows for
  asymmetry)
• Test of mediation (1) if 0 within CI
• (2) z-test for /seab

22
Baron Kenny Steps naïve mediation

• Effect of X on Y (c) must be significant
• Effect of X on M (a) must be significant
• Effect of M on X (b) must be significant
• When controlling for M, the direct effect of X on
  Y (c) must be non-significant

23
naïve mediation

• xiregress nosess rgrp psubtota i.centre
• --------------------------------------------------
  ----------------------------
• nosess Coef. Std. Err. t
  Pgtt 95 Conf. Interval
• -------------------------------------------------
  ----------------------------
• rgrp 13.82383 .5893788 23.45
  0.000 12.66366 14.98401
• psubtota .1047549 .0649339 1.61
  0.108 -.0230656 .2325754
• _Icentre_2 -1.387014 .7189374 -1.93
  0.055 -2.802223 .0281941
• _Icentre_3 -2.87773 .7188629 -4.00
  0.000 -4.292792 -1.462668
• _cons -1.210907 1.551379 -0.78
  0.436 -4.264754 1.84294
• --------------------------------------------------
  ----------------------------

24
naïve mediation

• xiregress enpstot nosess rgrp psubtota i.centre
• --------------------------------------------------
  ----------------------------
• enpstot Coef. Std. Err. t
  Pgtt 95 Conf. Interval
• -------------------------------------------------
  ----------------------------
• nosess .0417879 .0795974 0.52
  0.600 -.1151377 .1987135
• rgrp -2.81782 1.345742 -2.09
  0.037 -5.470936 -.1647028
• psubtota .1606631 .0871823 1.84
  0.067 -.011216 .3325421
• _Icentre_2 1.926335 .9083031 2.12
  0.035 .1356243 3.717046
• _Icentre_3 -2.54384 .9285473 -2.74
  0.007 -4.374462 -.7132184
• _cons 11.47856 2.103109 5.46
  0.000 7.332299 15.62482
• --------------------------------------------------
  ----------------------------

A13.80 (0.59) , B0.042 (0.08) A times B
0.58 (1.10) Sobel estimate of standard error
sqrt(13.820.0820.04220.592)1.10
25
Stata code for naïve mediation bootstrap 1

• global model1 nosess rgrp psubtota
  i.centre"global model2 enpstota nosess rgrp
  psubtota i.centre"program mediate,
  rclassversion 8xiregress model1matrix
  ae(b)xiregress model2matrix be(b)return
  scalar mediatea1,1b1,1endbootstrap
  mediate productr(mediate), reps(100) dots

26
Stata code for naïve mediation bootstrap 2

• bootstrap mediate productr(mediate), reps(100)
  dots
• command mediate
• statistic product r(mediate)
• ..................................................
  ..................................................
  
• Bootstrap statistics
  Number of obs 213
• 
  Replications 100
• --------------------------------------------------
  ----------------------------
• Variable Reps Observed Bias Std.
  Err. 95 Conf. Interval
• -------------------------------------------------
  ----------------------------
• product 100 .5776688 -.1432935
  1.057901 -1.521436 2.676773 (N)
• 
  -1.682636 2.333766 (P)
• 
  -1.682636 2.333766 (BC)
• --------------------------------------------------
  ----------------------------
• Note N normal
• P percentile

27
Mediation and measurement error mediation or
direct and indirect effects in SEM (Mplus)

• Testing for and estimating mediation can be
  susceptible to measurement error bias

28
Direct and Indirect Longitudinal and sleeper
effects

• y1 directly influences y2 through path a
• y1 only indirectly influences y3 through y2 on
  paths a and b
• In a longitudinal study if y1 influences y3
  directly (i.e. not through y2) this is a sleeper
  effect
• This structure of restricting effects to those
  from the previous occasion is known as first
  order autorgression (AR1)

29
Longitudinal Ability Data correct at ages 6,7,
9 and 11

• STANDARD DEVIATIONS
• 6.374 7.319 7.796 10.386
• CORRELATION MATRIX
• 1
• 0.809 1
• 0.806 0.850 1
• 0.765 0.831 0.867 1

30
AR1 Model ability1.inp

• TITLE Ability autoregressive model
• DATA FILE IS D\courses\mplus\ability.dat
• TYPE IS CORRELATION STDEVIATIONS
• NOBSERVATIONS204
• VARIABLE NAMES ARE y1-y4
• USEVARIABLES ARE y1-y4
• MODEL y2 on y1
• y3 on y2
• y4 on y3
• OUTPUT SAMPSTAT STANDARDIZED RESIDUAL

31
Indirect effects Ability1b.inp
TITLE Ability latent autoregressive
model DATA FILE IS D\courses\mplus\ability.d
at TYPE IS CORRELATION STDEVIATIONS
NOBSERVATIONS204 VARIABLE NAMES ARE
y1-y4 USEVARIABLES ARE y1-y4 MODEL
y2 on y1 y3 on y2 y4 on
y3 MODEL INDIRECT y4 IND y1 y3 IND
y1 OUTPUT STANDARDIZED CINTERVAL
32
AR1 Model Output-1
Effects from Y1 to Y4 Total
0.971 0.076 12.814 0.971 0.596
Total indirect 0.971 0.076 12.814
0.971 0.596 Specific indirect Y4
Y3 Y2 Y1 0.971 0.076
12.814 0.971 0.596 Effects from Y1 to
Y3 Total 0.841 0.056
14.956 0.841 0.688 Total indirect
0.841 0.056 14.956 0.841 0.688
Specific indirect Y3 Y2 Y1
0.841 0.056 14.956 0.841 0.688
33
Autoregressive Output-1
Chi-Square Test of Model Fit Value
62.124 ! This fits
Degrees of Freedom 3
! Very badly P-Value
0.0000 ESTIMATED MODEL AND RESIDUALS
(OBSERVED - ESTIMATED) Model Estimated
Covariances/Correlations/Residual Correlations
Y2 Y3 Y4
Y1 ________ ________
________ ________ Y2 53.305 Y3
48.263 60.481 Y4
55.745 69.857 107.341 Y1
37.556 34.003 39.275
40.429 Residuals for
Covariances/Correlations/Residual Correlations
Y2 Y3 Y4
Y1 ________ ________
________ ________ Y2 0.000 Y3
-0.001 -0.001 Y4
7.114 -0.001 -0.001 Y1
0.000 5.852 11.120 0.000
34
Autoregressive Output-2

• TOTAL, TOTAL INDIRECT, SPECIFIC INDIRECT, AND
  DIRECT EFFECTS
• Estimates S.E. Est./S.E.
  Std StdYX
• Effects from Y1 to Y4
• Total 0.971 0.076 12.814
  0.971 0.596
• Total indirect 0.971 0.076 12.814
  0.971 0.596
• Specific indirect
• Y4
• Y3
• Y2
• Y1 0.971 0.076 12.814
  0.971 0.596
• Effects from Y1 to Y3
• Total 0.841 0.056 14.956
  0.841 0.688
• Total indirect 0.841 0.056 14.956
  0.841 0.688
• Specific indirect
• Y3
• Y2
• Y1 0.841 0.056 14.956
  0.841 0.688

35
Simplex Model
Vs measured with error Autoregressive Fs
Age 7
Age 6
Age 11
Age 9
y1
y2
y3
y4
f1
f2
f3
f4
Curiously, middle part of model is identified
without restrictions, but the whole model is not
identified without some restrictive assumptions
e.g. measurement error and reliability constant
with age
36
Simplex Model ability2.inp

• TITLE Ability latent autoregressive model
• DATA FILE IS D\courses\mplus\ability.dat
• TYPE IS STDEVIATIONS CORRELATION
• NOBSERVATIONS204
• VARIABLE NAMES ARE y1-y4
• USEVARIABLES ARE y1-y4
• MODEL f1 by y1 (1)
• f2 by y2 (1)
• f3 by y3 (1)
• f4 by y4 (1)
• y1 y2 y3 y4 (2)
• f2 on f1
• f3 on f2
• f4 on f3
• MODEL INDIRECT f3 IND f1
• f4 IND f1
• OUTPUT STANDARDIZED

37
Simplex Model ability2.out

• TESTS OF MODEL FIT
• Chi-Square Test of Model Fit
• Value
  1.440
• Degrees of Freedom
  2
• P-Value
  0.4835
• TOTAL, TOTAL INDIRECT, SPECIFIC INDIRECT, AND
  DIRECT EFFECTS
• Estimates S.E. Est./S.E.
  Std StdYX
• Effects from F1 to F3
• Total 1.170 0.074 15.901
  0.925 0.925
• Total indirect 1.170 0.074 15.901
  0.925 0.925
• Specific indirect
• F3
• F2
• F1 1.170 0.074 15.901
  0.925 0.925
• Effects from F1 to F4

38
Simplex Model conclusion

• Conclusion.
• In the presence of measurement error in the
  mediator the mediated effect is underestimated
  (attenuated) and the residual direct effect
  over-estimated.
• With multiple predictors (mediators) measurement
  error can result in decreased, increased and
  quite spurious effects being estimated.
• But still ignores possible confounding to be
  addressed this afternoon