Estimating and Understanding Therapist Effects

1
Estimating and Understanding Therapist Effects

• Empirical Evidence

Bruce E. Wampold University of Wisconsin--Madison
2
History of Omission

• Historically, provider effects ignored
• Education
• Agriculture
• Medicine

3
Ignoring Therapists

• Therapists unimportant
• Therapist effects not estimated
• Focus on treatment
• Methodological issues
• Differences among treatments may be due to
  therapists
• Increases Type I error rate and effect size for
  Tx effects
• Confounds within and between group relationships

4
Goals

• How important is the provider relative to the
  treatment?
• Estimate the variability among therapists (within
  treatments)
• Therapist variability v. treatment variability
• Understand the characteristics and actions of
  effective therapists
• Discriminate between patient and therapist
  contributions to outcomes

5
Estimating Therapist Effects

• Sources of variability
• Treatment differences (fixed effect) aj
• Therapist variability (random effect) s2ther
• Error or patient variability s2error
• Therapist Effects
• intraclass correlation coefficient
• (Therapist variability) / (Total Variability)
• ? s2ther/(s2ther s2error)

6
Effects as Percentage of Variability of
Termination Score

• Pretest 40 to 50
• Tx v. No Treatment about 13
• Treatment A v. Treatment B at most 1
• Alliance 5 to 9
• Therapist.
• 9 (Chrits-Christoph et al., 1991)

7
NIMH TDCRP reanalysis

• Nested Design (CBT and IPT)
• Well trained therapists, adherence monitored,
  supervision
• Elkin
• The treatment conditions being compared in this
  study are, in actuality, packages of particular
  therapeutic approaches and the therapists who
  choose to and are chosen to administer them. The
  central question is whether the outcome findings
  for each of the treatments, and especially for
  differences between them, might be attributable
  to the particular therapists participating in the
  study.

8
Random Effects Modeling

• Therapists considered a random factor
• Therapists nested within treatments (multilevel
  model)
• Final observations, controlling for pretest at
  patient and therapist level
• Kim, Wampold, Bolt, Psychotherapy Research, 2006

9
Random Effects Modeling

• Therapists considered a random factor
• Therapists nested within treatments (multilevel
  model)
• Final observations, controlling for pretest at
  patient and therapist level
• Therapist slope fixed and random
• Kim, Wampold, Bolt, Psychotherapy Research, 2006

10
Variance due to Tx and Therapists
11
Variance due to Tx and Therapists
Note Elkin et al. (2006) found negligible
therapist effects in the same data
12
Variance due to therapists in practice

• 581 Therapists, 6146 patients
• More heterogeneous patients
• Outcome Questionnaire 30
• Diagnosis, degree, experience, 0 percent
• Medication, 1 percent (but dependent on
  psychotherapist)
• 5 percent
• Wampold Brown, JCCP, 2005

13
Cross-validation year 1 to year 2
At least 9 cases in yr 1 73 Therapists
14
Therapist Effects in Psychopharmacology (NIMH)

• Antidepressants Imipramine v. Placebo
• 3 due to treatment
• 9 due to therapist
• Best therapists get better outcome with placebo
  that worst therapists with imipramine
• McKay, Imel Wamold, 2006

15
Therapists Psychopharm
16
Conclusions

• Therapists make a difference
• Size of therapists effects at least an order of
  magnitude greater than treatment effects
• What are the characteristics or actions of
  effective therapists?

17
Characteristics and Actions of Effective
Therapists

• Consult Buetler (Handbook of Psychotherapy and
  Behavior Change)
• We dont know
• And we dont care
• Alliance?
• Alliance measured early in therapy related to
  outcome
• Therapist contribution?
• Patient contribution?
• Interaction?

18
Alliance Patient v. Therapist Contribution to
Alliance

• Counseling center consortium data
• OQ pre and post, Alliance 4th session
• 188 patients, 22 therapists
• 5 of variance due to therapists
• What is correlation of alliance with outcome
• Within therapists?
• Between therapists?
• And the results.

19
Within or between?
20
Size of Effect
21
Conclusions

• Method is vital to proper conclusions
• Improper models affect results
• Therapist effects are sizable, especially
  compared to treatment differences
• Every process and outcome study MUST include
  therapists in the model
• Multilevel modeling can answer the fundamental
  question