Sensitivity Analysis of Randomized Trials with Missing Data

1
Sensitivity Analysis of Randomized Trials with
Missing Data

• Daniel Scharfstein
• Department of Biostatistics
• Johns Hopkins University
• dscharf_at_jhsph.edu

2
ACTG 175

• ACTG 175 was a randomized, double bind trial
  designed to evaluate nucleoside monotherapy vs.
  combination therapy in HIV individuals with CD4
  counts between 200 and 500.
• Participants were randomized to one of four
  treatments AZT, AZTddI, AZTddC, ddI
• CD4 counts were scheduled at baseline, week 8,
  and then every 12 weeks thereafter.
• Additional baseline characteristics were also
  collected.

3
ACTG 175

• One goal of the investigators was to compare the
  treatment-specific means of CD4 count at week 56
  had all subjects remained on their assigned
  treatment through that week.
• The interest is efficacy rather than
  effectiveness.
• We define a completer to be a subject who stays
  on therapy and is measured at week 56.
  Otherwise, the subject is called a drop-out.
• 33.6 and 26.5 of subjects dropped out in the
  AZTddI and ddI arms, respectively.

4
ACTG 175

• Completers-only analysis

Treatment Mean CD4 SE 95 CI
AZTddI 385 8.5
ddI 360 7.7
Difference 25 11.5 (3,48) p0.0027
5
ACTG 175

• The completers-only means will be valid estimates
  if, within treatment groups, the completers and
  drop-outs are similar on measured and unmeasured
  characteristics.
• Missing at random (MAR), with respect to
  treatment group.
• Without incorporating additional information, the
  MAR assumption is untestable.
• It is well known from other studies that, within
  treatment groups, drop-outs tend to be very
  different than completers.

6
Goal

• Present a coherent paradigm for the presentation
  of results of clinical trials in which it is
  plausible that MAR fails (i.e., NMAR).
• Sensitivity Analysis
• Bayesian Analysis

7
Sensitivity Analysis
Step 1 Models

• For each treatment group, specify a set of models
  for the relationship between the distributions of
  the outcome for drop-outs and completers.
• Index the treatment-specific models by an
  untestable parameter (alpha), where zero denotes
  MAR.
• alpha is called a selection bias parameter and it
  indexes deviations from MAR.
• Pattern-mixture model

8
Treatment-specific imputed distributions of CD4
count at week 56 for drop-outs
9
Step 1 Models
Sensitivity Analysis

• Selection model
• The parameter alpha is interpreted as the log
  odds ratio of dropping out when comparing
  subjects whose log CD4 count at week 56 differs
  by 1 unit.
• alphagt0 (lt0) indicates that subjects with higher
  (lower) CD4 counts are more likely to drop-out.
• alpha0.5 (-0.5) implies that a 2-fold increase
  in CD4 count yields a 1.4 increase (0.7 decrease)
  in the odds of dropping out.

10
Step 2 Estimation
Sensitivity Analysis

• For a plausible range of alphas, estimate the
  treatment-specific means by taking a weighted
  average of the mean outcomes from the completers
  and drop-outs.

11
Treatment-specific imputed distributions of CD4
count at week 56 for drop-outs
12
Treatment-specific estimated mean CD4 at week 56
as function of alpha
13
Step 3 Testing
Sensitivity Analysis

• Test the null hypothesis of no treatment effect
  as a function of treatment-specific selection
  bias parameters.
• For each combination of the treatment-specific
  selection bias parameters, form a Z-statistic by
  taking the difference in the estimated means
  divided by the estimated standard error of the
  difference.

14
Step 3 Testing
Sensitivity Analysis

• If the selection bias parameters are correctly
  specified, this statistic is normal(0,1) under
  the null hypothesis.
• Reject the null hypothesis at the 0.05 level if
  the absolute value of the Z-statistic is greater
  than 1.96.

15
Contour Plot of Z-statistic
16
Contour Plot of Z-statistic
17
Bayesian Analysis

• Think of all model parameters as random.
• Place prior distributions on these parameters.
• Informative prior on alpha (e.g., normal with
  mean -0.5 and standard deviation 0.25).
• Non-informative priors on all other parameters
  (e.g., the distribution of the outcome).
• Results are summarized through posterior
  distributions.

18
Posterior Distributions
368 (342,391)
348 (330,365)
19
Posterior Distribution of Mean Difference
20 (-11,49) 91
20
Likelihood-based Inference

• A parametric model for the outcome and a
  parametric for the probability of being a
  completer given the outcome.
• For example, the outcome is log normal.
• Inference proceeds by maximum likelihood (ML).
• ML inference can be well approximated using
  Bayesian machinery.

21
Maximum Likelihood Distributions
303 (278,331)
368 (342,391)
-2.6 (-3.0,-2.1)
297 (271,324)
348 (330,365)
-2.8 (-3.3,-2.2)
22
Treatment-specific imputed distributions of CD4
count at week 56 for drop-outs
23
Maximum Likelihood Distribution of Mean Difference
20 (-11,49)
7 (-31,44)
24
Incorporating Auxiliary Information

• MAR (with respect to all observable data)
• Sensitivity analysis with respect alpha.
• Bayesian methods under development.

Longitudinal/Time-to-Event Data

• Same underlying principles.

25
LOCF

• Bad idea
• Imputing an unreasonable value.
• Results may be conservative or anti-conservative.
• Uncertainty is under-estimated.

26
Conjecture

• There is information from previously conducted
  clinical studies to help in the analysis of the
  current trials.
• Data from previous trials may be able to restrict
  the range of or estimate alpha.

27
Summary

• We have presented a paradigm for reporting the
  results of clinical trials where missingness is
  plausibly related to outcomes.
• We believe this approach provides a more honest
  characterization of the overall uncertainty,
  which stems from both sampling variability and
  lack of knowledge of the missingness mechanism.

28
dscharf_at_jhsph.edu

• Scharfstein, Rotnitzky A, Robins JM, and
  Scharfstein DO Semiparametric Regression for
  Repeated Outcomes with Non-ignorable
  Non-response, Journal of the American
  Statistical Association, 93, 1321-1339, 1998.
• Scharfstein DO, Rotnitzky A, and Robins, JM.
  Adjusting for Non-ignorable Drop-out Using
  Semiparametric Non-response Models (with
  discussion), Journal of the American Statistical
  Association, 94, 1096-1146, 1999.
• Rotnitzky A, Scharfstein DO, Su TL, and Robins
  JM A Sensitivity Analysis Methodology for
  Randomized Trials with Potentially Non-ignorable
  Cause-Specific Censoring, Biometrics, 5730-113,
  2001
• Scharfstein DO, Robin JM, Eddings W and Rotnitzky
  A Inference in Randomized Studies with
  Informative Censoring and Discrete Time-to-Event
  Endpoints, Biometrics, 57 404-413, 2001.
• Scharfstein DO and Robins JM Estimation of the
  Failure Time Distribution in the Presence of
  Informative Right Censoring, Biometrika
  89617-635, 2002.
• Scharfstein DO, Daniels M, and Robins JM
  Incorporating Prior Beliefs About Selection Bias
  in the Analysis of Randomized Trials with Missing
  Data, Biostatistics, 4 495-512, 2003.