Title: Meta-analysis with missing data: metamiss
1Meta-analysis with missing data metamiss
- Ian White and Julian HigginsMRC Biostatistics
Unit, Cambridge, UK - Stata users group, London
- 10 September 2007
2Motivation
- Missing outcome data compromise trials
- So they also compromise meta-analyses
- We may want to
- correct for bias due to missing data
- down-weight trials with more missing data
- NB missing data within trials, not missing trials
3Plan
- Meta-analysis of binary data
- Haloperidol example
- Standard approaches to missing data
- Imputation methods
- IMORs
- Methods that allow for uncertainty
- Demonstration
4Haloperidol meta-analysis
Haloperidol Haloperidol Haloperidol Haloperidol Placebo Placebo Placebo Placebo missing
r1 f1 m1 n1 r2 f2 m2 n2 missing
Arvanitis 25 25 2 52 18 33 0 51 2
Beasley 29 18 22 69 20 14 34 68 41
Bechelli 12 17 1 30 2 28 1 31 3
Borison 3 9 0 12 0 12 0 12 0
Chouinard 10 11 0 21 3 19 0 22 0
Durost 11 8 0 19 1 14 0 15 0
Garry 7 18 1 26 4 21 1 26 4
Howard 8 9 0 17 3 10 0 13 0
Marder 19 45 2 66 14 50 2 66 3
Nishikawa 82 1 9 0 10 0 10 0 10 0
Nishikawa 84 11 23 3 37 0 13 0 13 6
Reschke 20 9 0 29 2 9 0 11 0
Selman 17 1 11 29 7 4 18 29 50
Serafetinides 4 10 0 14 0 13 1 14 4
Simpson 2 14 0 16 0 7 1 8 4
Spencer 11 1 0 12 1 11 0 12 0
Vichaiya 9 20 1 30 0 29 1 30 3
rsuccesses ffailures mmissing ntotal
5Standard approaches to missing data
- Available cases (complete cases) ignore the
missing data - assumes MAR missingness is independent of
outcome given arm - Assume missingfailure
- implausible, but not too bad for health-related
behaviours - Neither assumption is likely to be correct
6Other ideas
- Sensitivity analyses, e.g. do both
missingfailure and available cases - but these could agree by chance
- Explore best / worst cases
- Use reasons for missingness
- Explicit assumptions about informative
missingness (IM) - IM missingness is dependent on outcome
7metamiss.ado
- Processes data on successes, failures and missing
by arm feeds results to metan - Available cases analysis (ACA)
- Imputed case analyses (ICA)
- impute as failure ICA-0
- impute as success ICA-1
- best-case ICA-b (missingsuccess in E, failure
in C) - worst-case ICA-w
- impute with same probability as in control arm
ICA-pC - impute with same probability as in experimental
arm ICA-pE - impute with same probability as in own arm ICA-p
(agrees with ACA) - impute using IMORs ICA-IMOR (see next slide)
8More general imputation IMORs
- Measure Informative Missingness using the
Informative Missing Odds Ratio (IMOR) - Odds ratio between outcome and missingness
- Cant estimate IMOR from the data, but given any
value of IMOR, we can analyse the data - Generalises other ideas e.g.
- ICA-0 uses IMORs 0, 0
- ICA-1 uses IMORs ?, ?
- ICA-b uses IMORs ?, 0
- ICA-p uses IMORs 1, 1
- ICA-pC uses IMORs OR, 1 where OR is odds ratio
between arm and outcome in available cases
9Getting standard errors (weighting) right
- Weight 1 treat imputed data as real
- Weight 2 use standard errors from ACA
- Weight 3 scale imputed data to same sample size
as available cases - Weight 4 algebraic standard errors
- same as weight 1 for ICA-0, ICA-1, ICA-b, ICA-w
- same as weight 2 for ICA-p
- uses Taylor expansion for ICA-IMOR
- for ICA-pC ICA-pE, we condition on the IMOR (I
can explain)
10(No Transcript)
11(No Transcript)
12(No Transcript)
13(No Transcript)
14(No Transcript)
15(No Transcript)
16(No Transcript)
17(No Transcript)
18(No Transcript)
19(No Transcript)
20(No Transcript)
21Allowing for reasons (ICA-R)
- Specify number of missing individuals in each arm
to be imputed by each scheme ICA-0, ICA-1,
ICA-pC, ICA-pE, ICA-p, ICA-IMOR. - Can take these data from a different outcome
metamiss scales to missing - If missing in a particular study, metamiss
imputes using combined studies
22(No Transcript)
23Allowing for uncertainty
- So far we have pretended we really know the IMORs
- This is never really correct
- Now we allow them to be unknown but from a
user-specified distribution
24Bayesian approach allowing for uncertain IMORs
(Rubin, 1977)
25Bayesian analysis
- Elicit prior for dE, dC or use N(0,12) or N(0,22)
- Get posterior distribution by integrating over
the 2-dimensional distribution of dE, dC. - metamiss does this fast accurately by
- Standard normal approximation to posterior given
dE, dC - Integrate using Gauss-Hermite quadrature.
- Alternatives
- Taylor expansion (inaccurate for large SD of log
IMOR) - Full Bayesian Monte Carlo (slow, little gain in
accuracy)
26Understanding priors for log IMOR implied prior
for P(success missing) when P(success
observed) 1/2
27(No Transcript)
28(No Transcript)
29(No Transcript)
30Proposal 4 sensitivity analyses
IMORs Options (e.g.) Sensitive to Works via
fixed equal imor(2 2) Imbalance in missingness Point estimates
fixed opposite imor(2 1/2) Amount of missing data Point estimates
random equal sdlogimor(2) corr(1) Imbalance in missingness Weightings
random uncorrelated sdlogimor(2) corr(0) Amount of missing data Weightings
31Summary
- Tool for sensitivity analysis
- Requires thought about plausible missing data
mechanisms - Would be nice to overlay sensitivity analysis
with ACA - Further work includes combining uncertainty with
reasons - I also have a program mvmeta for multivariate
meta-analysis
32References
- 1st part Higgins JPT, White IR, Wood A.
Imputation methods for missing outcome data in
meta-analysis of clinical trials. Clinical
Trials, submitted. - 2nd part White IR, Higgins JPT, Wood AM.
Allowing for uncertainty due to missing data in
meta-analysis. 1. Two-stage methods. Statistics
in Medicine, in press. - Related White IR, Welton NJ, Wood AM, Ades AE,
Higgins JPT. Allowing for uncertainty due to
missing data in meta-analysis. 2. Hierarchical
models. Statistics in Medicine, in press. - metamiss.ado available from http//www.mrc-bsu.ca
m.ac.uk/BSUsite/Software/Stata.shtml
33Extra slides
34(No Transcript)