Split-mouth studies and systematic reviews the analysi

1
Split-mouth studies and systematic reviews

• Ian Needleman1 Helen Worthington2
• 1Unit of Periodontology
• UCL Eastman Dental Institute
• International Centre for Evidence-Based Oral
  Health, London UK
• 2School of Dentistry, University of Manchester, UK

2
Split-mouth studies and systematic reviews

• What is the issue?
• Why include them?
• How to include them
• Examples

3
Split-mouth studies and systematic reviews the
analysis issue

• As we know, the analysis of split-mouth and
  parallel group studies is not the same.
• As a result, if a meta-analysis includes both
  types of trials without considering the
  differences, the result might be unreliable
• The confidence interval will be incorrect,
  possibly leading to
• An inappropriate conclusion on clinical
  importance (and statistical significance)
• Distortion of impact of clinical heterogeneity

4
Why include split-mouth studies in systematic
reviews?

• Because of the totality of the evidence
• Possible advantages of split-mouth trials over
  parallel group
• Each participant acts as own control
• Therefore, fewer participants are required to
  obtain same study power as parallel group
• Every participant receives each intervention,
  therefore good for determining preferences

5
How to include split-mouth studies in systematic
reviews

• Designing the systematic review
• Is split-mouth an appropriate design to answer
  this question?
• Are carry-over effects a risk?

6
How to include split-mouth studies in systematic
reviews

• Conducting the systematic review
• Fundamental question Is meta-analysis justified
  in principle?
• Are the trials similar enough in chief
  characteristics
• Types of populations
• Types of interventions
• Types of outcomes

7
How to include split-mouth studies in systematic
reviews

• Possible ways
• Narrative (qualitative) summary in evidence
  tables only.
• Advantage split-mouth studies contribute to
  totality of evidence and analytic issues (may be)
  avoided
• Disadvantage Do not contribute to summary
  estimate or to investigation of heterogeneity
• Analyse as if parallel group
• Not recommended due to potentially unreliable
  meta-analysis summary estimate

?
8
How to include split-mouth studies in systematic
reviews

• Possible ways
• Meta-analyse those split-mouth trials with
  adequate data separately from parallel group
  trials and ignore those without such data
• Advantage More information
• Disadvantage Selection bias
• Incorporate data from first intervention side if
  reported separately
• Advantage More information
• Disadvantage May be biased sample where
  trialists identified carry-over effects

9
How to include split-mouth studies in systematic
reviews

• Possible ways
• Approximate a paired analysis by inputing a
  measure describing the similarity of outcomes
  within each participant.
• Advantages Makes use of all trials
• Disadvantages
• May make assumptions about data (that can be
  tested)
• May need statistical support

10
Approximating a paired analysis - you will need
one of the following

• Individual patient data in publication or from
  contact with trialist
• Mean and SD/SE of patient specific differences
  between intervention A and B measurement

11
Approximating a paired analysis - you will need
one of the following

• Mean difference (or difference between means) and
  one of
• t-statistic (paired t-test)
• P-value from paired t-test
• Confidence interval from paired analysis
• Graph of measurement of intervention A and B from
  which matched individual data values can be
  extracted.

12
Approximating a paired analysis

• Step one. Calculate the correlation coefficient
  (r) for each study.
• The correlation coefficient describes how similar
  the measurement of intervention A and B were
  within a participant
• Assumes that mean and SD for each intervention
  treatment side/period would be the same
• If r is inconsistent between studies, then
  caution on proceeding further.
• If r cannot be calculated for a trial, use
  representative value from other trials as the
  trials should be very similar for a particular
  intervention in a systematic review

13
Approximating a paired analysis

• Step two. Calculate SE of the mean difference
  between the interventions
• Step three. Enter the data into meta-analysis
  software. Generic inverse variance method of
  Cochrane Software - RevMan - particularly
  flexible (free download).
• Step four. Conduct sensitivity analyses employing
  different values for r to investigate robustness
  of estimates especially for studies were r could
  not be calculated

14
How to include split-mouth studies in systematic
reviews

• Further issues
• Generally advisable to meta-analyse split-mouth
  and parallel group trials separately as
  sub-groups to investigate systematic differences
• Authors of systematic reviews please state
  explicitly how data from split-mouth studies has
  been managed

15
How to include split-mouth studies in systematic
reviews

• Example of meta-analysis of split mouth and
  parallel group trials.

16
Needleman, Worthington, Giedrys-Leeper, Tucker
2006, Cochrane Database of Systematic Reviews
17
Split-mouth studies and systematic reviews

• Conclusions
• Split-mouth studies should be included in
  systematic reviews when appropriate
• It is possible to combine split-mouth and
  parallel group studies in meta-analysis
• Our observation within the Cochrane Oral Health
  Group is that differences in effect sizes exist
  between split-mouth and parallel group studies of
  the same intervention

18
Split-mouth studies and systematic reviews

• Resources
• Cochrane Handbook and RevMan www.cochrane.org
• Elbourne et al. 2002. Int J Epidem, 31 140-149
• Follman et al. 1992. J Clin Epidem, 45 769-773
• Contact i.needleman_at_eastman.ucl.ac.uk