Cochrane Systematic Reviews

1
Cochrane Systematic Reviews Cochrane Oral
Health GroupLuisa FernandezAnne-Marie
GlennyCochrane Oral Health Group, University
Dental Hospital of ManchesterEmma
TavenderCochrane EPOC Satellite, Melbourne
Australia
2
Aims

• To discuss the role of Cochrane systematic
  reviews and meta-analyses
• Cochrane Oral Health Group progress,
  undertaking a systematic review with the group
  looking to the future

3
Cochrane Systematic Reviews Oral
HealthAnne-Marie GlennyCochrane Oral Health
Group, University Dental Hospital of Manchester
4
What are systematic reviews?

• The process of systematically locating,
  appraising and synthesising evidence from
  scientific studies in order to obtain a reliable
  overview
• Aim to find all studies addressing the reviews
  question using an objective and transparent
  process

5
Why are they important ?

• Reduce large quantities of information into
  manageable portions
• Formulate policy and develop guidelines
• Efficient use of resources
• Increased power/precision
• Limit bias and improve accuracy

6
A. Thrombolytic Therapy
Textbook/Review Recommendations
Odds Ratio (Log Scale)
Routine Specific Rare/Never Experimental Not
Mentioned
Cumulative 0.5
1 2 Year
RCT Pts I I I I I I
I I I I I
1 23 1960 2
85 1965 3 149 4
138 1970 7 1783
10 2544 11 2651 1975 15 3311
17 3929 22 5452
23 5767 1980 27 6125
30 6346 1985 33 6571 43 21050
54 22051 65 47185
67 47531 1990 70 48154
21 8
1 10 1
2 2 8
7 8 1
12 1 8 4 1
7 3 5 2 2 1 15 8
1 6 1
plt.01 plt.001 plt.00001
Favours Treatment Favours Control
7
Systematic review

• Structured process involving several steps
• Well formulated question
• Comprehensive data search
• Unbiased selection and abstraction process
• Validity assessment of papers
• Synthesis of data

8
(No Transcript)
9
Well formulated question

• Participants
• Interventions (Exposure)
• Comparisons
• Outcomes

10
Well formulated question

• Participants
• Interventions
• Comparisons
• Outcomes
• Pit and fissure sealants versus placebo for the
  prevention of dental caries in permanent teeth in
  children and adolescents

11
Inclusion criteria

• Participants
• Interventions
• Comparisons
• Outcomes
• Study design/methodological quality

12

• What type of study design?
• How effective is paracetamol at reducing pain?
• Does smoking increase the risk of oral cancer?

13

• STRONG Experimental studies/ clinical trials
• Randomised controlled trials
• Non-randomised controlled trials
• Observational studies
• Cohorts
• Case-controls
• Cross-sectional surveys
• Case series
• Case reports
• WEAK Expert opinion, consensus

14
Search strategy

• Needs to be as comprehensive as possible
• Consider
• Electronic databases (Cochrane Controlled Trials
  Register, Medline, Embase)
• Reference lists
• Handsearching
• English language/non-English language
• Sources of ongoing and/or unpublished studies

15
Reporting biases

• Statistically significant positive results are
  
• more likely to be published
• publication bias
• more likely to be published rapidly
• time lag bias
• more likely to be published in English
• language bias
• more likely to be cited by others
• citation bias

16
Unbiased selection and data abstraction process

• Selection of relevant papers

17
Unbiased selection and data abstraction process

• Selection of relevant papers
• Data abstraction/extraction

18
Data extraction is

• Time consuming
• Often subjective
• Prone to error
• Often difficult

19
Unbiased selection and abstraction process

• Predefined data abstraction form
• Independently and in duplicate

20
4. Validity assessment

• Can be used
• As a threshold for inclusion of studies
• As a possible explanation for differences in
  results between trials
• In sensitivity analyses
• As weights in statistical analysis of the results

21
Quality assessment tools

• Composite scales - assign numerical value to
  individual items to provide overall estimate of
  quality problematic
• Component approach - assesses relevant
  methodological aspects individually (e.g
  randomisation, blinding, drop-outs) - preferred

22
4. Validity assessment

• Process should be conducted independently by at
  least two reviewers
• Results of the quality assessment should be
  reflected in the analysis

23
Study synthesis

• Appropriate pooling
• qualitative (narrative)
• quantitative (meta-analysis)
• inappropriate when data are sparse or
  heterogeneity exists
• Clear presentation of individual studies included
  in the review

24
Meta-analysis

• The process of using statistical methods to
  combine the results of different studies.
• The aim is to integrate the findings, pool the
  data, and identify the overall trend of results
• (Dictionary of Epidemiology, 1995)

25
What is a meta-analysis?

• Optional part of a systematic review

Systematic reviews
Meta-analyses
26
When can/should you do a meta-analysis?

• When more than one study has estimated an effect
• When there are no differences in the study
  characteristics that are likely to substantially
  affect outcome
• When the outcome has been measured in similar
  ways
• When the data are available (beware when only
  some data are available)

27
Summary statistic for each study

• Calculate a single summary statistic to represent
  the effect found in each study
• - usually displayed with 95 confidence
  intervals (CI)

28
Weighting studies

• More weight to the studies which give us more
  information
• More participants
• More events
• Lower variance
• Weight is proportional to inverse variance

29
Displaying results graphically

• Forest plots
• Commonly used

30
theres a label to tell you what the
comparison is and what the outcome of interest is
Alderson 2002
31
At the bottom theres a horizontal line. This is
the scale measuring the treatment effect. Here
the outcome is death and towards the left
the scale is less than one, meaning the
treatment has made death less likely. Take care
to read what the labels say things to the left
do not always mean the treatment is better
than the control.
Alderson 2002
32
The vertical line in the middle is where
the treatment and control have the same effect
there is no difference between the two
Alderson 2002
33
The data for each trial are here, divided into
the experimental and control groups
This is the weight given to this study in the
pooled analysis
For each study there is an id
Alderson 2002
34
The data shown in the graph are also given
numerically
The label above the graph tells you what
statistic has been used

• Each study is given a blob, placed where the data
  measure the effect.
• The size of the blob is proportional to the
  weight
• The horizontal line is called a confidence
  interval and is a measure of
• how we think the result of this study might vary
  with the play of chance.
• The wider the horizontal line is, the less
  confident we are of the observed effect.

Alderson 2002
35
The pooled analysis is given a diamond
shape where the widest bit in the middle is
located at the calculated best guess (point
estimate), and the horizontal width is the
confidence interval
Note on interpretation If the confidence
interval crosses the line of no effect, this is
equivalent to saying that we have found no
statistically significant difference in the
effects of the two interventions
Alderson 2002
36
Heterogeneity

• Clinical heterogeneity differences in trial
  characteristics
• Statistical heterogeneity - the variability in
  the reported effect sizes between studies
• how similar are the results?
• are the differences among the results of the
  trials greater than could be expected by chance
  alone?

37
Heterogeneity
38
Heterogeneity

• Chi-squared test of heterogeneity
• Plt0.1 demonstrates statistically significant
  heterogeneity
• may not be appropriate to pool data

39
Subgroup analyses

• Where it is suspected in advance that certain
  features may alter the effect of an intervention
• Example
• women
• a particular age group
• those with a specific disease subtype

40
Subgroup analysis

• Often misleading-
• Is there indirect evidence in support of a
  difference?
• Did the hypothesis about the difference precede
  rather than follow the analysis?
• Is the subgroup analysis one of a small number of
  hypotheses tested?

41
(No Transcript)
42
Sensitivity analysis

• Does result change according to small variations
  in the data and methods?
• Choice of treatment effects or method for pooling
• Inclusion/exclusion of dubious data
• Inclusion/exclusion of trials

43
A common sensitivity analysis is to repeat the
analysis taking out lower quality trials
44
Useful websites

• http//www.shef.ac.uk/scharr/ir/netting/
• (comprehensive list of E.B.P. websites and links)
• http//www.york.ac.uk/inst/crd
• CRD manual on how to conduct a systematic review
• DARE - database of abstracts of reviews of
  effectiveness
• http//www.cochrane.org
• reviewers handbook
• Cochrane Library (abstracts only)

45
Cochrane Oral Health Group looking to the
futureEmma TavenderCochrane EPOC Satellite,
Melbourne Australia
46
Aims

• Progress of the group
• Process of undertaking a systematic review with
  the group support available
• Challenges for the future
• Questions answers

47
Cochrane Oral Health Group
Include all RCTs of oral health broadly
conceived to include the prevention, treatment
and rehabilitation of oral, dental and
craniofacial diseases and disorders
Achievements

• 624 members from 40 countries
• Panel of referees
• Set up Specialised Register of Trials (21,000)
• Handsearching programme
• Annual Evidence Based Dental Practice Course
• NIDCR funding for oral cancer reviews

48
Oral Health Group Reviews
68 Protocols
54 Reviews
21 Updates
49
Growth of Oral Health Group Trials
RegisterJanuary 1998 January 2006
50
Editorial Process

• Register title
• Prepare protocol
• Editorial and external review of protocol
• Protocol entered on Cochrane Library
• Identify trials
• Complete systematic review
• Peer review of systematic review
• Systematic review entered on Library
• Update the review regularly (every 2 years)

51
Editorial support

• Register title
• Check for overlap
• Find co-reviewers with similar interests
• Assigned Contact Editor

52
Editorial support

• Prepare protocol
• Reviewers Handbook
• RevMan software/support at editorial base
• Protocol training workshop (free of charge)
• Online learning materials
• Help with developing search strategy (TSC)

53
Editorial Support

• Identify trials
• Complete systematic review
• Search the OHG Trials Register
• Analysis workshop (free of charge)
• Statistical methodological support from
  editorial base
• Help with translations

54
Editorial support

• Update the review regularly
• Help with identifying new trials
• Procedures for changing lead author

55
Challenges for the future

• Prioritisation of reviews
• Diagnostic reviews
• Reviews of no trials including other levels of
  evidence?
• Umbrella reviews
• Funding
• Dissemination

56
Challenges for the future

• Prioritisation of reviews
• Increasing number of reviews/overlap
• Usefulness answering the right questions for
  clinicians, consumers, funders, policy makers?
• NIDCR funding of oral cancer reviews
• OHG Symposium identify questions/topics
• Future ways of identifying priorities for
  consumers etc

57
Challenges for the future

• Diagnostic reviews
• Important area within dentistry
• Cochrane Diagnostic Reviews of Test Accuracy
  Initiative
• 4 phases development of materials, piloting,
  implementation publication
• Piloting phase 10 test review teams
• Training implementation 2007

58
Challenges for the future

• Reviews of no trials including other levels of
  evidence?
• Scope of OHG includes RCTs and Quasi RCTs
• Reviews eg. Endocarditis included other levels of
  evidence
• OHG currently developing guidance for reviewers
  on how to deal with reviews of no trials and when
  to include other levels

59
Challenges for the future

• Umbrella reviews
• Combine the findings of a number of related
  reviews so information is in one document
• Incorporate summary of findings tables
• RevMan 5 released in 2007 will include this
  feature
• Cochrane Umbrella Reviews Working Group

60
Challenges for the future

• Funding
• 5 years funding from DoH, UK
• Covers salary of staff, administration at
  editorial base
• NIDCR grant to undertake series of oral cancer
  reviews
• NICE, Italian Govn., vCIOH
• Alternative methods?

61
Challenges for the future

• Dissemination
• The Cochrane Library
• Journal articles by products
• Courses/conferences
• Incorporation in guidelines (HTA, NICE, SIGN)
• Improve visibiltiy and utilisation of reviews

62
Any questions?