Systematic reviews and Meta-analyses

1
Systematic reviews and Meta-analyses

• Alison Brettle,
• Research Fellow (Information)
• Salford Centre for Nursing, Midwifery and
  Collaborative Research
• University of Salford

2
Aims

• To discuss the role of the systematic reviews and
  meta-analyses and cover issues involved in their
  critical appraisal and interpretation

3
Systematic Review

• A review of all the literature on a
  particular topic, which has been systematically
  identified, appraised and summarised giving a
  summary answer.

4
What is a systematic review?

• An overview of primary research studies conducted
  according to explicit and reproducible
  methodology
• A rigorous method of summarising research
  evidence
• Shows what we know and dont know about a topic
  area
• Provides evidence of effectiveness (or not) by
  summarising and appraising relevant evidence

5
Systematic reviews aim

• To find all relevant research studies (published
  and unpublished)
• To assess each study on basis of defined criteria
• Synthesise the findings in an unbiased way
• Present a balanced and impartial summary of the
  findings taking any flaws into consideration

6
Advantages of systematic reviews

• Summarise evidence, keep people up to date
  without reading all published research literature
• Allow large amounts of data to be assimilated (eg
  by busy clinicians, policy makers etc)
• A clearer picture by collating results of
  research
• Reduce bias removes reviewers personal
  opinions, preferences and specialist knowledge
• Explicit methods - allow the reader to assess how
  review has been compiled
• More reliable conclusions because of methods used

7
Systematic review models

• Medical/Health care
• Cochrane Collaboration, NHS Centre for Reviews
  and Dissemination
• Usually includes high quality research evidence
  RCTs
• Often includes meta-analysis (mathematical
  synthesis of results of 2 studies that addressed
  same hypothesis in same way)
• Social care/Social Sciences
• SCIE, EPPI Centre, Campbell Collaboration
• Often include wider range of studies including
  qualitative
• Often narrative synthesis of evidence

8
Systematic review process

• Define/focus the question
• Develop a protocol
• Search the literature (possibly 2 stages scoping
  and actual searches)
• Refine the inclusion/exclusion criteria
• Assess the studies (data extraction tools, 2
  independent reviewers)
• Combine the results of the studies to produce
  conclusion can be a qualitative or quantitative
  (meta-analysis)
• Place findings in context quality and
  heterogeniety of studies, applicability of
  findings

9
Methodology for a systematic review of randomised
controlled trials1
Greenhalgh, T, BMJ 1997315672-675
10
What type of study design?

• How effective is paracetamol at reducing pain?
• Does smoking increase the risk of oral cancer?

11

• STRONG Experimental studies/ clinical trials
• Randomised controlled trials
• Non-randomised controlled trials
• Observational studies
• Cohorts
• Case-controls
• Cross-sectional surveys
• Case series
• Case reports
• WEAK Expert opinion, consensus

12
Experimental studies

• Randomised controlled trial
• Non-randomised controlled clinical trial

• Evaluating the effectiveness of an intervention

13
Observational studies

• Measuring the incidence of a disease looking at
  the causes of disease determining prognosis
• Looking at the causes of disease identification
  of risk factors suitable for examining rare
  diseases
• Measuring the prevalence of a disease examining
  the association

• Cohort
• Case-control
• Cross-sectional
• survey

14
What is a meta-analysis?

• Optional part of a systematic review

Systematic reviews
Meta-analyses
15
Meta-analysis

• The process of using statistical methods to
  combine the results of different studies.
• The aim is to integrate the findings, pool the
  data, and identify the overall trend of results
• (Dictionary of Epidemiology, 1995)

16
Systematic Reviews

• Understanding the jargon and the blobs!

17
Odds Ratio, Relative Risk Measures of risk
The likelihood of something happening V The
likelihood of something not happening
18
Odds Ratio Graph (Blobbogram)
LEFT E S S
M O RIGHT E
Line of no significance
19
Odds Ratio
Best estimate
Confidence Interval (wobble factor)
20
Odds Ratio (Blobbogram)
21
Confidence Interval
Is the range within which the true size of effect
(never exactly known) lies, with a given degree
of assurance (95 or 99).
22
Confidence Intervals(Wobble factor)
23
Confidence Interval (CI) the wobble factor,
how sure are we about the results? - the
shorter the CI the more certain we are about the
results - if it crosses the line of 1 (no
treatment effect) the intervention might not be
doing any good and could be doing harm
24
Heterogeneity

• Clinical heterogeneity differences in trial
  characteristics
• Statistical heterogeneity - the variability in
  the reported effect sizes between studies
• how similar are the results?
• are the differences among the results of the
  trials greater than could be expected by chance
  alone?

25
Number needed to treat (NNT)
the number of people you would need to
treat with a specific intervention to see one
additional occurrence of a specific outcome
26
The p-value in a nutshell
How often you would see a similar result by
chance, when actually there was no effect by the
drug or treatment.
0
1
Impossible
Certain Absolutely
p0.001 Very unlikely 1 in 1000 p0.05
Fairly unlikely 1 in 20 p0.5 Fairly
likely 1 in 2 p0.75 Very likely 3 in 4
27
Critical appraisal

• Is the study valid?
• Trustworthy
• What are the results?
• Is it useful in practice?
• Relevant?
• Generalisable?

28
Evaluating quality of systematic reviews

• Is there a clearly defined question?
• Thorough and comprehensive search
• Was methodological quality assessed and studies
  weighted accordingly? (Were studies reliable and
  valid?)
• How sensitive are the results to the way the
  review was done ie if you changed the inclusion
  criteria how would this affect results?
• Interpretation of numerical results

29
Further reading

• Greenhalgh T (1997) How to read a paper papers
  that summarize other papers (systematic reviews
  and meta-analyses), BMJ, 315672-675

30
Useful resources

• Cochrane Collaboration
• http//www.cochrane.org/
• http//www.cochrane.org/docs/irmg.htm
• Centre for Reviews and Dissemination
• http//www.york.ac.uk/inst/crd/
• Finding studies for systematic reviews
• http//www.york.ac.uk/inst/crd/revs.htm
• EPPI-Centre Stages of a review
• http//eppi.ioe.ac.uk/cms/Default.aspx?tabid89
• SCIE - The conduct of systematic research reviews
  for SCIE knowledge reviews
• http//www.scie.org.uk/publications/details.asp?pu
  bID111