Systematic Reviews and Meta analyses

1
Systematic ReviewsandMeta analyses

• Robert Gibson
• Fellow, Nutrition Society of Australia
• NHMRC Senior Research Fellow

2
European Championship Spain 1 Germany 0
3
The answer? Review the literature
4
Traditional, narrative review

• Usually written by experts in the field
• Open to bias
• Qualitative, narrative summaries of evidence on a
  given topic
• Typically involve informal and subjective methods
  to collect and interpret information
• Selection, assessment and integration of included
  studies often not stated

Example PhD literature review
5
What is a systematic review

• Overview of all primary studies
• Contains an explicit statement of objectives,
  materials, and methods
• Conducted according to explicit and reproducible
  methods
• May include statistical synthesis of data
• Meta-analysis
• Undertaken by more than one person

6
The need for systematic reviews

• Making sense of differing results from different
  studies
• Need same rigour in secondary research as is
  expected from primary research
• Not just confined to health
• Advertising, agriculture, astronomy, biology,
  law, manufacturing, zoology

7
Advantages

• Explicit methods limit bias in identifying and
  rejecting studies
• Conclusions are more reliable and accurate
  because of methods used
• Large amounts of information can be assimilated
  quickly by healthcare providers, researchers, and
  policymakers

8
Advantages

• Reduce delay between research discoveries and
  implementation
• Can point to research needs
• Requirement for all NHMRC proposals

9
Regulators require the best evidence
I Systematic review of all relevant trials II At
least one well-designed RCT III-1 Non-randomised
well-designed CT III-2 Well-designed cohort or
case control study III-3 Multiple time
series IV Opinion of respected authorities, based
on clinical experience, descriptive studies,
reports expert committees
US Preventative Task Force
10
The Cochrane Collaboration

• Named in honour of Archie Cochrane, a British
  researcher
• In 1979 he wrote
• "It is surely a great criticism of our profession
  that we have not organised a critical summary, by
  specialty or subspecialty, adapted periodically,
  of all relevant randomized controlled trials

Source http//www.cochrane.org/cochrane/archieco.
htm
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
Step 1 Protocol

• As important as a protocol for a primary research
  study.
• Systematic review less likely to be biased if
• questions are well developed
• methods decided apriori.

15
Protocol components

• Components of a protocol
• Title
• Background
• Review questions, objectives
• Search strategy including search terms and
  resources to be searched
• Study selection criteria and procedures
• Study quality assessment checklists and
  procedures
• Data extraction strategy
• Synthesis of the extracted evidence
• Project timetable
• Protocols sent out to external review

16
The effect of dairy foods on coronary heart
disease a systematic review of prospective
cohort studies

• Bob Gibson
• Professor Functional Food Science
• University of Adelaide
• In association with
• Maria Makrides, Lisa Smithers
• Melanie Voevodin, Andy Sinclair

17
Ideally

• We have a large number of well conducted
  Randomised Controlled Trials (RCTs) with large
  numbers of subjects to combine
• Only cohort studies were available for the Dairy
  review

18
Background

• Dairy foods may increase the risk of coronary
  heart disease because of contribution they make
  to total saturated fat intake
• Many of the studies have focussed on
• Saturated fats
• Calcium
• Protein
• All have a good theoretical basis for
  consideration
• However, dairy fats contain other bioactives
  (CLA, EPA, DPA) and also calcium that may
  counteract the effect of saturates

19
Objectives

• To assess the effect of dairy foods on coronary
  heart disease using all available prospective
  cohort studies.
• Studies with retrospective components such as
  case controlled studies were excluded

20
Search strategy

• A search of electronic databases included
  MEDLINE, EMBASE, CENTRAL, CINAHL, citation index
  (Web of Science) and the Australian and
  International dissertation libraries.
  Cross-referencing from collected citations was
  used to identify additional studies. Date of
  last search February 2008.
• Selection criteria
• Prospective cohort studies where intake of dairy
  foods was measured and these intakes were related
  to heart disease and death end points.

21
Data collection analysis

• Eligible studies were assessed for quality and
  data extracted. Primary outcomes were death,
  death from coronary heart disease (CHD), ischemic
  heart disease (IHD), or episode of myocardial
  infarct. Two reviewers assessed study quality
  and extracted data with discrepancies resolved by
  consensus.

22
Results

• 12 studies eligible for inclusion assessing
  gt215,000 subjects.
• Most studies had close to or greater than 80
  follow-up rate, made adjustment for three or more
  confounders in the statistical analysis and used
  standard criteria to determine CHD/IHD end
  points.
• About half the studies used a validated food
  frequency questionnaires (FFQ), administered the
  FFQ more than once, or had a follow-up duration
  of 20 years or more.
• Less than half the studies involved subjects with
  characteristics representative of the general
  population.

23
Results summary

• Eight of twelve studies reported no association
  between dairy intake and CHD/IHD.
• Four of twelve studies suggested some association
  between some aspect of dairy intake and CHD/IHD.

24
Cohort studies showing no association
25
Cohort studies showing no association
26
Cohort studies showing some effect
27
But..

• Very few designed specifically to address issue
  of dairy consumption and heart disease
• In most the information for dairy intake was
  collected as part of FFQ
• Attempting to ascribe effects of a single food to
  multi-factorial events such as CVD is extremely
  difficult
• Studies do consistently show that high intakes of
  saturated fats are associated with increased risk
  with CHD
• Vegetarian diets generally protective

28
Conclusions(Subjective but testable)

• This assessment of twelve prospective cohort
  studies indicates that there is no consistent
  evidence in support of the concept that dairy
  intake is consistently associated with higher
  CHD/IHD risk.
• However, most studies support the concept that
  high saturated fat intake are associated with
  higher risks of CHD/IHD
• It is almost impossible to tease out the relative
  effects
• Should do a meta analysis but raw data not
  available

29
What is a meta-analysis?

• A meta-analysis involves pooling separate
  experimental determinations of some quantity to
  give a weighted average likely to be more
  accurate than any of the individual contributing
  values. The accuracy of the result is likely to
  be improved if experiments of poor quality are
  given low weighting or discarded altogether.
  Important parts of the process are to collect all
  available relevant studies and to assess them
  against a rigid set of quality criteria. Provided
  that some key facts have not been overlooked or
  misinterpreted, the combined review and
  meta-analysis process can be expected to give
  results that are more reliable and accurate than
  results from typical single studies, and far
  better than generally indicated by experience,
  anecdotes or common beliefs

30
Averaging studies

• A simple average gives each study equal weight
• This seems intuitively wrong
• Some studies are more likely to give an answer
  closer to the true effect than others

31
Weighting studies

• More weight to the studies which give us more
  information
• More participants
• More events
• Lower variance
• Weight is proportional to inverse variance
• (1/?SE)
• Larger studies with smaller standard errors
  more weight

32
Meta-analysis

• Categorical data
• Risk ratio (or relative risk)
• risk of event on treatment
• risk of event on control
• risk number of events
• total number of observations
• Where risk ratio 1, this implies no difference
  in effect
• Risk difference
• risk on control - risk on treatment

33
Example Effect of omega 3 LCPUFA on growth of
preterm infants

• Animal and one clinical trial suggested that
  omega 3 LCPUFA had positive effect on mental
  development
• However, 3 trials had suggested a negative effect
  on growth
• As there was no treatment to overcome the mental
  delay seen in preterm infants there was a need to
  do a large RCT to test effect of DHA on mental
  development

34
Meta analysisEffect of diet on growth of
preterm infants
35
What did this tell us?

• Long chain polyunsaturates were safe they
  didnt reduce the growth of infants
• Allowed us to apply for significant funding to
  NHMRC to do a large scale intervention study
• Convinced the Ethics committees that we would do
  no harm

36
DINO Centres across Australia
657 infants (from 545 women) were randomized to
either High or Low DHA
37
Percentage of infants with Mild and Significant
Mental Delay
Analyses account for clustering of infants within
mother
38
Percentage of Infants lt1250g with Mild and
Significant Mental Delay
Analyses account for clustering of infants within
mother
39
Antioxidants and deathJAMA, February 28, 2007
40
(No Transcript)
41
(No Transcript)
42
The small risk effects in individual studies only
became significant when all studies were
aggregated in the meta analysis
43
What does this tell us?

• Vitamins and minerals are not necessarily safe
  because they are nutrients
• There may be a difference between nutrients in
  food and chemically derived nutrients
• Mega dose nutrient intakes may be harmful
• Australian regulators are now looking at
  permitted levels in foods and vitamin/mineral
  supplements

44
Summary

• Systematic reviews and meta analyses are only
  tools
• Limited by the quality of the studies garbage
  in, garbage out
• Highlights needs future research directions
• Can stop useless repetition
• Can be a powerful tool to detect small but
  significant effects both ve and -ve
• Certainly better than opinion