Metaanalysis

1
Meta-analysis

• What is it, is it useful?

2
What is meta-analysis?

• An analysis of analyses trying to make
  generalisations from a series of experiments in
  an unbiased, quantitative way

3
Making Generalisations I

• Traditional narrative review (X found this, on
  the other hand Y found that ..)
• Subjective
• Not quantitative
• Can be overwhelmed by large number of studies
  (e.g. high CO2 research)

4
Making Generalisations II

• Vote counting

NB Cant take mean response Conclusions biased
because significance level of a study is a
function of the sample size i.e. some studies
are better than others
5
Making Generalisations III

• Meta analysis
• weights each study according to sample size

General procedure - find a set of studies with
similar data - extract an effect size from each
study - calculate a weighted average effect size
across studies - construct a confidence
interval - divide data into groups (e.g.
terrestrial vs. marine) and test to see if groups
behave similarly
6
Example

• Stomatal conductance (gs) of trees at elevated CO2

Open-top chamber, Antwerp
CO2 response of gs of Sitka spruce (Craig Barton)
7
1) Which studies?

• Should you exclude those of dubious quality?

FACE
pots
chambers
8
Three Major Meta-analyses of gs

• Curtis Wang 1998 all studies published to
  date (pots, chambers, branch bags etc)
• Medlyn et al 2001 ECOCRAFT (long-term,
  field-based experiments) but
  cf. Curtis Wang showed studies gt 1 year were
  consistent
• Ainsworth Rogers 2007 FACE studies only (but
  all growth forms)

9
Potential Biases

• Non-significant results dont get published the
  desk-drawer problem
• Re-publication
• Overcoming bias
• Know the literature!
• Calculate how many studies would be needed to
  give a barely non-significant result
• Funnel plots

10
Funnel Plots
no bias
Sample Size
Effect Size
bias
Hunter Schmidt 2004
data from Medlyn et al. 2001
11
2) Extracting data

• a) Obtain from publications
• (graphical data can be digitised e.g. DataThief)
• b) Ask nicely
• Whats needed?
• Mean, standard deviation, number of replicates
  for control and experimental treatments
• Remember to include these data in your own
  articles!

12
3) Which data points?

• Data points are assumed to be independent.
• Different species OK (but beware Bazzaz
  effect)
• Different treatments OK (e.g. fertilized /
  unfertilized)
• What about repeated measures??
• Curtis Wang 1 data point from each
  experiment mid-growing season, final year of
  experiment
• Ainsworth Rogers all measurements (98 data
  points from 4 experiments!)

13
Different leaves?

• What if data are presented separately for
  different leaf categories, e.g.
• age class
• aspect
• canopy height
• Try to make comparable across studies.

e.g. Barker et al. 2005 CO2 and photoinhibition
in snow gum
14
A current problem

• Duke FACE experiment
• pine overstorey
• sweetgum understorey, some emergents
• pine gs measured on large number of trees each
  year
• sweetgum gs measured on 2 trees/ring, once
• BUT same number of replicates
• How to weight?

15
4) Calculating effect size
You can use a number of different statistics to
represent effect size e.g. correlation
coefficient r (ask Chris Lusk) Hedges d
standard deviation units
What does d mean? d effect size 0 none
0.2 small 0.5 medium 0.8 large
Xis are means of experimental (E) control (C)
groups si is the pooled standard deviation of
both groups
16
Ln Response Ratio

• A metric that many ecologists are comfortable
  with.
• Response ratio r XiE / XiC
• Ln response ratio L ln (r)
• The variance of L is approximately equal to

Confidence interval is given by
Hedges et al. 1999
17
Presenting Data
Rustad et al. 2001
18
5) Combining effect sizes

• Fixed effects model
• Studies have a common true effect size
• (differences just due to sampling error)
• Mixed effects model
• Studies have a common mean effect
• Random variation as well as sampling error
• (Most appropriate in ecological studies)

19
Combining effect sizes Fixed Model

• The cumulated effect size across all expts is a
  weighted average of the effect size estimates
• Weights wi are the reciprocals of the sampling
  variances, wi 1/vi.
• Variance of d is
• Can then construct a confidence interval for d

20
Combining effect sizes Mixed Model

• We need to modify the variance
• vi vi s2pooled
• where s2pooled is the pooled within-class
  variance.
• (Formula Gurevitch Hedges 1993)
• The weights wi are the reciprocals of the
  sampling variances, wi 1/vi
• Weights can be modified to avoid undue influence.

21
6) Division into Classes

• Effect of high CO2 might vary among studies e.g.
  by
• functional group
• nutrient availability
• exposure time
• Can test for differences among classes
• calculate between-class heterogeneity
• QB Si1 to m Sj 1 to kiwij (di - d)

22
Presenting Comparisons
Effects of high CO2 on gs in FACE
Ainsworth Rogers 2007
23
Beware Confounding
Medlyn et al. 2001
Number of studies Mature Young Coniferous 6
2 Broadleaf evergreen 3 0 Broadleaf
deciduous 1 17
24
Philosophical Issues

• What does meta-analysis really tell us?
• Is meta-analysis a Good Thing??

Luo et al. 2006 Effects of high CO2 on carbon
accumulation in plant biomass high variability
ln
25
References

• Gurevitch, J. and L.V. Hedges, 1993.
  Meta-analysis Combining the results of
  independent experiments. Pages 378-398 in S.M.
  Scheiner and J. Gurevitch, editors. Design and
  Analysis of Ecological Experiments. Chapman and
  Hall, New York.
• Rosenberg, M.S., D.C. Adams, and J. Gurevitch.
  1997. MetaWin Statistical Software for
  Meta-Analysis with Resampling Tests. Version 1.0.
  Sinauer Associates, Sunderland, Massachusetts.
• Hedges LV, Gurevitch JC, Curtis PS (1999) The
  meta-analysis of response ratios in experimental
  ecology. Ecology 80 1150-1156.
• Hunter JE, Schmidt FL (2004) Methods of
  Meta-Analysis, 2nd ed. HA 29.H847
• Cooper H, Hedges LV (1994) The Handbook of
  Research Synthesis. Q180.55.M4.H35