Meta Analysis

1
Meta Analysis

2
What is meta-analysis?
the combined analysis and quantitative synthesis
of multiple studies
Traditionally (medical applications) relied on
using summary statistics
In fisheries meta-analysis often uses the data
from individual studies
3
Types of meta-analysis

• The literature review Gerber Hilborn
• Vote counting Myers depensation
• The histogram of frequencies of parameter
  estimates
• Pauly 1980 synthesis of natural mortality vs
  temperature and k
• Hierarchic models

4
History of fisheries formal meta-analysis
5
Citations
Dorn, M. W. (2002). "Advice on West Coast
Rockfish harvest rates from Bayesian
meta-analysis of stock-recruit relationships."
North American Journal of Fisheries Management
22 280-300. DuMouchel, W. H. and S.-L. T.
Normand (2000). Computer modeling strategies for
meta-analysis. Meta-analysis in medicine and
health policy. D. Stang and D. Berry, Marcel
Dekker 127-178. Gelman, A., J. B. Carlin, H. S.
Stern and D. B. Rubin (1995). Bayesian Data
Analysis. Boca Raton, Chapman
Hall/CRC. Gurevitch, J., P. S. Curtis and M. H.
Jones (2001). "Meta-analysis in Ecology."
Advances in Ecological Research 32
199-247. Harley, S. H. and R. A. Myers (2001).
"Hierarchical Bayesian models of length-specific
catchability of research trawl surveys." Canadian
Journal of Fisheries and Aquatic Sciences 58
1569-1584. Harley, S. H., R. A. Myers and A. Dunn
(2001). "Is catch-per-unit-effort proportional to
abundance?" Canadian Journal of Fisheries and
Aquatic Sciences 58 1760-1772. Hedges, L. V. and
I. Olkin (1985). Statistical methods for
meta-analysis. Orlando, Academic Press,
Inc. Hilborn, R. and M. Liermann (1998).
"Standing on the shoulders of giants learning
from experience in fisheries." Reviews in Fish
Biology and Fisheries 8 273-283. Liermann, M.
and R. Hilborn (1997). "Depensation in fish
stocks a hierarchic Bayesian meta-analysis."
Canadian Journal of Fisheries and Aquatic
Sciences 54 1976-1984. Mann, C. (1990).
"Meta-analysis in the breech." Science 249
479-480. Millar, C. and R. D. Methot (2002).
"Age-structured meta-analysis of U.S. West Coast
rockfish (Scorpaenidae) populations and
hierarchical modeling of trawl survey
catchabilities." Canadian Journal of Fisheries
and Aquatic Sciences 59 383-392. Myers, R. A.
(2000). "The synthesis of dynamic and historical
data on marine populations and communities
putting dynamics into the Ocean Biogeographical
Information System (OBIS)." Oceanography 13(3)
56-59. Myers, R. A. (2001). "Stock and
recruitment generalizations about maximum
reproductive rate, density dependence, and
variability using meta-analytic approaches." ICES
Journal of Marine Science 58 937-951. Myers, R.
A., N. J. Barrowman, R. Hilborn and D. G. Kehler
(2002). "Infering Bayesian priors with limited
direct data applications to risk analysis."
North American Journal of Fisheries Management
22 351-364. Myers, R. A., K. G. Bowen and N. J.
Barrowman (1999). "Maximum reproductive rate of
fish at low population sizes." Canadian Journal
of Fisheries and Aquatic Sciences 56
2404-2419. Myers, R. A., B. R. MacKenzie, K. G.
Bowen and N. J. Barrowman (2001). "What is the
carrying capacity for fish in the ocean? A
meta-analysis of population dynamics of North
Atlantic cod." Canadian Journal of Fisheries and
Aquatic Sciences 58 1464-1476. Pinheiro, J. C.
and D. M. Bates (2000). Mixed-effects models in S
and S-PLUS. New York, Springer. Wachter, K. W.
(1988). "Disturbed by meta-analysis?" Science
241 1407-1408.
6
Historical motivation

• Mann 1990 medical , Gurevitch et al. 2001 ecology
• Synthesis of multiple replicated experiments
• There are often contradictory results in the set
  of experiments
• Critique (Wachter 1988)
• How to decide what studies to include

7
An example Myers et al 1995 depensation
8
Myers Analysis
9
Myers results

• Explored over 100 data sets
• Found few significant cases of depensation
• Fewer than expected by chance
• Of these data sets about 27 had high power

10
Problems with Myers method

• Parameterization has no biological interpretation
  except dgt1 implies depensation
• Used p values to test for significant
  depensation, ignores biological significance

11
Other issues

• Reanalyzed the data sets rather than relying on
  summary statistics
• The data were generally stock assessment
  results, so the uncertainty in the data was not
  represented accurately
• Which stocks to choose
• Selection bias against unproductive stocks

12
Results from Myers et al.

• The obvious deficiencies of non-interchangability
  of depensation parameter and the use of p-values
• Led to Liermann and Hilborn using hierarchic
  Bayesian methods

13
Fixed effects, random effects, and mixed models
14
Model I all populations the same
15
Adding population effects

• Assume that each population has its own average
  with observation error around it

16
Model II fixed effects
17
This is a fixed effect model

• We assume that each population mean
• No relationship between the populations, they are
  fixed effects

18
A random effects model

• Assume that the populations are drawn from a
  population of population, and there is a
  distribution of means in the overall population
  of populations

19
Model III Random effects
20
Use simple data set to explore all three types of
models and show how mixed effects model needs
integration
21
Hierarchic meta analysis the simple data set

• Assume that the population mean is normally
  distributed across all stocks
• The hyper parameters are a mean and a variance

22
The parameters to estimate

• The mean for each population , the standard
  deviation assumed the same for all populations
• The hyper mean and hyper sd

23
The likelihoods

• For each population the likelihood of the
  observed data
• For each populations ?, the likelihood of it
  given the hyper mean and hyper s.d.
• The likelihood of the hyper mean and hyper s.d.
  given the priors for them, and sigma (within
  population s.d.)

24
The hyper distribution
25
Symbols
26
The likelihood for an individual observation for
a population
27
The Priors
28
The process

• Do MCMC on all the parameters
• the hyper mean
• the hyper s.d.
• the parameters for the individual populations
  including ?

29
What then

• You end up with a posterior distribution of both
  the hyper mean, and the hyper s.d.
• You then need to integrate across these two
  dimensions to get the posterior distribution on
  the initial slope that is the parameter of
  interest

30
Liermanns trick

• Define a new stock that has no data
• Then calculate the posterior distribution for
  this stock - this posterior distribution will be
  the posterior distribution for the parameter for
  an unknown stock.