Impact Assessment

1
Impact Assessment

• ESM 206A
• 21 November 2007

2
Impact assessment

• What is the impact of one or more management
  techniques on an environmental variable of
  interest?
• Effects of grazing on biodiversity in California
  grasslands
• Establishment of marine protected areas in
  Channel Islands how do they affect lobster
  abundance?
• Has some event caused a change in the
  environment?
• Power plant goes on line
• Hurricane
• Challenge is to establish causation
• This is a conceptual issue, not a statistical
  issue!

3
San Onofre Nuclear Power Plant

• Units 2 3 came online in May 1983
• Cooling water discharge creates turbidity due to
  volume of flow
• This has potential negative effects on giant
  kelp
• Reduced light at sea floor reduces growth and
  survival of baby kelp
• Sediments bury hard substrate
• Data collected on total area of kelp forest,
  using sidescan sonar
• Question did the power plants negatively impact
  the kelp bed adjacent to the cooling water
  discharge site?

4
Before-After (BA)

• Suppose we have data from power plant outflow
  site before and after operation began
• Is kelp area lower after the power plant went
  online?

5
Two-sample t-test

• If two samples, X and Y, are from populations
  with the same mean
• Then the quantity
• follows a t distribution with
  nx ny 2 degrees of freedom

6
Assumptions of 2-sample t-test

• Populations from which x and y are sampled are
  normally distributed
• Test is pretty robust as long as sample sizes are
  similar and 2-tailed tests are being considered
• The more samples, the better
• If non-normality is strong, dont put confidence
  in alpha levels below 0.01
• Populations from which x and y are sampled have
  same variances
• Violations mean that P values will be somewhat
  too small
• Use Welchs approximate t test instead

7
Welchs approximate t-test

• If two samples, X and Y, are from populations
  with the same mean but different variances
• Then the quantity
• follows a t distribution with
• degrees of freedom

8
BA (continued)
9
BA (conclusion)

• The assumptions of the t-test are moderately
  violated
• However, P value is extremely small we can be
  confident that the true P is less than a
  reasonable alpha (0.05 or even 0.01)
• Thus, with high confidence we can reject the null
  hypothesis that the kelp area was the same before
  and after the power plant went online
• Does this mean the power plant has caused this
  difference?

10
Control-Impact (CI)

• Suppose we have data from power plant outflow
  site and a control site nearby, but only after
  operation began
• Is kelp area lower in impact site than control
  site?

11
Again, do a 2-sample t-test
12
CI

• Conclusion Fail to reject the null hypothesis
  that Control and Impact sites have same amount of
  kelp!
• Does this mean that power plant has no effect on
  kelp?

13
Before-After Control-Impact (BACI)

• If we have data from both sites (Control and
  Impact) at both periods (Before and After) then
  we can
• Use the Control site to control for temporal
  changes in kelp that are unrelated to the power
  plant coming on line
• Also called a counterfactual
• Use the Before period to determine the relative
  quality of the two sites before the power plant
• We want to focus on the difference of
  differences

14
Estimate the effect size
15
Before
After
16
Combining all the information simple BACI using
ANOVA

• ANOVA (ANalysis Of VAriance) allows us to
  simultaneously compare the means of more than two
  groups
• Single factor ANOVA simply compare among groups
  (e.g., streamflow at five streams)
• Null hypothesis all samples are drawn from
  populations with the same means
• Alternate hypothesis the samples are not all
  drawn from population with the same means
• We could use this compare our four groups (BC,
  BI, AC, AI), but thats not very interesting

17
Multi-factor ANOVA

• Each factor is a way of classifying observations,
  and has two or more levels
• E.g., environmental attitude might differ among
  nationalities (Canadian, American, Mexican) and
  religions (Catholic, Protestant, Jew, Muslim,
  None)
• ANOVA looks at main effects and interactions
• Main effects averaging across all levels of
  factor B, do the levels of factor A have
  different means?
• Do people from Mexico, US and Canada differ in
  their environmental attitudes, regardless of
  religion?
• Interactions Do the differences between levels
  of factor A depend on which level of factor B you
  are in?
• Is the difference in attitudes between Jews and
  Catholics different in the three countries?

18
ANOVA in Rcmdr

• Statistics -gt Means -gt Multi-way ANOVA

19
BACI with ANOVA
Anova Table (Type II tests) Response Kelp
Sum Sq Df F value Pr(gtF) Period
7312 1 11.0990 0.0016300 Site
4779 1 7.2543 0.0096014 PeriodSite 11259
1 17.0902 0.0001359 Residuals 32939 50
--- Signif. codes 0 ''
0.001 '' 0.01 '' 0.05 '.' 0.1 ' ' 1
Averaging over sites, kelp differs between periods
Averaging over periods, kelp differs between sites
The difference in period means depends on which
site you are in or The difference in site means
depends on which period you are in
20
BACI

• Conclusion Reject the null hypothesis that the
  difference between the impact and control sites
  did not change from Before to After
• P 0.0001
• The change was negative relative to control
  site, kelp density in impact site was lower after
  the power plant went online
• Does this mean that power plant has caused this
  change?

21
We can do even better!
22
Before-After Control-Impact Paired Series (BACIPS)

• Calculate deltas for each sample time by
  subtracting value at control site from value at
  impact site
• controls for temporal variability in environment
• Do a two-sample t-test to see if mean delta
  changes from before to after

23
BACIPS continued
24
BACI BACIPS conclusions

• Reject null hypothesis that true effect size is
  zero
• BACIPS gives more power than BACI (much smaller
  P)
• Could have drawn conclusion sooner
• Gives more confidence under violation of
  assumptions
• Does this mean the power plant has caused this
  effect?

25
Impact assessment general considerations

• We need some way to control for variability that
  might confound our conclusions
• Temporal changes unrelated to event in question
• Differences in underlying quality between sites
• Can also control by including measurements of
  additional variables that might affect response
• SST, amount of rocky reef, etc.

• What if monitoring only starts after event occurs?

26
Environmental challenge

• The problem There is a great deal of controversy
  about how to manage California grasslands for
  biodiversity values. In particular, there is
  debate about whether grazing by cattle is
  promotes or is deleterious to plant diversity
• Data on plant diversity and relative abundance
  have been collected from plots at Sedgwick
  Reserve where grazing has been either allowed or
  excluded.
• Your job Using these data, determine whether
  grazing increases or decreases plant diversity.

27
Diversity in Sedgwick grassland plots
28
2-sample t-test of H
29
Paired t-test

• If observations naturally come in pairs
• Control treatment plots next to each other
• Calculate differences, di xi - yi
• Use one-sample t-test to test H0 md 0

30
Paired t-test
31
2-sample vs paired t-test

• TWO SAMPLE t-TEST
• Difference of means
• Use when observations are independent between
  groups
• Assumes each population is normally distributed

• PAIRED t-TEST
• Mean of differences
• Use when observations are naturally paired
• Assumes population of differences is normally
  distributed

• Both will estimate same mean difference
• If there is variation among pairs (e.g., due to
  location, soil type, habitat), then paired test
  will have more power to reject null hypothesis

32
Independence and randomness
33
Further Reading

• Schmitt, R.J., and C.W. Osenberg, eds. 1996.
  Detecting Ecological Impacts. Academic Press, San
  Diego.
• Stewart-Oaten, A., W.W. Murdoch, and K.R. Parker.
  1986. Environmental impact assessment
  pseudoreplication in time? Ecology 67 929-940.
• Helsel Hirsch Chapters 5-7.