Multiple comparisons

1
Multiple comparisons

• - multiple pairwise tests
• - orthogonal contrasts
• - independent tests
• - labelling conventions

2
Card example number 1
3
Multiple tests
Problem Because we examine the same data in
multiple comparisons, the result of the first
comparison affects our expectation of the next
comparison.
4
Multiple tests
ANOVA shows at least one different, but which
one(s)?

• T-tests of all pairwise combinations

significant
significant
Not significant
5
Multiple tests
6
Multiple tests
Solution Make alpha your overall
experiment-wise error rate
T-test lt5 chance that this difference was a
fluke
affects likelihood (alpha) of finding a
difference in this pair!
7
Multiple tests
Solution Make alpha your overall
experiment-wise error rate e.g. simple
Bonferroni Divide alpha by number of tests
8
Card example 2
9
Orthogonal contrasts
Orthogonal perpendicular independent Contras
t comparison
Example. We compare the growth of three types of
plants Legumes, graminoids, and asters. These
2 contrasts are orthogonal 1. Legumes vs.
non-legumes (graminoids, asters) 2. Graminoids
vs. asters
10
Trick for determining if contrasts are
orthogonal 1. In the first contrast, label all
treatments in one group with and all
treatments in the other group with - (doesnt
matter which way round).
Legumes Graminoids Asters
- -
11
Trick for determining if contrasts are
orthogonal 1. In the first contrast, label all
treatments in one group with and all
treatments in the other group with - (doesnt
matter which way round). 2. In each group
composed of t treatments, put the number 1/t as
the coefficient. If treatment not in contrast,
give it the value 0.
Legumes Graminoids Asters 1
- 1/2 -1/2
12
Trick for determining if contrasts are
orthogonal 1. In the first contrast, label all
treatments in one group with and all
treatments in the other group with - (doesnt
matter which way round). 2. In each group
composed of t treatments, put the number 1/t as
the coefficient. If treatment not in contrast,
give it the value 0. 3. Repeat for all other
contrasts.
Legumes Graminoids Asters 1
- 1/2 -1/2 0 1
-1
13
Trick for determining if contrasts are
orthogonal 4. Multiply each column, then sum
these products.
14
Trick for determining if contrasts are
orthogonal 4. Multiply each column, then sum
these products. 5. If this sum 0 then the
contrasts were orthogonal!
15
What about these contrasts? 1. Monocots
(graminoids) vs. dicots (legumes and asters). 2.
Legumes vs. non-legumes
16
Important!

• You need to assess orthogonality in each pairwise
  combination of contrasts.
• So if 4 contrasts
• Contrast 1 and 2, 1 and 3, 1 and 4, 2 and 3, 2
  and 4, 3 and 4.

17
How do you program contrasts in JMP (etc.)?
Treatment SS

Contrast 1

Contrast 2
18
How do you program contrasts in JMP (etc.)?
Legumes vs. non-legumes
Normal treatments
There was a significant treatment effect (F).
About 53 of the variation between treatments was
due to differences between legumes and
non-legumes (F1,20 6.7).
Legume 1 1 Legume 1 1 Graminoid 2 2 Graminoid 2 2
Aster 3 2 Aster 3 2 SStreat
122 67 Df treat 2 1 MStreat 60 MSerror 10 Df
error 20
19
Even different statistical tests may not be
independent ! Example. We examined effects of
fertilizer on growth of dandelions in a pasture
using an ANOVA. We then repeated the test for
growth of grass in the same plots. Problem?
20
Multiple tests
b
Convention Treatments with a common letter are
not significantly different
a,b
a
significant
Not significant
Not significant
21
(No Transcript)
22
(No Transcript)