Linear Contrasts and Multiple Comparisons Chapter 9

1
Linear Contrasts and Multiple Comparisons(Chapte
r 9)

• One-way classified design AOV example.
• Develop the concept of multiple comparisons and
  linear contrasts.
• Multiple comparisons methods needed due to
  potentially large number of comparisons that may
  be made if Ho rejected in the one-way AOV test.

Terms Linear Contrasts Multiple comparisons Data
dredging Mutually orthogonal contrasts Experimentw
ise error rate Comparisonwise error rate
MCPs Fishers Protected LSD Tukeys W
(HSD) Studentized range distribution Student-Newma
n-Keuls procedure Duncans new MRT Dunnetts
procedure
2
One-Way Layout Example
A study was performed to examine the effect of a
new sleep inducing drug on a population of
insomniacs. Three (3) treatments were
used Standard Drug New Drug Placebo (as a
control)
What is the role of the placebo in this
study? What is a control in an experimental study?
18 individuals were drawn (at random) from a list
of known insomniacs maintained by local
physicians. Each individual was randomly
assigned to one of three groups. Each group was
assigned a treatment. Neither the patient nor
the physician knew, until the end of the study,
which treatment they were on (double-blinded).
Why double-blind?
A proper experiment should be randomized,
controlled, and double-blinded.
3
Response
Average number of hours of sleep per night.
Placebo 6.5, 5.7, 5.1, 3.8, 4.6, 5.1 Standard
Drug 8.4, 8.2, 8.8, 7.1, 7.2, 8.0 New
Drug 10.6, 6.6, 8.0, 8.0, 6.8, 6.6
yij response for the j-th individual on the
i-th treatment.
Hartleys test for equal variances Fmax 4.77
lt Fmax_critical 10.8
4
Excell Analysis Tool Output
What do we conclude here?
5
Linear Contrasts and Multiple Comparisons
If we reject H0 of no differences in treatment
means in favor of HA, we conclude that at least
one of the t population means differs from the
other t-1.
Which means differ from each other?
Multiple comparison procedures have been
developed to help determine which means are
significantly different from each other.
Many different approaches - not all produce the
same result.
Data dredging and data snooping - analyzing only
those comparisons which look interesting after
looking at the data affects the error rate!
Problems with the confidence assumed for the
comparisons
1-a for a particular pre-specified
comparison? 1-a for all unplanned comparisons as
a group?
6
Linear Comparisons
Any linear comparison among t population means,
m1, m2, ...., mt can be written as
Where the ai are constants satisfying the
constraint
7
Linear Contrast
A linear comparison estimated by using group
means is called a linear contrast.
Ho l 0 vs. Ha l ? 0
8
Orthogonal Contrasts
These two contrasts are said to be orthogonal if
in which case l1 conveys no information about l2
and vice-versa.
A set of three or more contrasts are said to be
mutually orthogonal if all pairs of linear
contrasts are orthogonal.
9
Compare average of drugs (2,3) to placebo
(1). Contrast drugs (2,3).
Orthogonal
Non-orthogonal
Contrast Standard drug (2) to placebo
(1). Contrast New drug (3) to placebo (1).
10
Drug Comparisons
11
Importance of Mutual Orthogonality
Assume t treatment groups, each group having n
individuals (units).

• t-1 mutually orthogonal contrasts can be formed
  from the t means. (Remember t-1 degrees of
  freedom.)
• Treatment sums of squares (SSB) can be computed
  as the sum of the sums of squares associated with
  the t-1 orthogonal contrasts. (i.e. the treatment
  sums of squares can be partitioned into t-1 parts
  associated with t-1 mutually orthogonal
  contrasts).

t-1 independent pieces of information about the
variability in the treatment means.
12
Example of Linear Contrasts
Objective Test the wear quality of a new
paint. Treatments Weather and wood combinations.
Treatment Code Combination A m1 hardwood, dry
climate B m2 hardwood, wet climate C m3 softwood,
dry climate D m4 softwood, wet climate
(Obvious) Questions
Q1 Is the average life on hardwood the same as
average life on softwood? Q2 Is the average
life in dry climate the same as average life in
wet climate? Q3 Does the difference in paint
life between wet and dry climates depend upon
whether the wood is hard or soft?
13
Q1
Q1 Is the average life on hardwood the same as
average life on softwood?
Comparison
Estimated Contrast
Test H0 l1 0 versus HA l1 ? 0
What is MSl1 ?
Test Statistic
Rejection Region Reject H0 if
14
Conclusion Since F29.4 gt 5.32 we reject H0 and
conclude that there is a significant difference
in average life on hard versus soft woods.
15
Q2
Q2 Is the average life in dry climate the same
as average life in wet climate?
Comparison
Estimated Contrast
Test H0 l2 0 versus HA l2 ? 0
Test Statistic
Rejection Region Reject H0 if
16
Conclusion Since F0.6 lt 5.32 we do not reject
H0 and conclude that there is not a significant
difference in average life in wet versus dry
climates.
17
Q3
Q3 Does the difference in paint life between wet
and dry climates depend upon whether the wood is
hard or soft?
Comparison
Estimated Contrast
Test H0 l3 0 versus HA l3 ? 0
Test Statistic
Rejection Region Reject H0 if
18
Conclusion Since F0 lt 5.32 we do not reject H0
and conclude that the difference between average
paint life between wet and dry climates does not
depend on wood type. Likewise, the difference
between average paint life for the wood types
does not depend on climate type (i.e. there is no
interaction).
19
Mutual Orthogonality
20
Types of Error Rates
Compairsonwise Error Rate - the probability of
making a Type I error in the comparison of two
means. (what we have been discussing for all
tests up to this point).
Experimentwise Error Rate - the probability of
observing an experiment in which one or more of
the pairwise comparisons are incorrectly declared
significantly different. (Type I error.)
21
Error Rates Problems
Suppose we make c mutually orthogonal
(independent) comparisons, each with Type I
comparisonwise error rate of a. The
experimentwise error rate, e, is then
(If the comparisons are not orthogonal, then the
experimentwise error rate is smaller.) Solution
(Bonferroni) set e0.05 and solve for ?. But
theres a problem E.g. if c8, we get ?0.0064!
22
Multiple Comparison Procedures

• Terms
• If the multiple comparison procedure (MCP)
  requires a significant overall F test, then the
  procedure is labeled a Protected method.
• Not all procedures produce the same results.
• The major differences among all of the different
  MCPs is in the calculation of the yardstick
  used to determine if two means are significantly
  different. The yardstick can generically be
  referred to as the least significant difference.
  Any two means greater than this difference are
  declared significantly different.

• Yardsticks are composed of a standard error term
  and a critical value from some tabulated
  statistic.
• Some procedures have fixed yardsticks, some
  have variable yardsticks. The variable
  yardsticks will depend on how far apart two
  observed means are in a rank ordered list of the
  mean values.
• Some procedures control Comparisonwise Error,
  other Experimentwise Error, and some attempt to
  control both.

23
Fishers Least Significant Difference - Protected
Mean of group i (mi) is significantly different
from the mean of group j (mj) if
if all groups have same size n.
tabled valuestandard error of difference
Type I (comparisonwise) error rate a
This procedure controls Comparisonwise Error.
Experimentwise error control comes from requiring
a significant overall F test prior to performing
any means comparisons.
How well does it work?
24
Tukeys W (Honestly Significant Difference)
Procedure
Primarily suited for all pairwise comparisons
among t means. Means are different if
Table 10 - critical values of the studentized
range.
Experimentwise error rate a
This MCP controls experimentwise error rate!
Comparisonwise error rates is thus very low.
How well does it work?
25
Student Newman Keul Procedure
A modified Tukeys MCP. Rank the t sample means
from smallest to largest. For two means that are
r steps apart in the ranked list, we declare
the population means different if
Table 10 - critical values of the studentized
range. Depends on which mean pair is being
considered!
varying yardstick
r6
r5
r2
r3
r4
26
Duncans New Multiple Range Test
Neither an experimentwise or comparisonwise error
rate control alone. Based on a ranking of the
observed means. Introduces the concept of a
protection level (1-a)r-1
Table A -11 (later) in these notes
27
Dunnetts Procedure
A MCP that is used for comparing treatments to a
control. It aims to control the experimentwise
error rate. Compares each treatment mean (i) to
the mean for the control group (c).

• da(k,v) is obtained from Table A-11 (in the book)
  and is based on
• a the desired experimentwise error rate
• k t-1, number of noncontrol treatments
• v error degrees of freedom.

28
Scheffés S Method
For any linear contrast
Estimated by
With estimated variance
To test H0 l 0 versus Ha l ¹ 0
For a specified value of a, reject H0 if
where
Confidence interval
29
Geometric Mean

• If the sample sizes are not equal in all groups,
  the value of n in the previous equations is
  replaced with the geometric mean of the sample
  sizes

E.g. Tukeys procedure becomes
30
Comparisonwise error rates for different MCP
31
Experimentwise error rates for different MCP