Post Hoc Tests

1
Post Hoc Tests
2
What is a Post Hoc Test?

• Review
• Adjusting Alpha Level
• Multiple A Priori Comparisons
• What makes a test Post Hoc?
• Many tests could be Post Hoc But, there are set
  Post Hoc tests

3
Studentized Range Statistic q
4
Studentized Range Statistic q
Independent Groups
Largest mean
Smallest mean
Example

8.2 8.2 11.8
Fail to Reject
Note arrange means in ascending order!
3.77
critical value
5
Studentized Range Statistic q

• qs can tell us where differences are (more
  specific than F)
• Solving qs is just like solving ts
• But we solve a lot of qs can we speed things up?

6
Solving for the Smallest Significant Difference
Example

8.2 8.2 11.8
3.77
critical value
7
Solving for the Smallest Significant Difference

• Solving for the smallest significant difference
  will help us make quicker comparisons
• But we still need a way to organize things nicely

8
Newman-Kewls
smallest difference required was 6.61
smallest significant difference
If 2 steps
Example
T1 T2 T3 r
T1 0 3.6 3 6.61
T2 3.6 2 5.41
T3

8.2 8.2 11.8
9
A Better Newman-Kewls Example
Example

2 3 3 9 10
10
A Better Newman-Kewls Example
T1 T2 T3 T4 T5
T1
T2
T3
T4
T5
T1 T2 T3 T4 T5 r
T1 1 1 7 8 5 4.04
T2 0 6 7 4 3.79
T3 6 7 3 3.44
T4 1 2 2.86
T5
Read Right to Left
UNTIL
1.
The row is completed
2.
A nonsignificant difference is found
3.
Reaching a column which was nonsignificant on the
previous row
11
Newman-Kewls Summarized

• Newman-Kewls tables help organize your qs
• When doing a set of post hoc comparisons its
  best to use a Newman-Kewls table

12
Unequal Ns
Tukey-Kramer
Replace
with
13
Unequal Ns
Behrens-Fisher


Each particular pairing of means must be examined
with a different critical value and their own
Thus, the smallest significant difference will
vary even for a given
14
A Problem with q So Far

• Why are we doing qs anyway? Why not do ts
  instead?
• But is q really controlling our alpha level?
• NO!
• Using qs will give use more Type I Errors

15
Trying to fix q
Tukey's HSD
Tukey's WSD
N-K except
If there are 4 means, all differences are treated
as 4 steps.
r of steps between the two
means to be compared.
What Happens to Alpha Level? Power?
16
Tukeys HSD and WSD
T1 T2 T3 T4 T5 r
T1 1 1 7 8 5 4.04
T2 0 6 7 4 3.79
T3 6 7 3 3.44
T4 1 2 2.86
T5
Use Tukeys WSD, not normal method for q
17
Back to Post Hoc in General

• What is a post hoc test again?
• What are the real issues with Post Hoc tests?
  Alpha and Power
• q is just one type of post hoc (one way to
  balance alpha and power), what are others?

18
Dunnetts Control vs. Treatment
run standard
and use
or, solve for critical difference (CV)
Example
Go to Table for

Pros? Cons?
19
Sheffés Test
Linear contrast
To evaluate
MS(contrast)
1)
consult F table and find critical value F.05
(k-1, dferror) (CV)
F
MS(error)
2)
multiply CV by (k-1). (new CV)
It sets the family-wise Type-I Error rate (
in our case) for ALL possible linear
contrasts, not merely the pair-wise comparisons.
Dont use when only doing pair-wise, because it
will be overly conservative.
20
Post Hoc Summary

• When to use what
• q in most situations but use Tukeys WSD for
  critical value
• Put things in a Newman-Kewls table
• when Ns are unequal, use Tukeys correction
• Dunnetts when you have one control and multiple
  treatments
• Sheffés ONLY when you are doing complex
  comparisons (i.e., contrasts)

21
Post Hoc Summary

• Be aware of the alpha level and power issues
• Why cant we have a perfect test (i.e., low
  alpha level and high power)?
• How does Tukeys WSD and HSD relate to this?
• How does Dunnetts relate to this?
• How does Sheffés relate to this?