Repeated Measures ANOVA

1
Repeated Measures ANOVA

• Starting with One-Way RM
• More fascinating than a bowl of porridge. Really

2
One-Way Repeated Measures ANOVA

• Data considerations
• One continuous dependent variable (Likert type
  data also acceptable)
• One nominal independent variable of gt 2 levels
• Analogous to dependent t-test, but for more than
  2 levels of the independent variable

3
One-Way Repeated Measures ANOVA

• Advantages of repeated measures
• Again, as per paired t-test...
• Sensitivity
• Reduction in error variance (subjects serve as
  own controls)
• So, more sensitive to experimental effects
• Economy
• Need less participants
• With many levels, this might be even more
  important for ANOVA than t-test
• (need to be careful of fatigue effects, though)

4
One-Way Repeated Measures ANOVA

• Possible uses of 1-way RM ANOVA
• Same people measured 3 times
• Pre-test, post-test, follow-up
• Same people measured under three or more
  different treatments
• Drug 1, drug 2, drug, 3

5
One-Way Repeated Measures ANOVA

• Possible serious disadvantage of RM
• Order effects and treatment carry-over effects
  (goes for paired t-test too)
• Should counterbalance (by random assignment to
  order of treatment)
• E.g. (for 2 levels of RM A B)

Order of administration Order of administration
1 2
of subjects 50 A B
of subjects 50 B A
6
One-Way Repeated Measures ANOVA

• Possible serious disadvantage of RM
• Order effects and treatment carry-over effects
  (goes for paired t-test too)
• E.g. (for 3 levels of RM A, B C)

This type of control for order effects is known
as a Latin Square design
Order of administration Order of administration Order of administration
1 2 3
of subjects 33 A B C
of subjects 33 B C A
of subjects 33 C A B
7
One-Way Repeated Measures ANOVA

• Possible serious disadvantage of RM
• Treatment carry-over effects (goes for paired
  t-test too)
• Even if order effects are controlled for, there
  must be sufficient time between treatments so
  that you can be sure that the score on each level
  of the RM is due to only one treatment (not a
  combination of two or more)
• Note order treatment carry-over effects are
  design rather than statistical issues, but very
  important nevertheless

8
One-Way Repeated Measures ANOVA

• Example, with chat about variance partitioning
  and assumptions...
• Remember the one-way between subjects ANOVA?
• Data looked like this in SPSS
• And the trick was to make variance due to
  treatments bigger than variance due to everything
  else ( everything else included variance due to
  individual differences)
• Well, what if you could take out variance due to
  individual differences?

9
One-Way Repeated Measures ANOVA

• Thats what the one-way RM ANOVA does
• Data now looks like this, as each person is
  measured on all levels of the IV
• Variation due to individual differences can then
  be separated from variation due to chance, as the
  same people are present within each condition.
• It goes something like this...(cue horror movie
  music)

10
One-Way Repeated Measures ANOVA
Heres the output from the between subject
ANOVA. Note the size of the error term (within
groups variance measure). That really diminishes
the F-size. And no-one likes a small F-size.
11
One-Way Repeated Measures ANOVA

• Now a pause before we consider variance
  partitioning in RM ANOVA, as we see how to
  conduct the wee devil. Suffice to say Ill be
  keeping it simple.
• Heres the first step

Choose this...
12
One-Way Repeated Measures ANOVA
You have to specify the independent variable, and
the number of levels it has (4 here)

• Now...

Then click add and proceed by clicking define
13
One-Way Repeated Measures ANOVA
...next (long process here) you choose all the
levels of the repeated measures factor (i.v.)...

• Then...

And slide them over to the within subjects
variables box just another name for repeated
measures variable...or factor
14
One-Way Repeated Measures ANOVA

• And...output!

This first bit is from the multivariate (more
than one dependent variable 4 here) approach to
repeated measures. It has some potential
advantages (essentially that one does not have to
meet the sphericity assumption...see next slide)
15
One-Way Repeated Measures ANOVA

• And...more output...

This bit is important. Its a test of one of the
more important assumptions of RM ANOVA
sphericity. Its kind of like the homogeneity of
variance test, but its the variance of the
difference scores between the levels of the
independent variable that are being testedyou
really have to adjust for it, we see how on the
next slide (if this is NOT significant, its
good)
Another important bitthe Huynh-Feldt
Epsilon...see next slide
16
One-Way Repeated Measures ANOVA

• And...still more output...

Finally, the bit that counts. Note there are FOUR
(count em) separate versions of each effect.
Heres the rule (Schutz Gessaroli, 1993) If
Huynh-Feldt Epsilon (see previous slide) is gt .7,
use Huynh-Feldt adjusted F (third line). If it is
less than .7, use G-G (second line)
17
One-Way Repeated Measures ANOVA

• Same bit once again...

Here, you can see that, as the epsilon is 1,
there is no correction, and the F statistic stays
the same throughout. Now, what about that
variance partitioning???? Remember we were going
to talk about that?
18
One-Way Repeated Measures ANOVA

• One last bit (that you can ignore)...

Lets just look at this first. In the top box,
you can see a bunch of stuff like linear,
quadratic, cubic thats to do with the
shape of the difference that the change in scores
might take as they progress from drug 1 to drug
4, and only really makes sense in trend analysis,
which is again beyond our scope.
Finally, down here you see between subjects
effects. There are none here (just one I.V., and
its RM). The error variance here is essentially
a measure of individual differences, as well see
in a minute...
19
One-Way Repeated Measures ANOVA

• So, how does the variance thing work?
• Lets compare the two methods (between subjects
  and repeated measures) directly, bearing in
  mind where the variances in the output tables
  come from
• In this way, my goal is simply to indicate the
  benefit of taking out variation due to individual
  differences
• Well start with the between subjects
  method...(see next slide)

20
One-Way Repeated Measures ANOVA
Here the between groups variance is 698.2 this
is just variation of mean scores on the different
treatments about the overall mean...so this is
the bit that is essentially the treatment effect
And here is the within subjects variation...it is
calculated from the sum of the variation within
each of the treatments about each of the
treatment means...so its like a combination of
variation due to individual differences,
variation due to treatments, and variation within
treatment across individuals (what?)
26.4
25.6
15.6
32
21
One-Way Repeated Measures ANOVA

• Now for the repeated measures version

Note that the average score for each subject
across the four treatments is different. This is
due to individual differences...and is the
between subjects error variance
27
16
23
34
24.5

• The thing that makes repeated measures powerful
  is that this variation is taken out of the within
  subjects error term...see next slide

Sum of squares 680.8 ( sum of squared
deviations from the mean of these 5 scores, which
is 24.9, multiplied by the levels of the I. V.))
22
One-Way Repeated Measures ANOVA
An example from excel
23
One-Way Repeated Measures ANOVA
Now what you have to see is that the SS for the
denominator in the F test in RM is now 112.8,
which is derived from 793.6 680.8 112.8
individual differences
Error variation in between subjects ANOVA
24
One-Way Repeated Measures ANOVA
And finally, as a direct consequence of all this,
the numerator in the F-test is unchanged (698.2),
but the denominator has been reduced from 49.6 to
9.4, resulting in an increase in F from 4.69 to
24.76!
which of course means...more significance, more
power
25
One-Way Repeated Measures ANOVA

• So, to summarize
• Because of the way RM ANOVA partitions variance
  for the RM factors, we have a far more powerful
  test for the RM factors
• But you have to make sure you control for
  spurious effects by controlling for order effects
  and carryover effects

26
Example of interpretation of results
Note partial ?2 is reported too

• Interpretation
• A one-way repeated measures ANOVA was conducted
  to students confidence in statistics prior to
  the class, immediately following the class, and
  three months after the class. Due to a mild
  violation of the sphericity assumption (? .82),
  the Huynh-Feldt adjusted F was used. There was a
  significant difference in confidence levels
  across time, F (1.421, 41.205) 33.186, p lt
  .001, partial ?2 .86. Dependent t-tests were
  used as post-hoc tests for significant
  differences with Bonferroni-adjusted ? .017.
  Confidence levels after three months (M 25.03,
  SD 5.20) were significantly higher than
  immediately following the class (M 21.87, SD
  5.59), which in turn were significantly higher
  than pre-test levels (M 19.00, SD 5.37).

27
ANOVA/Inferential Statistics Wrap-up

• Inferential tests to compare differences in
  groups
• Independent t-tests ?
• Dependent t-tests ?
• One-way ANOVA ?
• Factorial ANOVA ?
• One-way repeated measures ANOVA ?
• Factorial repeated measures ANOVA
• Mixed between-within groups ANOVA (split-plot)
• Analysis of covariance (ANCOVA)
• Multivariate analysis of variance (MANOVA)
• Nonparametric tests (next)

28
Factorial RM ANOVA

• Same notions as for factorial ANOVA main
  effects, interactions and so on
• Data set up a bit tricky

29
Two-way ANOVA with repeated measures on one factor

• Sometimes referred to as a split plot or
  Lindquist type 1 or (most commonly in my
  experience) a Two-way ANOVA with repeated
  measures on one factor.
• Research question Which diet (traditional, low
  carb, exercise only) is more effective in weight
  loss across three time periods (before diet,
  three months later, six months later)? Is there
  a weight loss across time?

30
Two-way ANOVA with repeated measures on one factor

• Diet RQ (continued)
• Looks like a 3x3 two-factor ANOVA, except that
  one of the factors is a repeated measure (one
  group of subjects tested three times)
• As such, a two-factor between-groups ANOVA is not
  appropriate rather, we have one factor that is
  between diet types and another that is within a
  single group of subjects

31
Two-way ANOVA with repeated measures on one factor

• Use a mixed design ANOVA when
• A nominal between-subjects IV with 2 levels
• A nominal within-subjects IV with 2 levels
• A continuous interval/ratio DV
• Note you can add additional IVs to this test,
  but just as with Factorial ANOVA, when you get 3
  IVs, interpreting findings gets really nasty due
  to all of the interactions

32
Two-way ANOVA with repeated measures on one factor

• Interactions like a two-factor between-subjects
  ANOVA, there may be both main effects for each of
  the two IVs plus an interaction between the two
  IVs

33
Analysis of Covariance

• An extension of ANOVA that allows you to explore
  differences between groups while statistically
  controlling for an additional continuous variable
• Can be used with a nonequivalent groups
  pre-test/post-test design to control for
  differences in pre-test scores with pre-existing
  groups
• You could use a mixed design ANOVA here, but with
  small sample size, ANCOVA may be a better
  alternative due to increased statistical power
• Be careful of regression towards the mean as a
  cause of post-test differences (after using the
  covariate to adjust pre-test scores)

34
Multivariate Analysis of Variance

• An extension of ANOVA for use with multiple
  dependent variables
• With multiple DVs, you could simply use multiple
  ANOVAs (one per DV), but risk inflated Type 1
  error
• Same reason we didnt conduct multiple t-tests
  instead of an ANOVA
• Ex. Do differences exist in GRE scores and grad
  school GPA based on race?
• There are such things as Factorial MANOVAs, RM
  MANOVAs, and even MANCOVAs

35
Finito!