ANOVA: Analysis of Variance

1
ANOVA Analysis of Variance

• 1-way ANOVA

2
ANOVA

• What is Analysis of Variance
• The F-ratio
• Used for testing hypotheses among more than two
  means
• As with t-test, effect is measured in numerator,
  error variance in the denomenator
• Partitioning the Variance
• Different computational concerns for ANOVA
• Degrees Freedom for Numerator and Denominator
• No such thing as a negative value
• Using Table B.4
• The Source Table
• Hypothesis testing

3
M3
M1
M2
4
ANOVA

• Analysis of Variance
• Hypothesis testing for more than 2 groups
• For only 2 groups t2(n) F(1,n)

5
BASIC IDEA
Grp 1 Grp 2 Grp 3
Is the Effect Variability Large Compared to the
Random Variability
M1 1 M2 5 M3 1
Effect V

Random V

• As with the t-test, the numerator expresses the
  differences among the dependent measure between
  experimental groups, and the denominator is the
  error.
• If the effect is enough larger than random error,
  we reject the null hypothesis.

6
BASIC IDEA

• If the differences accounted for by the
  manipulation are low (or zero) then F 1
• If the effects are twice as large as the error,
  then F 3, which generally indicates an effect.

7
Sources of Variance
8
Why Is It Called Analysis of Variance?Arent We
Interested In Means, Not Variance?

• Most statisticians do not know the answer to this
  question?
• If were interested in differences among means
  why do an analysis of variance?
• The misconception is that it compares ?12 to ?22.
  No
• The comparison is between effect variance
  (differences in group means) to random variance.

9
Learning Under Three Temperature Conditions
T is the treatment total, G is the Grand total
M2
M1
M3
10
Computing the Sums of Squares
11
How Variance is Partitioned

• This simply disregards group membership and
  computes an overall SS
• Variability Between and Within Groups is
  Included

12
How Variance is Partitioned

• Imagine there were no individual differences at
  all.
• The SS for all scores would measure only the
  fact that there were group differences.

Grp 1 Grp 2 Grp 3
1 5 1 1 5 1 1 5 1 1 5 1 1 5 1
13
How Variance is Partitioned

• SS computed within a column removes the mean.
• Thus summing the SSs for each column computes
  the overall variability except for the mean
  differences between groups.

Grp 1 Grp 2 Grp 3
1-12-12-10-10-1
0-11-13-11-1 0-1
4-53-56-53-54-5
M1 1 M2 5 M3 1
14
How Variance is Partitioned
Grp 1 Grp 2 Grp 3
M1 1 M2 5 M3 1
15
Computing Degrees Freedom

• df between is k-1, where k is the number of
  treatment groups (for the prior example, 3, since
  there were 3 temperature conditions)
• df within is N-k , where N is the total number of
  ns across groups. Recall that for a t-test with
  two independent groups, df was 2n-2? 2n was all
  the subjects N and 2 was the number of groups, k.

16
Computing Degrees Freedom
17
How Degrees Freedom Are Partitioned

• N-1 (N - k) (k - 1)
• N-1 N - k k 1

18
Partitioning The Sums of Squares
19
Computing An F-Ratio
20
Consult Table B-4
Take a standard normal distribution, square each
value, and it looks like this
21
Table B-4
22
Two different F-curves
23
ANOVA Hypothesis Testing
24
Basic Properties of F-Curves
Property 1 The total area under an F-curve is
equal to 1. Property 2 An F-curve starts at 0
on the horizontal axis and extends indefinitely
to the right, approaching, but never touching,
the horizontal axis as it does so. Property 3
An F-curve is right skewed.
25
Finding the F-value having area 0.05 to its right
26
Assumptions for One-Way ANOVA

• 1. Independent samples The samples taken from
  the populations under consideration are
  independent of one another.
• 2. Normal populations For each population, the
  variable under consideration is normally
  distributed.
• Equal standard deviations The standard
  deviations of the variable under consideration
  are the same for all the populations.

27
Learning Under Three Temperature Conditions
M1 1 M2 5 M3 1
28
Learning Under Three Temperature Conditions
29
Learning Under Three Temperature Conditions
30
Learning Under Three Temperature Conditions
31
Learning Under Three Temperature Conditions
32
Learning Under Three Temperature Conditions
33
Learning Under Three Temperature Conditions
M2
M1
M3
34
Learning Under Three Temperature Conditions
SX2 106
16936916
144
191
M2
M1
M3
35
Learning Under Three Temperature Conditions
M2
M1
M3
36
Learning Under Three Temperature Conditions
M2
M1
M3
37
Calculating the F statistic
Sstotal X2-G2/N 46 SSbetween
SSbetween 30 SStotal Ssbetween
SSwithin Sswithin 16
38
Distribution of the F-Statistic for One-Way ANOVA
Suppose the variable under consideration is
normally distributed on each of k populations and
that the population standard deviations are
equal. Then, for independent samples from the k
populations, the variable has the
F-distribution with df (k 1, n k) if the
null hypothesis of equal population means is
true. Here n denotes the total number of
observations.
39
ANOVA Source Table for a one-way analysis of
variance
40
The one-way ANOVA test for k population means
(Slide 1 of 3)
Step 1 The null and alternative hypotheses
are Ho ?1 ?2 ?3 ?k Ha Not all the
means are equal Step 2 Decide On the significance
level, ? Step 3 The critical value of F?, with df
(k - 1, N - k), where N is the total number of
observations.
41
The one-way ANOVA test for k population means
(Slide 2 of 3)
42
The one-way ANOVA test for k population means
(Slide 3 of 3)
Step 4 Obtain the three sums of squares, STT,
STTR, and SSE Step 5 Construct a one-way ANOVA
table Step 6 If the value of the
F-statistic falls in the rejection region, reject
H0
43
Post Hocs

• H0 ?1 ?2 ?3 ?k
• Rejecting H0 means that not all means are equal.
• Pairwise tests are required to determine which of
  the means are different.
• One problem is for large k. For example with k
  7, 21 means must be compared. Post-Hoc tests are
  designed to reduce the likelihood of groupwise
  type I error.

44
Criterion for deciding whether or not to reject
the null hypothesis
45
One-Way ANOVA
A researcher wants to test the effects of St.
Johns Wort, an over the counter, herbal
anti-depressant. The measure is a scale of
self-worth. The subjects are clinically
depressed patients. Use a 0.01
46
One-Way ANOVA
Compute the treatment totals, T, and the grand
total, G
47
One-Way ANOVA
Count n for each treatment, the total N, and k
48
One-Way ANOVA
Compute the treatment means
49
One-Way ANOVA
(0-1)21 (1-1)20 (3-1)24 (0-1)21 (1-1)20
sum
Compute the treatment SSs
50
One-Way ANOVA
Compute all X2s and sum them
51
One-Way ANOVA
Compute SSTotal SSTotal ?X2 G2/N
52
One-Way ANOVA
Compute SSWithin SSWithin ?SSi
53
One-Way ANOVA
Determine d.f.s d.f. WithinN-k d.f.
Betweenk-1 d.f. TotalN-1 Note that
(N-k)(k-1)N-1
54
One-Way ANOVA
Ready to move it to a source table
55
One-Way ANOVA

• Compute the missing values

56
One-Way ANOVA

• Compute the missing values

57
One-Way ANOVA

• Compute the missing values

58
One-Way ANOVA

• Compare your F of 17.5 with the critical value at
  2,12 degrees of freedom, ? 0.01 6.93
• reject H0

59
One-Way ANOVA
Students want to know if studying has an impact
on a 10-point statistics quiz, so they divided
into 3 groups low studying (0-5hrs./wk), medium
studying (6-15 hrs./wk) and high studying (16
hours/week). At a0.01, does the amount of
studying impact quiz scores?
60
One-Way ANOVA
Compute the treatment totals, T, and the grand
total, G
61
One-Way ANOVA
Count n for each treatment, the total N, and k
62
One-Way ANOVA
Compute the treatment means
63
One-Way ANOVA
(2-2)20 (4-2)24 (3-2)21 (0-2)24 (2-2)20 (1-2)
21 sum
Compute the treatment SSs
64
One-Way ANOVA
Compute all X2s and sum them
65
One-Way ANOVA
Compute SSTotal SSTotal ?X2 G2/N
66
One-Way ANOVA
Compute SSWithin SSWithin ?SSi
67
One-Way ANOVA
Determine d.f.s d.f. WithinN-k d.f.
Betweenk-1 d.f. TotalN-1 Note that
(N-k)(k-1)N-1
68
One-Way ANOVA

• Fill in the values you have

69
One-Way ANOVA

• Compute the missing values

70
One-Way ANOVA

• Compare your F of 37.97 with the critical value
  at 2,15 degrees of freedom, ? 0.01 6.36
• reject H0