Between Groups

1
Between Groups Within-Groups ANOVA

• BG WG ANOVA
• Partitioning Variation
• making F
• making effect sizes
• Things that influence F
• Confounding
• Inflated within-condition variability
• Integrating stats methods

2

• ANOVA ? ANalysis Of VAriance
• Variance means variation
• Sum of Squares (SS) is the most common variation
  index
• SS stands for, Sum of squared deviations
  between each of a set of values and the
  mean of those values
• SS ? (value
  mean)2
• So, Analysis Of Variance translates to
  partitioning of SS
• In order to understand something about how ANOVA
  works we need to understand how BG and WG ANOVAs
  partition the SS differently and how F is
  constructed by each.

3
Variance partitioning for a BG design
Called error because we cant account for why
the folks in a condition -- who were all treated
the same have different scores.
Tx C
20 30 10 30 10
20 20 20
Mean 15 25
Variation among all the participants represents
variation due to treatment effects and
individual differences
Variation among participants within each
condition represents individual differences
Variation between the conditions represents
variation due to treatment effects
SSTotal SSEffect
SSError
4
How a BG F is constructed
Mean Square is the SS converted to a mean ?
dividing it by the number of things
SSTotal SSEffect SSError
dfeffect k - 1 represents conditions
in design
MSeffect
SSeffect / dfeffect
F
SSerror / dferror
MSerror
dferror ?n - k represents participants
in study
5
How a BG r is constructed
r2 effect / (effect error) ?
conceptual formula SSeffect / ( SSeffect
SSerror ) ? definitional formula F /
(F dferror) ?
computational forumla
MSeffect
SSeffect / dfeffect
F
SSerror / dferror
MSerror
6
An Example
SStotal SSeffect SSerror 1757.574
605.574 1152.000
r2 SSeffect / ( SSeffect SSerror
) 605.574 / ( 605.574 1152.000 )
.34
r2 F / (F dferror)
9.462 / ( 9.462 18) .34
7
Variance partitioning for a WG design
Called error because we cant account for why
folks who were in the same two conditions -- who
were all treated the same two ways have
different difference scores.
Sum Dif
Tx C
20 30 10 30 10
20 20 20
50 40 30 40
10 20 10 0
Mean 15 25
Variation among participants difference scores
represents individual differences
Variation among participants estimable because
S is a composite score (sum)
SSTotal SSEffect SSSubj
SSError
8
How a WG F is constructed
Mean Square is the SS converted to a mean ?
dividing it by the number of things
SSTotal SSEffect SSSubj SSError
dfeffect k - 1 represents conditions
in design
MSeffect
SSeffect / dfeffect
F
SSerror / dferror
MSerror
dferror (k-1)(n-1) represents data
points in study
9
How a WG r is constructed
r2 effect / (effect error) ?
conceptual formula SSeffect / ( SSeffect
SSerror ) ? definitional formula F /
(F dferror) ?
computational forumla
MSeffect
SSeffect / dfeffect
F
SSerror / dferror
MSerror
10
An Example
Dont ever do this with real data !!!!!!
SStotal SSeffect SSsubj
SSerror 1757.574 605.574 281.676
870.325
Professional statistician on a closed course. Do
not try at home!
r2 SSeffect / ( SSeffect SSerror
) 605.574 / ( 605.574 281.676 )
.68
r2 F / (F dferror)
19.349 / ( 19.349 9) .68
11
What happened????? Same data. Same means
Std. Same total variance.
Different F ???
BG ANOVA SSTotal SSEffect SSError
WG ANOVA SSTotal SSEffect SSSubj
SSError
The variation that is called error for the BG
ANOVA is divided between subject and error
variation in the WG ANOVA. Thus, the WG F is
based on a smaller error term than the BG F ? and
so, the WG F is generally larger than the BG F.
12
What happened????? Same data. Same means
Std. Same total variance.
Different r ???
r2 effect / (effect error) ?
conceptual formula SSeffect / ( SSeffect
SSerror ) ? definitional formula F /
(F dferror) ?
computational forumla
The variation that is called error for the BG
ANOVA is divided between subject and error
variation in the WG ANOVA. Thus, the WG r is
based on a smaller error term than the BG r ? and
so, the WG r is generally larger than the BG r.
13
Both of these models assume there are no
confounds, and that the individual differences
are the only source of within-condition
variability
SSeffect / dfeffect
BG SSTotal SSEffect SSError
F
WG SSTotal SSEffectSSSubjSSError
SSerror / dferror
A more realistic model of F IndDif ?
individual differences
BG SSTotal SSEffect SSconfound SSIndDif
SSwcvar
WG SSTotal SSEffect SSconfound SSSubj
SSIndDif SSwcvar
SSconfound ? between condition variability caused
by anything(s) other than the IV
(confounds) SSwcvar ? inflated within condition
variability caused by anything(s) other
than natural population individual
differences
14
Imagine an educational study that compares the
effects of two types of math instruction (IV)
upon performance ( - DV) Participants were
randomly assigned to conditons, treated, then
allowed to practice (Prac) as many problems as
they wanted to before taking the DV-producing
test Control Grp Exper. Grp
Prac DV Prac DV S1 5 75
S2 10 82 S3 5 74 S4 10
84 S5 10 78 S6 15 88 S7
10 79 S8 15 89

• IV
• compare Ss 52 - 74
• Confounding due to Prac
• mean prac dif between cond
• WG variability inflated by Prac
• wg corrrelation or prac DV
• Individual differences
• compare Ss 13, 57, 24, or 68

15

• The problem is that the F-formula will
• Ignore the confounding caused by differential
  practice between the groups and attribute all BG
  variation to the type of instruction (IV) ?
  overestimating the effect
• Ignore the inflation in within-condition
  variation caused by differential practice within
  the groups and attribute all WG variation to
  individual differences ? overestimating the error
• As a result, the F r values wont properly
  reflect the relationship between type of math
  instruction and performance ? we will make a
  statistical conclusion error !
• Our inability to procedurally control variables
  like this will lead us to statistical models that
  can statistically control them

SSeffect / dfeffect
F
r F / (F dferror)
SSerror / dferror
16
How research design impacts F ? integrating
stats methods!
SSeffect / dfeffect
SSTotal SSEffectSSconfoundSSIndDifSSwcvar
F
SSerror / dferror
SSEffect ? bigger manipulations produce larger
mean difference between the conditions
? larger F

• SSconfound ? between group differences other
  than the IV -- change mean
  difference ? changing F
• if the confound augments the IV ? F will be
  inflated
• if the confound counters the IV ? F will be
  underestimated

SSIndDif ? more heterogeneous populations have
larger within- condition differences ?
smaller F

• SSwcvar ? within-group differences other than
  natural individual differences ? smaller
  F
• could be procedural ? differential treatment
  within-conditions
• could be sampling ? obtain a sample that is
  more heterogeneous than the
  target population