Effect Sizes

1
Effect Sizes Power Analyses for k-group
Designs

• Effect Size Estimates for k-group ANOVA designs
• Power Analysis for k-group ANOVA designs
• Effect Size Estimates for k-group X2 designs
• Power Analysis for k-group X2 designs

2
k-BG Effect Sizes
When you have more than 2 groups, it is possible
to compute the effect size for the whole study.
Include the F-value, the df (both for the effect
and the error), and click the button for the type
of design you have (BG or WG)
However, this type of effect size is not very
helpful, because -- you dont know which
pairwise comparison(s) make up the r -- it can
only be compared to other designs with exactly
the same combination of conditions
3
k-BG Effect Sizes
Just as RH for k-group designs involve comparing
2 groups at a time (pairwise comparisons) The
most useful effect sizes for k-group designs are
computed as the effect size for 2 groups (effect
sizes for pairwise comparisons)
Since you wont have F-values for the pairwise
comparisons, you will use Computator to
complete a 2-step computation Using info from
the SPSPS output d (M1 - M2 ) / ?
MSerror d²
r ----------
? d² 4
4
Pairwise effect sizes computation for k-BG designs
For no therapy vs. weekly therapy
For a BG design be sure to press
5
k-WG Effect Sizes
Just as RH for k-group designs involve comparing
2 groups at a time (pairwise comparisons) Effect
sizes for k-group designs are computed as the
effect size for 2 groups (effect sizes for
pairwise comparisons)
Since you wont have F-values for the pairwise
comparisons, you will use Computator to
complete a 3-step computation Using info from
the SPSPS output d
(M1 - M2 ) / ? (MSerror 2)
dw d 2 dw²
r
---------- ? dw²
4
6
Pairwise effect sizes computation for k-WG designs
For no intake vs. mid
For a WG design be sure to press
7

• Determining the power you need ..
• For a 2-condition design...
• the omnibus-F is sufficient -- retain or reject,
  youre done !
• you can easily determine the sample size needed
  to test any expected effect size with a given
  amount of power
• For a k-condition design
• the power of the omnibus-F - isnt what matters
  !
• a significant omnibus-F only tells you that the
  two most different means are significantly
  different
• follow-up (pairwise) analyses will be needed to
  test if the pattern of the mean differences
  matches the RH
• you dont want to have a pattern of results
  that is really just a pattern of differential
  statistical power
• you need to assure that you have sufficient
  power for the smallest pairwise effect needed to
  test your specific RH

8
k-group Power Analyses As before, there are two
kids of power analyses

• A priori power analyses
• conducted before the study is begun
• start with r desired power to determine the
  needed N
• Post hoc power analysis
• conducted after retaining H0
• start with r N and determine power Type II
  probability

9
Power Analyses for k-BG designs

• Important Symbols
• S is the total of participants in that
  pairwise comp
• n S / 2 is the of participants in each
  condition
• of that pairwise comparison
• N n k is the total number or participants
  in the study
• Example
• the smallest pairwise effect size for a 3-BG
  study was .25
• with r .25 and 80 power S 120
• for each of the 2 conditions n S / 2
  120 / 2 60
• for the whole study N n
  k 60 3 180

10
Power Analyses for k-WG designs

• Important Symbols
• S is the total of participants in that
  pairwise comp
• For WG designs, every participant is in every
  condition, so S is also the number of
  participants in each condition
• Example
• the smallest pairwise effect size for a 3-WG
  study was .20
• with r .20 and 80 power S 191
• for each condition of a WG design n S 191
• for the whole study N
  S 191

11

• Combing LSD r
• Cx Tx1
  mean M dif r
  M dif r
• Cx 20.3
• Tx1 24.6 4.3 .32
• Tx2 32.1 11.8 .54 7.5 .41
  
• indicates mean difference is significant based
  on LSD criterion (min dif 6.1)
• Something to notice
• The effect size of Cx vs. Tx1 is substantial
  (Cohen calls .30 medium effect), but is not
  significant, suggesting we should check the power
  of the study for testing an effect of this size.

12
k-group Effect Sizes
When you have more than 2 groups, it is possible
to compute the effect size for the whole study.
Include the X², the total N and click the button
for df gt 1
However, this type of effect size is not very
helpful, because -- you dont know which
pairwise comparison(s) make up the r -- it can
only be compared to other designs with exactly
the same combination of conditions
13
Pairwise Effect Sizes
Just as RH for k-group designs involve comparing
2 groups at a time (pairwise comparisons) The
most useful effect sizes for k-group designs are
computed as the effect size for 2 groups (effect
sizes for pairwise comparisons)

The effect size computator calculates the effect
size for each pairwise X² it computes
14
k-group Power Analyses As before, there are two
kinds of power analyses

• A priori power analyses
• conducted before the study is begun
• start with r desired power to determine the
  needed N
• Post hoc power analysis
• conducted after retaining H0
• start with r N and determine power Type II
  probability

15
Power Analyses for k-group designs

• Important Symbols
• S is the total of participants in that
  pairwise comp
• n S / 2 is the of participants in each
  condition
• of that pairwise comparison
• N n k is the total number or participants
  in the study
• Example
• the smallest pairwise X² effect size for a 3-BG
  study was .25
• with r .25 and 80 power S 120
• for each of the 2 conditions n S / 2
  120 / 2 60
• for the whole study N n
  k 60 3 180