Effect Size Definitions

1
Effect Size Definitions
2
Meta-analysis weights

• Meta-analysis takes an average
• Unit weights (unweighted average w1) (Bonett)
• Sample size weights (w N) (Schmidt Hunter)
• Inverse variance weights (w 1/V) (Hedges
  Olkin)
• There are arguments in favor of each. We will
  mostly focus on inverse variance weights.

3
Single Variable Effect Sizes

• Use for central tendency
• E.g., what is the graduation rate from college?
• What is the time to complete college?
• What is the proportion of female college
  graduates?

4
Proportion (Direct)
ES Effect size. P is the proportion of things
of interest.
e.g., p proportion field goals made from less
than 40 yards.
Precision
5
Proportion (Logit)
Logit has nice statistical properties.
Precision
6
Aritmethic Mean
e.g., mean achievement test score.
Precision
7
Conventional Effect Sizes
Most effect sizes show the relations between two
variables, either a difference between groups
(IV) on some criterion of interest (DV), such as
d, the standardized mean difference, or an
association between two continuous variables
(e.g., the correlation), or between two
categorical variables (e.g. odds ratios).
8
Mean Difference (Unstandardized)
Used ONLY if measures are the same across all
studies (e.g., used the Beck Depression Inventory
to study the effectiveness of a treatment for
depression (experimental vs. control group
design).
9
Mean Difference (Standardized)
Spooled is the pooled Standard deviation. Note
that the variance of d depends upon the magnitude
of d (actually delta, estimated by d).
The estimated standard deviation in Excel is
stdev.s. Example stdev.s(a1a10)
10
Denominators of d and t
This is the pooled standard deviation within
group SD, the yardstick for computing d.
This is the standard error of the difference
between means. This is the yardstick for the
independent samples t-test. Which will show a
larger difference between group means?
11
Mean Difference(Standardized)
Bias correction
Formulas from Borenstein et al., 2009, p. 27
The effect size d is sometimes called Cohens d
and the effect size g is sometimes called
Hedges g but in practice they are essentially
the same. It is now conventional to use g.
12
Binary IV DV risk ratio
Events Non-Events
Treated A B n1
Control C D n2
Total
Heart Attack No attack
Treated 5 45 50
Control 10 40 50
100
13
Binary - odds ratio
Events Non-Events
Treated A B n1
Control C D n2
Total
14
Correlation (Pearsons r)
Fishers r to z transformation.
The Excel function for correlation is
correl(rangeX, rangeY). Example
correl(a1a10, b110). The r to z in Excel is
atanh(correlation) e.g., atanh(c11).
15
Class Exercise 1a
Group 1 Group 2
4 5
5 5
6 7
7 8
5 4
8 9
7 8
9 11
Compute Cohens d for these data. Compute
Hedges g for these data. I would use Excel if I
were you.
16
Class Exercise 1b
Variable X Variable Y
4 5
5 5
6 7
7 8
5 4
8 9
7 8
9 11
Compute the correlation coefficient r for these
data (note the data are the same as exercise 1a,
but we have only one group of people and two
variables. Compute Fishers z for these data.