Power (Reading Packet Sect III)

1
Power (Reading Packet Sect III)

• Mon, March 29th

2
Power

• The ability of your statistical test to correctly
  reject the null hypothesis
• Remember, the null hypothesis generally states
  there is no effect or no differences between your
  sample pop
• Power refers to the power to find any differences
  that truly exist
• Want to maximize power

3
(cont.)

• When we fail to reject Ho, we want to be sure it
  is because there are truly no differences that
  exist (or no effect)
• Not because our statistical test didnt have
  enough power to find a difference that is
  actually there.
• In your 2x2 Decision Table,
• power is 1-probability of a Type 2 error

4
2x2 Decision Table (Ho No disease present)
Reality Reality Reality
You Decide Ho is correct Ho is incorrect
You Decide Reject Ho Type 1 error (alpha) False Pos Correct! (POWER)
You Decide Fail to Reject Ho Correct! Type 2 error (beta) False Neg.
5
What Influences Power?

• 4 factors that affect power
• 1) Alpha level increasing alpha (chance of Type
  1 error), increases power
• Going from alpha .01 to alpha .05, gives you
  larger chance of finding a difference/effect that
  is really there.
• Consider effect on your critical region of .01 v
  .05 region becomes larger by using alpha .05,
  more likely to reject Ho

6
Example from last Wed (salary)

• Compare ybar 24,100 w/ ?y 28,985 (and ?y
  23,335, N100)
• For 1-sample z test, z (ybar - ?y) / ?ybar,
  where ?ybar ?y / sqrt N), or ?ybar 23,335/
  sqrt (100) 2333.5
• Z obtained 24,100 28,985 / 2333.5 -2.09
• Books approach ? look up z -2.09 in Appendix
  (Col C), find its probability .0183
• If alpha .05 (1 tail) use .0183, then p lt ?, so
  REJECT Ho

7
Ex (cont.)

• Another approach is to find the critical z value
  associated with an alpha level
• When z obtained (from formula) gt z
  critical (from table) ? REJECT Ho
• ? .05, 1 tail, z critical 1.645 or 1.645
• .05 2 tails, z criticals -1.96 and 1.96
• .01, 1 tail, z critical 2.33 or 2.33
• .01, 2 tails, z criticals 2.57 and 2.57

8
Ex (cont.)

• Using this approach, z obtained -2.09,
• z critical (? .05, 1 tail) -1.645, since
  obtained gt critical ? Reject Ho
• Same conclusion as for p lt alpha
• Notice critical region becomes smaller as alpha
  level becomes smaller
• And as you move from 1 tail to 2 tails
• Harder to reject Ho w/ smaller critical region

9
Other Influences on Power

• 2) Sample Size larger N, more power
• With larger sample size, more likely a
  representative sample with less error
• 3) 1- v 2-tailed test 1-tailed test has more
  power
• The critical region for rejecting Ho is larger
  w/1-tailed test (dont have to split into 2
  tails)
• 4) Effect size larger effect size, more power
• Refers to the effect of the manipulation (in an
  experiment) or the difference betw sample pop

10
(cont.)

• If you have a strong manipulation, will create
  larger differences among groups or betw sample
  population
• easier to see the effect if its really there

11
Determining Power

• Failure to reject Ho could be due to many things
• There really is no effect / no difference
• Your study had low reliability, validity, etc.
• Your study didnt have enough power
• How can we determine the amount of power of our
  study?
• Cant be set by you (as alpha can), but can
  calculate after the fact or try to predict before
  the experiment
• Can calculate needed sample size for certain
  level of power (in adv stat class youll do this!)

12
Stat v Practical Significance

• Its also possible to have too much power!
• With large enough sample sizes, youll have so
  much power that even very small differences
  effects will turn out statistically significant
• w/N5,000, a difference between µ3.4 and µ3.5
  on a 1-7 scale is significant, but is it
  important? Practical?
• Pay attention to whether a statistically signif
  finding has any practical significance (is it
  meaningful? Important?)