Seth M. Noar, Ph.D.

1
Hypothesis Testing

• Seth M. Noar, Ph.D.
• Department of Communication
• University of Kentucky

2
Hypothesis Testing

• Widely used in the social sciences
• Statistical significance is one way to examine
  whether our findings are important statistically
• Numerous statistical tests allow us to test for
  statistical significance
• Logic for numerous statistical tests is the same,
  even though the tests are different

3
Hypotheses

• Research hypothesis (W M, p. 63)
• Null hypothesis (W M, p. 63)
• Two possibilities
• M1 equal to M2 (or different only to the extent
  to which we would expect based on sampling error)
• M1 unequal to M2
• We cannot directly test our research hypothesis.
  We test the null hypothesis.

4

• ARE THESE MEANS DIFFERENT???

5
Direction of tests

• We can also use directional or non-directional
  tests
• Two-tailed test Non-directional hypothesis
• One-tailed test directional hypothesis
• Note one-tailed test are more powerfully
  statistically.

6
Probability and Error

• We must choose a probability level (p
• The extent to which we risk error is directly
  related to the probability level
• Type 1 error rejecting null hypothesis when it
  was true
• However, we must balance this with statistical
  power probability of correctly rejecting null
  hypothesis

7

• Rejection regions for 2-tailed test, p

8

• Rejection regions for 1-tailed test, p

9

• Probability of type 1 error, p

10
(No Transcript)
11

• Comparison of Type 1 2 errors, p

12
t-test

• t-test is a significance test
• Focuses on mean differences
• When one would apply it
• 1 IV 2-level nominal (dichotomous)
• 1 DV continuous (must be interval)
• For example Do treatment and control
  participants differ at the end of a treatment
  program?

13
Logic of t-test

• M1 - M2
• SE diff
• SE diff standard error of the difference
  between means
• M1 M2 is straightforward.
• Larger the difference, larger the t value.
• SE diff is a bit more complex.

14
Standard Error of Difference

• Standard error of the difference between means
  takes into account
• Sampling error
• Does this by including sample size and
  variability in the formula.
• As t grows larger, associated values of
  probability grow smaller (higher chance of
  significance)
• Note Larger sample size often results in larger
  t values

15
Procedure

• Calculate t value
• Calculate degrees of freedom
• (n-1) (n-1) OR (n n 2)
• Look up critical value in Table
• Our calculated t-value must be greater than the
  value in the table to be statistically
  significant

16
(No Transcript)
17
Results

• If our t value is significant at the probability
  level we have chosen (phypothesis.
• This suggests that there are real differences
  between these means (e.g., they come from
  different populations).

18
Cautions related to significance

• Statistical significance is NOT clinical
  significance.
• Because two means are different statistically
  does NOT mean that the difference is meaningful
  or important.
• Statistical significance must be interpreted
  within the context of any given study.
• Measures of effect size are also very important.

19
More cautions on significance

• Significant test controversy gets revisited every
  so often.
• Some have suggested abandoning traditional
  statistical tests (e.g., Jacob Cohen).
• Lines of argument include
• Over-reliance on dichotomous statistical tests to
  tell us whats important
• Statistical tests are often misused
• Should rely more on effect sizes and confidence
  intervals

20
Hypothesis Testing 2

• Seth M. Noar, Ph.D.
• Department of Communication
• University of Kentucky

21
ANOVA

• Single Factor ANOVA Analysis of Variance
• Focuses on mean differences
• When one would apply it
• 1 IV 2 or more nominal levels
• 1 DV continuous (must be interval)

22
ANOVA

• In many ways, similar to t test
• Major difference is that it can handle more than
  2 levels of IV
• For example Experiment with three groups
  Treatment, placebo, and no-treatment control.

23
Logic of ANOVA

• MS between (among)
• MS within
• Calculate between group variance
• Calculate within group variance
• Divide to get an F value
• We always expect within group variability
• However, we would only expect between group
  variability if the groups were different

24
Degrees of Freedom

• Degrees of Freedom
• Between (among) groups (numerator)
• Number of groups minus 1
• K-1 3 1 2
• Within groups (denominator)
• N-1 in each group, times number of groups
• N-1 5 1 4 x 3 12
• Similar to t look up critical value in table to
  examine significance
• In fact, t2 F (get same result whether applying
  t test or ANOVA)

25
(No Transcript)
26
Results

• Similar to t, if our F value is significant at
  the probability level we have chosen, we reject
  the null hypothesis.
• F (2, 12) 16.25, p
• However, if there are more than 2 levels of the
  IV, we do NOT know where the differences lie.
• We need follow-up tests (post hoc tests) to know
  exactly where the differences are.

27
Follow-up Tests

• Such tests allow us to know exactly where
  differences lie
• Tests include Tukey, Scheffe, Duncan, and Newman
  Keuls
• Some are more conservative, some more liberal
• Tukey HSD test is often used, and is a reasonably
  balanced test

28
Variations on ANOVA

• Repeated measures ANOVA
• Instead of multiple groups, we have multiple time
  points
• ANCOVA Analysis of covariance
• Same as ANOVA, except that we control for a
  variable called a covariate (e.g., stress, other
  meds)
• MANOVA Multivariate Analysis of Variance
• Allows for multiple DVs

29
More cautions on significance

• Significant test controversy gets revisited every
  so often.
• Some have suggested abandoning traditional
  statistical tests (e.g., Jacob Cohen).
• Lines of argument include
• Over-reliance on dichotomous statistical tests to
  tell us whats important
• Statistical tests are often misused
• Should rely more on effect sizes and confidence
  intervals