Limitations of Significance Tests - PowerPoint PPT Presentation

About This Presentation
Title:

Limitations of Significance Tests

Description:

A significance test merely indicates whether the particular parameter value in H0 is plausible ... little about which potential parameter values are plausible ... – PowerPoint PPT presentation

Number of Views:43
Avg rating:3.0/5.0
Slides: 18
Provided by: katemcl
Category:

less

Transcript and Presenter's Notes

Title: Limitations of Significance Tests


1
Section 8.5
  • Limitations of Significance Tests

2
Statistical Significance Does Not Mean Practical
Significance
  • When we conduct a significance test, its main
    relevance is studying whether the true parameter
    value is
  • Above, or below, the value in H0 and
  • Sufficiently different from the value in H0 to be
    of practical importance

3
What the Significance Test Tells Us
  • The test gives us information about whether the
    parameter differs from the H0 value and its
    direction from that value

4
What the Significance Test Does Not Tell Us
  • It does not tell us about the practical
    importance of the results

5
Statistical Significance vs. Practical
Significance
  • A small P-value, such as 0.001, is highly
    statistically significant, but it does not imply
    an important finding in any practical sense
  • In particular, whenever the sample size is large,
    small P-values can occur when the point estimate
    is near the parameter value in H0

6
Significance Tests Are Less Useful Than
Confidence Intervals
  • A significance test merely indicates whether the
    particular parameter value in H0 is plausible
  • When a P-value is small, the significance test
    indicates that the hypothesized value is not
    plausible, but it tells us little about which
    potential parameter values are plausible

7
Significance Tests are Less Useful than
Confidence Intervals
  • A Confidence Interval is more informative,
    because it displays the entire set of believable
    values

8
Misinterpretations of Results of Significance
Tests
  • Do Not Reject H0 does not mean Accept H0
  • A P-value above 0.05 when the significance level
    is 0.05, does not mean that H0 is correct
  • A test merely indicates whether a particular
    parameter value is plausible

9
Misinterpretations of Results of Significance
Tests
  • Statistical significance does not mean practical
    significance
  • A small P-value does not tell us whether the
    parameter value differs by much in practical
    terms from the value in H0

10
Misinterpretations of Results of Significance
Tests
  • The P-value cannot be interpreted as the
    probability that H0 is true

11
Misinterpretations of Results of Significance
Tests
  • It is misleading to report results only if they
    are statistically significant

12
Misinterpretations of Results of Significance
Tests
  • Some tests may be statistically significant just
    by chance

13
Misinterpretations of Results of Significance
Tests
  • True effects may not be as large as initial
    estimates reported by the media

14
Section 8.6
  • How Likely is a Type II Error?

15
Type II Error
  • A Type II error occurs in a hypothesis test when
    we fail to reject H0 even though it is actually
    false

16
Calculating the Probability of a Type II Error
  • To calculate the probability of a Type II error,
    we must do a separate calculation for various
    values of the parameter of interest

17
Power of a Test
  • Power 1 P(Type II error)
  • The higher the power, the better
  • In practice, it is ideal for studies to have high
    power while using a relatively small significance
    level
Write a Comment
User Comments (0)
About PowerShow.com