Significance Tests:

1
Significance Tests

• THE BASICS
• Could it happen by chance alone?

2
Statistical Inference

• Confidence IntervalsUse when you want to
  estimate a population parameter
• Significance TestsUse when you want to assess
  the evidence provided by data about some claim
  concerning a population
• AN OUTCOME THAT WOULD RARELY HAPPEN BY CHANCE IF
  A CLAIM WERE TRUE IS GOOD EVIDENCE THAT THE CLAIM
  IS NOT TRUE

3
Overview of a Significance Test

• A test of significance is intended to assess the
  evidence provided by data against a null
  hypothesis H0 in favor of an alternate hypothesis
  Ha.
• The statement being tested in a test of
  significance is called the null hypothesis.
  Usually the null hypothesis is a statement of no
  effect or no difference.
• A one-sided alternate hypothesis exists when we
  are interested only in deviations from the null
  hypothesis in one direction
• H0 ?0
• Ha ?gt0 (or ?lt0)
• If the problem does not specify the direction of
  the difference, the alternate hypothesis is
  two-sided
• H0 ?0
• Ha ??0

4
HYPOTHESES

• NOTE Hypotheses ALWAYS refer to a population
  parameter, not a sample statistic.
• The alternative hypothesis should express the
  hopes or suspicions we have BEFORE we see the
  data. Dont cheat by looking at the data first.

5
CONDITIONS

• These should look the same as in the last chapter
  (for confidence intervals)
• SRS
• Normality
• For meanspopulation distribution is Normal or
  you have a large sample size (n30)
• For proportions--np10 and n(1-p)10
• Independence

6
CAUTION

• Be sure to check that the conditions for running
  a significance test for the population mean are
  satisfied before you perform any calculations.

7
Test Statistic

• A test statistic comes from sample data and is
  used to make decisions in a significance test
• Compare sample statistic to hypothesized
  parameter
• Values far from parameter give evidence against
  the null hypothesis (H0)
• Standardize your sample statistic to obtain your
  TEST STATISTIC

8
P-values statistical significance

• The probability (computed assuming H0 is true)
  that the test statistic would take a value as
  extreme or more extreme than that actually
  observed is called the P-value of the test. The
  smaller the P-value, the stronger the evidence
  against the null hypothesis provided by the data.
• Significant in the statistical sense doesnt
  mean important. It means simply not likely to
  happen just by chance.
• The significance level a is the decisive value of
  the P-value. It makes not likely more exact.
• If the P-value is as small or smaller than a, we
  say that the data is statistically significant at
  level a.

9
INFERENCE TOOLBOX (p 705)
DO YOU REMEMBER WHAT THE STEPS ARE???
Steps for completing a SIGNIFICANCE TEST

• 1PARAMETERIdentify the population of interest
  and the parameter you want to draw a conclusion
  about. STATE YOUR HYPOTHESES!
• 2CONDITIONSChoose the appropriate inference
  procedure. VERIFY conditions (SRS, Normality,
  Independence) before using it.
• 3CALCULATIONSIf the conditions are met, carry
  out the inference procedure.
• 4INTERPRETATIONInterpret your results in the
  context of the problem. CONCLUSION, CONNECTION,
  CONTEXT(meaning that our conclusion about the
  parameter connects to our work in part 3 and
  includes appropriate context)

10
Step 1PARAMETER

• Read through the problem and determine what we
  hope to show through our test.
• Our null hypothesis is that no change has
  occurred or that no difference is evident.
• Our alternative hypothesis can be either one or
  two sided.
• Be certain to use appropriate symbols and also
  write them out in words.

11
Step 2CONDITIONS

• Based on the given information, determine which
  test should be used. Name the procedure.
• State the conditions.
• Verify (through discussion) whether the
  conditions have been met. For any assumptions
  that seem unsafe to verify as met, explain why.
• Remember, if data is given, graph it to help
  facilitate this discussion
• For each procedure there are several things that
  we are assuming are true that allow these
  procedures to produce meaningful results.

12
Step 3CALCULATIONS

• First write out the formula for the test
  statistic, report its value, mark the value on
  the curve.
• Sketch the density curve as clearly as possible
  out to three standard deviations on each side.
• Mark the null hypothesis and sample statistic
  clearly on the curve.
• Calculate and report the P-value
• Shade the appropriate region of the curve.
• Report other values of importance (standard
  deviation, df, critical value, etc.)

13
Step 4INTERPRETATION

• There are really two parts to this step
  decision conclusion. TWO UNIQUE SENTENCES.
• Based on the P-value, make a decision. Will you
  reject H0 or fail to reject H0.
• If there is a predetermined significance level,
  then make reference to this as part of your
  decision. If not, interpret the P-value
  appropriately.
• Now that you have made a decision, state a
  conclusion IN THE CONTEXT of the problem.
• This does not need to, and probably should not,
  have statistical terminology involved. DO NOT
  use the word prove in this statement.

14
Example 1

• Your buddy (Jake) claims to be an A student
  (meaning he has a 90 average). You dont know
  all of his grades but based on what you have seen
  you think this claim is an overstatement. You
  took a simple random sample of his grades and
  recorded them. They are 92, 87, 86, 90, 80,
  91. You also know that all his grades in the
  class have a standard deviation of 3.5.

15
Step 1

• We want to determine whether Jake is accurate in
  his measure of his course grade.
• Our null hypothesis is that Jake has a course
  average of 90.
• Our alternative hypothesis is that Jakes course
  average is below a 90.
• H0 ? 90
• Ha ? lt 90

16
Step 2

• Since we know the population standard deviation
  we will be performing a z-test of significance.
• We were told that our selection of grades was an
  SRS of Jakes scores.
• The box plot shows moderate left skewness. Our
  sample is not large so we must assume that the
  population of all of Jakes grades are
  approximately normal in distribution in order for
  our sampling distribution to be approximately
  normal. Using the IQR(1.5) method for
  determining outliers we see that there are no
  outliers in this sample of grades.
• Provided Jake has at least 60 overall grades, we
  are safe assuming independence and using the
  necessary formula for standard deviation.

17
Step 3

• A curve should be drawn, labeled, and shaded.
• You can use the formula to calculate your z test
  statistic for this problem
• ? In this case z -1.6330
• Mark this on your sketch.
• Based on our calculations the P-value is 0.0512.
• , s3.5, n6

18
Step 4

• Since there is no predetermined level of
  significance if we are seeking to make a
  decision, this could be argued either way. If
  Jake were correct about being an A student, we
  would only get a sample of grades with an average
  this low in roughly 5.1 of all samples.
• There is not overwhelming evidence against H0,
  however, this is enough to convince me that H0
  can be rejected.
• Our evidence may not be strong enough to convince
  Jake that he is wrong. However, based on this
  evidence, I do not believe Jake is accurate about
  his average being a 90. It doesnt appear that
  Jake is the A student he claims to be.

19
WARNINGS

• Tests of significance assess evidence against H0
• If the evidence is strong, reject H0 in favor of
  Ha
• Failure to find evidence against H0 means only
  that data are consistent with H0, not that we
  have clear evidence that H0 is true
• If you are going to make a decision based on
  statistical significance, then the significance
  level a should be stated before the data are
  produced.