Inferential Statistics

1
Inferential Statistics
2
Purpose of Inferential Statistics

• Try to reach conclusions that extend beyond the
  immediate data
• Make judgments about whether an observed
  difference between groups is a dependable one or
  one that might have happened by chance
• Differs from descriptive statistics where we
  simply describe what's going on in the data

3
Effect of a Variable Compared across Groups
(revisited)

• Simplest situation a control group and an
  experimental group
• Three possible reasons why two groups can differ
• The variable has had a true effect
• Random variability or error
• Extraneous variables
• Statistics are used to factor out random error

4
Hypotheses

• Experimental hypothesis - the prediction that an
  IV will cause an effect
• Null hypothesis (H0) - the hypothesis that an IV
  has no effect
• If difference between groups is not due to random
  error, the null hypothesis is rejected
• If there is no difference between groups, there
  are two possibilities the effect is not there,
  or the effect was really there but not exhibited

5
Statistical Inference

• If the stats show that the difference is greater
  than could be expected if random error were at
  work, then the results are statistically
  significant
• Type 1 Error (alpha error) - mistaking a random
  error difference for a treatment difference
• Type 2 Error (beta error) - overlooking a
  treatment effect because you think it's due to
  random error

6
Power (revisited)

• Goal is for the study to have POWER
• Power - ability to avoid making Type 2 errors
• Power - ability to correctly reject the null
  hypothesis
• Power is affected by
• chosen alpha level
• size of the sample
• size of the effect produced by the IV

7
Too Much Power?

• If enough P's used, even the most minute and
  trivial differences will be found statistically
  significant
• Sample should be large enough to be sensitive to
  differences between treatments, but not so large
  as to produce trivial results

8
Types of Inferential Statistics

• Parametric tests
• 3 assumptions underlying the use of parametric
  statistics
• scores have been sampled randomly from the
  population
• scores are normally distributed
• within-groups variance is homogenous
• Non-Parametric tests
• Tests used when any of the above assumptions are
  not met

9
Types of Parametric Tests

• T-test (t-value)
• Use when you have 2 levels of an IV. Versions for
  between (independent samples) and within
  (correlated or paired samples) subjects designs
• ANOVA (F-ratio)
• 1-factor between S's
• 1-factor within S's (repeated measures)
• 2-factor between S's (main effects and
  interactions)
• 2-factor within S's (repeated measures)
• Mixed designs
• ANCOVA (analysis of covariance)
• MANOVA

10
Types of Nonparametric Tests

• Chi-Square
• Use when you have a dichotomous variable
• Mann-Whitney U Test
• Use when you have ordinal data. Alternative to a
  t-test when assumptions underlying t-test are not
  met.

11
Parametric vs. Nonparametric Tests

• Parametric tests are more powerful
• Nonparametrics should be used when data do not
  meet assumptions of parametric tests
• Nonparametrics not always available for complex
  designs

12
Statistical vs. Practical Significance

• The fact that the treatment means differ
  significantly may or may not be important.
• Sometimes statistical and practical significance
  overlap, sometimes not.
• Alpha levels must be chosen with the goal of the
  research in mind.
• Replication is sometimes a substitute for
  statistics
• Effect sizes instead of statistics?
• Visualizing data instead of statistics?