Hypothesis testing

1
Hypothesis testing
2
Hypothesis testing

• Null hypothesis
• Alternative (experimental) hypothesis

3
Example
Der Mann, der dich gesehen hat. 21 Der Mann, den
du gesehen hast. 6 Der Film, der dir gefallen
hat. 12 Der Film, den du gesehen hast. 17
Null hypothesis There is no relationship
between the animacy of the head noun and the
syntactic role of the relative pronoun. Alternati
ve hypothesis There is a relationship between
the animacy of the head noun and the syntactic
role of the relative pronoun.
4
Population
Sample
5
Animate Inanimate
Subject 50 50
Object 50 50
Animate Inanimate
Subject 21 12
Object 6 17
6
Statistical tests determines the probability that
the relationship we observe has arisen from
sample error.
If that probability is very low (i.e. gt 5), we
can reject the null hypothesis, i.e. the
hypothesis that there is no relationship between
variables.
Statistical hypothesis testing does not prove
that the (explanation for the) alternative
hypothesis.
7
p-value
The p-value is a conditional probability.
The p-value indicates that, given that there is
no relationship between x and y, what is the
probability that we obtain the distribution in
our sample.
If there is no relationship (correlation) between
X and Y in the true population, then there is a
less than 5 chance (i.e. 1 out of 20 chance)
that there is a correlation in the sample.
8
p-value
P 0.05. What does that mean?
The probability of the null hypothesis to be true
is 5.
False
The probability of the alternative hypothesis to
be true is 95.
False
Given that the null hypothesis is true, there is
a 5 chance of obtaining the distribution in the
given sample.
Correct
9
Type 1 and type 2 errors

• Type 1 error The p-value is significant (p lt
  .05) and you reject the null hypothesis although
  there is no correlation between X and Y.

• Type 2 error The p-value is not significant (p gt
  .05) and you accept the null hypothesis although
  there is a difference between X and Y.

10
Type 1 and type 2 errors
The p-value indicates the probability of making a
type 1 error. It does not say anything about the
probability of a type 2 error occurring. While a
type 2 error is as fatal as a type 1 error, in
practice it is less dramatic. Why?
If p gt 0.05 and you accept the null-hypothesis,
it is not automatically assumed that there is no
correlation (or difference) between conditions.
Why?
Because sample error is only one possible source
for the non-significant p-value. Other sources
experimental design.
11
One-tailed and two-tailed tests
A researcher wants to find out if sex influences
language development during childhood. He has
collected MLU values from a group of 3 year-old
boys and 3 year-old girls. State the hypotheses.

• Sex does not influence development (i.e. MLU).
• Sex influences development (i.e. MLU)
• Girls have a higher MLU.
• Boys have a higher MLU.

12
One-tailed and two-tailed tests
13
Statistical measures

• p-value
• Confidence intervals
• Effect size