Hypothesis Testing

1
Hypothesis Testing

• Repeated-Measures Data with One Sample

2
Repeated-Measures Testing with One Sample

• One-Sample Z/T-Test
• Looks at difference between one sample and its
  population at one point in time
• What if we wanted to compare a sample with its
  population when we had data at multiple time
  points (i.e. one subject provided data for
  multiple groups/samples)?

3
Repeated-Measures Testing with One Sample

• Why would we want to do this?
• If we were devising a treatment for a particular
  disorder, wed want to know how our subjects
  fared post-treatment as a function of their
  pre-treatment score
• I.e. if we know that Bob has a score of 10/21 on
  a measure of anxiety after participating in a
  treatment for social phobia, treatment is only
  successful if Bob scored much less than this
  before treatment
• Knowing Bobs score at two time points (before
  and after treatment), helps us to make this
  determination

4
Repeated-Measures Testing with One Sample

• Taking multiple assessments of one subject over
  time gives us more data from which to draw
  conclusions, allows us to use more powerful
  statistical tests, and therefore requires fewer
  subjects to be run
• Powerful the test can detect a real
  difference between two groups, or between two or
  more time points with fewer subjects (Remember
  sample size effects the likelihood of obtaining a
  significant p-value)

5
Repeated-Measures Testing with One Sample

• OK, well then why not just pretend the data from
  Time 1 is one sample, the data from Time 2 is
  another sample, and compare the means on the two?
• This is a big statistical no-no one of the
  assumptions of the Two-Independent Samples T-Test
  (covered in the next chapter) is that the samples
  are independent/unrelated
• If subjects provide data at Time 1(pre-Tx sample)
  and Time 2 (post-Tx sample), these two samples
  are not independent, they are probably related.
  If Bob experiences the most anxiety (has the
  highest score on our anxiety scale) before
  treatment, he probably experiences the most
  anxiety after treatment (has the highest score
  after treatment)

6
Repeated-Measures Testing with One Sample

• OK, so what do we do then?
• Use the Two Related Samples/Repeated
  Measures/Matched Samples T-Test
• How does this test differ from other t-tests?
• It uses difference/gain scores
• Difference Score scores representing the
  difference between performance on two occasions
• i.e. Time 2 Time 1, or visa-versa

7
Repeated-Measures Testing with One Sample

• Difference Scores

Time 1 Time 2 Difference Score
2 1 1
5 4 1
8 6 2
6 3 3
12 9 3
15 7 8
23 12 11
14 5 9
11 3 8
9 2 7
8
Repeated-Measures Testing with One Sample

• Ho for Repeated Measures Tests
• For a Two-Tailed Test
• There is no difference in scores from Time 1 to
  Time 2
• AKA The average of the population of difference
  scores (µD) 0
• AKA µD µ1 (population mean of Time 1 scores)
  µ2 (population mean of Time 2 scores) 0
• H1 for Repeated Measures Tests
• For a Two-Tailed Test
• There is a difference in scores from Time 1 to
  Time 2
• µD µ1 µ2 ? 0

9
Repeated-Measures Testing with One Sample

• Ho for Repeated Measures Tests
• For a One-Tailed Test
• If the Time 1 scores are lower than Time 2 (if
  theyre higher than you would just use gt
  instead of lt)
• µ1 (population mean of Time 1 scores) lt µ2
  (population mean of Time 2 scores)
• H1 for Repeated Measures Tests
• For a Two-Tailed Test
• The Time 1 scores are equal to or higher than
  Time 2 (if not, use ? instead of ?)
• µD µ1? µ2

10
Repeated-Measures Testing with One Sample

• How do we calculate a Repeated Measures T-Test?
• Same as previous t-tests, just using difference
  scores as opposed to raw scores

11
Repeated-Measures Testing with One Sample

• the mean of the difference scores
• s the standard deviation of difference
  scores
• N the number of difference scores (NOT the
  total number of scores, which is twice the number
  of difference scores)
• df N 1 number of pairs minus 1

12
Repeated-Measures Testing with One Sample

• Review When to use Repeated Measures T-Tests
• 1. One subject provides data for two time points
• 2. One subjects provides data for two
  groups/samples

13
Repeated-Measures Testing with One Sample

• Example 1
• The following data is from an experimental
  treatment that I conducted to help prevent
  depression by taking pessimistic people (who are
  at risk for developing depression) and making
  them more optimistic. The data are levels of
  depression as assessed with the Beck Depression
  Inventory-II (BDI-II), both before and after
  treatment, for a subset of those in the group
  receiving treatment (we also had a group that
  didnt receive treatment).
• Why did we have a no treatment group?

14
Repeated-Measures Testing with One Sample
BDI_at_Time 1 BDI_at_ Time 2 Difference Score
19 19 0
0 1 -1
11 9 2
5 5 0
23 23 0
4 0 4
13 3 10
9 2 7
3 8 -5
3 3 0
15
Repeated-Measures Testing with One Sample

• Example 1
• Difference Scores 0, -1, 2, 0, 0, 4, 10, 7,
• -5, 0
• Diff. Scores2 0, 1, 4, 0, 0, 16, 100, 49, 25, 0
• 17/10 1.7
• s2 195 - (172/10)/9 18.46

16
Repeated-Measures Testing with One Sample

• s v18.46 4.30
• t (1.7 0)/(4.30/v10) 1.25
• Critical t (df 9, two-tailed, p lt .05) 2.262
• We would fail to reject Ho

17
Repeated-Measures Testing with One Sample

• Why not to use Repeated Measures data?
• 1. If subjects are asked to take the same
  measures at two time points, they may respond
  similarly to Time 2 as to Time 1 if they can
  recall their earlier responses.
• Solution Create two measures that are similar,
  but not identical
• Although this involves proving that theyre
  similar

18
Repeated-Measures Testing with One Sample

• Why not to use Repeated Measures data?
• 2. The questions asked on the test at Time 1 may
  tip off subjects to the point of the experiment,
  and contaminate the results.
• Solution Throw in filler items to the measures
  that have nothing to do with your experiment and
  that youll essentially ignore in data analysis
  to throw off your wiley subjects