Nonparametric tests II

1
Nonparametric tests II

• as randomisation tests

2
Lecture Outline

• Background Nonparametric tests as randomisation
  tests
• The sign test
• The Wilcoxon signed ranks test
• The Mann-Whitney test
• General remarks on randomisation tests
• Brief Review of the course so far

3
after before 640.0 1050.0 70.0 84.0 83.0
77.0 64.0 110.0 420.0 440.0 6.4 4.8
26.0 48.0 2.2 16.0 75.0 340.0 16.0 430.0
4
after before change 640.0 1050.0 -410.0 70.0
84.0 -14.0 83.0 77.0 6.0 64.0 110.0
-46.0 420.0 440.0 -20.0 6.4 4.8 1.6
26.0 48.0 -22.0 2.2 16.0 -13.8 75.0
340.0 -265.0 16.0 430.0 -414.0
5
after before change 640.0 1050.0 -410.0 70.0
84.0 -14.0 83.0 77.0 6.0 64.0 110.0
-46.0 420.0 440.0 -20.0 6.4 4.8 1.6
26.0 48.0 -22.0 2.2 16.0 -13.8 75.0
340.0 -265.0 16.0 430.0 -414.0
schange -414.0 -410.0 -265.0 -46.0 -22.0 -20.0 -14
.0 -13.8 1.6 6.0
6
MTB gt stest 'change' Sign Test for Median Sign
test of median0.000 versus N.E. 0.000
N BELOW EQUAL ABOVE P-VALUE MEDIAN change 10
8 0 2 0.1094 -21.00 MTB gt
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
MTB gt stest 'change' Sign Test for Median Sign
test of median0.000 versus N.E. 0.000
N BELOW EQUAL ABOVE P-VALUE MEDIAN change 10
8 0 2 0.1094 -21.00 MTB gt
14
MTB gt stest 'change' Sign Test for Median Sign
test of median0.000 versus N.E. 0.000
N BELOW EQUAL ABOVE P-VALUE MEDIAN change 10
8 0 2 0.1094 -21.00 MTB gt
which added number of non-zero datapoints (in
this case there are no zeroes)
15
So if we take ten items that might be plus or
minus,
16
So if we take ten items that might be plus or
minus, and randomly choose them, we get the set
of relevant comparisons for our dataset of 8
minus and 2 plus. This is the randomisation part
of the test.
17
So if we take ten items that might be plus or
minus, and randomly choose them, we get the set
of relevant comparisons for our dataset of 8
minus and 2 plus. This is the randomisation part
of the test.
To decide whether our actual dataset is extreme
in the distribution, we calculate the test
statistic in each case - just the number of
plusses. We count in what fraction of cases, the
relevant comparison has a more extreme number of
plusses, that is, either 2 or fewer, or 8 or
more.
18
MTB gt stest 'change' Sign Test for Median Sign
test of median0.000 versus N.E. 0.000
N BELOW EQUAL ABOVE P-VALUE MEDIAN change 10
8 0 2 0.1094 -21.00 MTB gt
19
(No Transcript)
20
(No Transcript)
21
The truth about confidence intervals

. .. .
. . .. -----------------------------------
----------------change -400 -320
-240 -160 -80 0
22
MTB gt stest 'change' Sign Test for Median Sign
test of median0.000 versus N.E. 0.000
N BELOW EQUAL ABOVE P-VALUE MEDIAN change 10
8 0 2 0.1094 -21.00 MTB gt
23
MTB gt stest 'change' Sign Test for Median Sign
test of median0.000 versus N.E. 0.000
N BELOW EQUAL ABOVE P-VALUE MEDIAN change 10
8 0 2 0.1094 -21.00 MTB gt
24
MTB gt stest 0 c3 Sign Test for Median C3 Sign
test of median 0.00000 versus not 0.00000
N Below Equal Above P
Median C3 10 8 0 2
0.1094 -21.00 MTB gt stest 10 c3 Sign Test
for Median C3 Sign test of median 10.00
versus not 10.00 N Below
Equal Above P Median C3
10 10 0 0 0.0020 -21.00

. .. .
. . .. -----------------------------------
----------------change -400 -320
-240 -160 -80 0
25
H0median N Below Equal Above P
Median -50 10 3 0 7
0.3438 -21.00 -40 10 4 0
6 0.7539 -21.00 -35 10
4 0 6 0.7539 -21.00 -30
10 4 0 6 0.7539 -21.00 -25
10 4 0 6 0.7539
-21.00 -20 10 5 1 4
1.0000 -21.00 -15 10 6 0
4 0.7539 -21.00 -10 10
8 0 2 0.1094 -21.00 -5
10 8 0 2 0.1094 -21.00 0
10 8 0 2 0.1094
-21.00 5 10 9 0 1
0.0215 -21.00 10 10 10 0
0 0.0020 -21.00 15 10
10 0 0 0.0020 -21.00

. .. .
. . .. -----------------------------------
----------------change -400 -320
-240 -160 -80 0
26
H0median N Below Equal Above P
Median -50 10 3 0 7
0.3438 -21.00 -40 10 4 0
6 0.7539 -21.00 -35 10
4 0 6 0.7539 -21.00 -30
10 4 0 6 0.7539 -21.00 -25
10 4 0 6 0.7539
-21.00 -20 10 5 1 4
1.0000 -21.00 -15 10 6 0
4 0.7539 -21.00 -10 10
8 0 2 0.1094 -21.00 -5
10 8 0 2 0.1094 -21.00 0
10 8 0 2 0.1094
-21.00 5 10 9 0 1
0.0215 -21.00 10 10 10 0
0 0.0020 -21.00 15 10
10 0 0 0.0020 -21.00

. .. .
. . .. -----------------------------------
----------------change -400 -320
-240 -160 -80 0
27
H0median N Below Equal Above P
Median -50 10 3 0 7
0.3438 -21.00 -40 10 4 0
6 0.7539 -21.00 -35 10
4 0 6 0.7539 -21.00 -30
10 4 0 6 0.7539 -21.00 -25
10 4 0 6 0.7539
-21.00 -20 10 5 1 4
1.0000 -21.00 -15 10 6 0
4 0.7539 -21.00 -10 10
8 0 2 0.1094 -21.00 -5
10 8 0 2 0.1094 -21.00 0
10 8 0 2 0.1094
-21.00 5 10 9 0 1
0.0215 -21.00 10 10 10 0
0 0.0020 -21.00 15 10
10 0 0 0.0020 -21.00

. .. .
. . .. -----------------------------------
----------------change -400 -320
-240 -160 -80 0
28
H0median N Below Equal Above P
Median -50 10 3 0 7
0.3438 -21.00 -40 10 4 0
6 0.7539 -21.00 -35 10
4 0 6 0.7539 -21.00 -30
10 4 0 6 0.7539 -21.00 -25
10 4 0 6 0.7539
-21.00 -20 10 5 1 4
1.0000 -21.00 -15 10 6 0
4 0.7539 -21.00 -10 10
8 0 2 0.1094 -21.00 -5
10 8 0 2 0.1094 -21.00 0
10 8 0 2 0.1094
-21.00 5 10 9 0 1
0.0215 -21.00 10 10 10 0
0 0.0020 -21.00 15 10
10 0 0 0.0020 -21.00

. .. .
. . .. -----------------------------------
----------------change -400 -320
-240 -160 -80 0
29
H0median N Below Equal Above P
Median -50 10 3 0 7
0.3438 -21.00 -40 10 4 0
6 0.7539 -21.00 -35 10
4 0 6 0.7539 -21.00 -30
10 4 0 6 0.7539 -21.00 -25
10 4 0 6 0.7539
-21.00 -20 10 5 1 4
1.0000 -21.00 -15 10 6 0
4 0.7539 -21.00 -10 10
8 0 2 0.1094 -21.00 -5
10 8 0 2 0.1094 -21.00 0
10 8 0 2 0.1094
-21.00 5 10 9 0 1
0.0215 -21.00 10 10 10 0
0 0.0020 -21.00 15 10
10 0 0 0.0020 -21.00

. .. .
. . .. -----------------------------------
----------------change -400 -320
-240 -160 -80 0
30
H0median N Below Equal Above P
Median -50 10 3 0 7
0.3438 -21.00 -40 10 4 0
6 0.7539 -21.00 -35 10
4 0 6 0.7539 -21.00 -30
10 4 0 6 0.7539 -21.00 -25
10 4 0 6 0.7539
-21.00 -20 10 5 1 4
1.0000 -21.00 -15 10 6 0
4 0.7539 -21.00 -10 10
8 0 2 0.1094 -21.00 -5
10 8 0 2 0.1094 -21.00 0
10 8 0 2 0.1094
-21.00 5 10 9 0 1
0.0215 -21.00 10 10 10 0
0 0.0020 -21.00 15 10
10 0 0 0.0020 -21.00

. .. .
. . .. -----------------------------------
----------------change -400 -320
-240 -160 -80 0
31

. .. .
. . .. -----------------------------------
----------------change -400 -320
-240 -160 -80 0
32

. .. .
. . .. -----------------------------------
----------------change -400 -320
-240 -160 -80 0

. .. .
. . .. -----------------------------------
----------------change -400 -320
-240 -160 -80 0
The green values cannot be rejected at the 5
level, while the red values can.
33

. .. .
. . .. -----------------------------------
----------------change -400 -320
-240 -160 -80 0

. .. .
. . .. -----------------------------------
----------------change -400 -320
-240 -160 -80 0
The green values cannot be rejected at the 5
level, while the red values can.
The range of green values is therefore the 95
confidence interval for the median based on the
sign test.
34
The real definition of 95 confidence interval

• is the set of values of a parameter that cannot
  be rejected at the 5 level
• is therefore not the set of values that the
  parameter has a 95 chance of belonging to, as
  many textbooks claim. (This is called a fiducial
  interval.)

35
MTB gt sinterval 'change' Sign Confidence
Interval Sign confidence interval for median
ACHIEVED
POSI N MEDIAN CONFIDENCE CONFIDENCE
INTERVAL TION change 10 -21.000 0.8906
(-265.000, -13.800) 3
0.9500 (-314.640, -8.528) NLI
0.9785 (-410.000, 1.600) 2 MTB gt
36
H0median N Below Equal Above P
Median -50 10 3 0 7
0.3438 -21.00 -40 10 4 0
6 0.7539 -21.00 -35 10
4 0 6 0.7539 -21.00 -30
10 4 0 6 0.7539 -21.00 -25
10 4 0 6 0.7539
-21.00 -20 10 5 1 4
1.0000 -21.00 -15 10 6 0
4 0.7539 -21.00 -10 10
8 0 2 0.1094 -21.00 -5
10 8 0 2 0.1094 -21.00 0
10 8 0 2 0.1094
-21.00 5 10 9 0 1
0.0215 -21.00 10 10 10 0
0 0.0020 -21.00 15 10
10 0 0 0.0020 -21.00

. .. .
. . .. -----------------------------------
----------------change -400 -320
-240 -160 -80 0
37
after before change 640.0 1050.0 -410.0 70.0
84.0 -14.0 83.0 77.0 6.0 64.0 110.0
-46.0 420.0 440.0 -20.0 6.4 4.8 1.6
26.0 48.0 -22.0 2.2 16.0 -13.8 75.0
340.0 -265.0 16.0 430.0 -414.0
schange -414.0 -410.0 -265.0 -46.0 -22.0 -20.0 -14
.0 -13.8 1.6 6.0
97.85
89.06
89.06
97.85
38
MTB gt sinterval 'change' Sign Confidence
Interval Sign confidence interval for median
ACHIEVED
POSI N MEDIAN CONFIDENCE CONFIDENCE
INTERVAL TION change 10 -21.000 0.8906
(-265.000, -13.800) 3
0.9500 (-314.640, -8.528) NLI
0.9785 (-410.000, 1.600) 2 MTB gt

. .. .
. . .. -----------------------------------
----------------change -400 -320
-240 -160 -80 0
39
Why does Minitab give three confidence intervals
for the sign test?

• the p-value for rejecting a value changes in a
  step function at observed values
• so exact confidence intervals are given between
  observed values, at whatever level of confidence
  is attained
• the NLI (Non-Linear Interpolation) confidence
  interval is a confidence trick

40
Lecture Outline

• Background Nonparametric tests as randomisation
  tests
• The sign test
• The Wilcoxon signed ranks test
• The Mann-Whitney test
• General remarks on randomisation tests
• Brief Review of the course so far

41
after before change 640.0 1050.0 -410.0 70.0
84.0 -14.0 83.0 77.0 6.0 64.0 110.0
-46.0 420.0 440.0 -20.0 6.4 4.8 1.6
26.0 48.0 -22.0 2.2 16.0 -13.8 75.0
340.0 -265.0 16.0 430.0 -414.0
schange -414.0 -410.0 -265.0 -46.0 -22.0 -20.0 -14
.0 -13.8 1.6 6.0
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
MTB gt stest 'change' Sign Test for Median Sign
test of median0.000 versus N.E. 0.000
N BELOW EQUAL ABOVE P-VALUE MEDIAN change 10
8 0 2 0.1094 -21.00
MTB gt wtest 'change' Wilcoxon Signed Rank
Test TEST OF MEDIAN 0.000 VERSUS MEDIAN N.E.
0.000 N FOR WILCOXON
ESTIMATED N TEST STATISTIC
P-VALUE MEDIAN change 10 10
3.0 0.014 -46.00 MTB gt
49
MTB gt stest 'change' Sign Test for Median Sign
test of median0.000 versus N.E. 0.000
N BELOW EQUAL ABOVE P-VALUE MEDIAN change 10
8 0 2 0.1094 -21.00
MTB gt wtest 'change' Wilcoxon Signed Rank
Test TEST OF MEDIAN 0.000 VERSUS MEDIAN N.E.
0.000 N FOR WILCOXON
ESTIMATED N TEST STATISTIC
P-VALUE MEDIAN change 10 10
3.0 0.014 -46.00 MTB gt
50
MTB gt stest 'change' Sign Test for Median Sign
test of median0.000 versus N.E. 0.000
N BELOW EQUAL ABOVE P-VALUE MEDIAN change 10
8 0 2 0.1094 -21.00
MTB gt wtest 'change' Wilcoxon Signed Rank
Test TEST OF MEDIAN 0.000 VERSUS MEDIAN N.E.
0.000 N FOR WILCOXON
ESTIMATED N TEST STATISTIC
P-VALUE MEDIAN change 10 10
3.0 0.014 -46.00 MTB gt
The Wilcoxon test is more powerful than the Sign
Test
51
MTB gt sinterval 'change' Sign Confidence
Interval Sign confidence interval for median
ACHIEVED
POSI N MEDIAN CONFIDENCE CONFIDENCE
INTERVAL TION change 10 -21.000 0.8906
(-265.000, -13.800) 3
0.9500 (-314.640, -8.528) NLI
0.9785 (-410.000, 1.600) 2 MTB gt
winterval 'change' Wilcoxon Signed Rank
Confidence Interval ESTIMATED
ACHIEVED N MEDIAN CONFIDENCE
CONFIDENCE INTERVAL change 10 -46
94.7 ( -218, -8) MTB gt
52
MTB gt sinterval 'change' Sign Confidence
Interval Sign confidence interval for median
ACHIEVED
POSI N MEDIAN CONFIDENCE CONFIDENCE
INTERVAL TION change 10 -21.000 0.8906
(-265.000, -13.800) 3
0.9500 (-314.640, -8.528) NLI
0.9785 (-410.000, 1.600) 2 MTB gt
winterval 'change' Wilcoxon Signed Rank
Confidence Interval ESTIMATED
ACHIEVED N MEDIAN CONFIDENCE
CONFIDENCE INTERVAL change 10 -46
94.7 ( -218, -8) MTB gt
The Wilcoxon confidence interval is narrower
53
Sign vs Wilcoxon Signed Ranks
54
Sign vs Wilcoxon Signed Ranks

• Less powerful

• More powerful

55
Sign vs Wilcoxon Signed Ranks

• Less powerful
• Less sensitive
• Wider confidence intervals

• More powerful
• More sensitive
• Narrower confidence intervals

56
Sign vs Wilcoxon Signed Ranks

• Less powerful
• Less sensitive
• Wider confidence intervals
• Uses less information
• only sign of difference

• More powerful
• More sensitive
• Narrower confidence intervals
• Uses more information
• also size of difference

57
after before change 640.0 1050.0 -410.0 70.0
84.0 -14.0 83.0 77.0 6.0 64.0 110.0
-46.0 420.0 440.0 -20.0 6.4 4.8 1.6
26.0 48.0 -22.0 2.2 16.0 -13.8 75.0
340.0 -265.0 16.0 430.0 -414.0
schange -414.0 -410.0 -265.0 -46.0 -22.0 -20.0 -14
.0 -13.8 1.6 6.0
58
Lecture Outline

• Background Nonparametric tests as randomisation
  tests
• The sign test
• The Wilcoxon signed ranks test
• The Mann-Whitney test
• General remarks on randomisation tests
• Brief Review of the course so far

59
(No Transcript)
60
(No Transcript)
61
(No Transcript)
62
(No Transcript)
63
(No Transcript)
64
(No Transcript)
65
Lecture Outline

• Background Nonparametric tests as randomisation
  tests
• The sign test
• The Wilcoxon signed ranks test
• The Mann-Whitney test
• General remarks on randomisation tests
• Brief Review of the course so far

66
In these randomisation tests,

• there is a simple direct connection between the
  null hypothesis and the randomisation procedure
• there is freedom of choice of test statistic
• estimation relies on scales of measurement and so
  is not as principled as hypothesis tests

67
Lecture Outline

• Background Nonparametric tests as randomisation
  tests
• The sign test
• The Wilcoxon signed ranks test
• The Mann-Whitney test
• General remarks on randomisation tests
• Brief Review of the course so far

68
Last remarks

• Randomisation tests are powerful tools
• All parametric and nonparametric tests can be
  understood as randomisation tests
• Nowadays they are used when no others can be
  used.
• NEXT WEEK Conclusion to course and some exam
  questions. READ Chapter 14 of textbook.