Curva ROC figuras esquem

1
Curva ROCfiguras esquemáticas
Prof. Ivan Balducci FOSJC / Unesp
2
Receiver Operating Characteristic (ROC) Curve
Each threshold c corresponds to a point (FPR, DR)
on the X-Y plane. A ROC curve is obtained as c
sweeps from - ? to ?.
2
3
Lab Tests What is Abnormal?
4
Desempenho
True Positive
True Negative
A
B
False Negative
False Positive
TP Classe é A e classificamos como A TN
Classe é B e classificamos como B FP Classe é B
e classificamos como A FN Classe é A e
classificamos como B
5
The Cut-off Value Trade off

• Sensitivity and specificity depend on the cut off
  value between what we define as normal and
  abnormal
• Assume high test values are abnormal then,
  moving the cut-off value to a higher one
  increases FN results and decreases FP results
  (i.e. more specific) and vice versa
• There is always a trade off in setting the
  cut-off point

6
Receiver Operating Characteristic (ROC) Curve
Each threshold c corresponds to a point (FPR, DR)
on the X-Y plane. A ROC curve is obtained as c
sweeps from - ? to ?.
6
7
ROC Curve
8
Receiver Operating Characteristic (ROC) Curves
9
Goodness-Of-Fit Other Measures of Model
Performance

• ROC (Receiver Operating Characteristic) Curve
• Sensitivity and Specificity are dependent on a
  given cut-point c.
• An ROC curve is obtained by plotting sensitivity
  against (1-specificity) for an entire range of
  possible cut-points.
• The area under the ROC curve is a measure of the
  models ability to discriminate between event and
  non-event in the following fashion
• Among all possible pairs (event, non-event), the
  proportion of pairs for which the event has
  higher probability than the corresponding
  non-event is equal to the area under ROC.

10
ROC Curve

• Interpretation Area Under ROC Curve
• If randomly selected pairs of subjects (one with
  event and one with non-event) are classified in
  such a way that the subject with higher estimated
  probability of the event belongs to the event
  group and the other subject to non-event group,
  then the proportion of correctly classified such
  pairs of subjects would be equal to the area
  under ROC
• Generally Accepted Rule
• ROC 0.5 no discrimination (no
  better than coin toss)
• 0.7 lt ROC lt 0.8 acceptable discrimination
• 0.8 lt ROC lt 0.9 excellent discrimination
• ROC gt 0.9 outstanding
  discrimination
• Roc area is often used to compare predictive
  ability of different models

11
The ROC

• The ROC shows the tradeoff between PFP and PTP
  as the threshold is varied

12
(No Transcript)
13
(No Transcript)
14
Developmental characteristics Cut-points and
Receiver Operating Characteristic (ROC)
Healthy
15
Developmental characteristics Cut-points and
Receiver Operating Characteristic (ROC)
Healthy Sick
16
Developmental characteristics Cut-points and
Receiver Operating Characteristic (ROC)
Fals pos 20 True pos82
17
Developmental characteristics Cut-points and
Receiver Operating Characteristic (ROC)
Fals pos 9 True pos70
18
Developmental characteristics Cut-points and
Receiver Operating Characteristic (ROC)
Healthy
19
Developmental characteristics Cut-points and
Receiver Operating Characteristic (ROC)
Healthy Sick
20
Developmental characteristics Cut-points and
Receiver Operating Characteristic (ROC)
Fals pos 20 True pos82
21
Developmental characteristics Cut-points and
Receiver Operating Characteristic (ROC)
Fals pos 9 True pos70
22
Evaluating the results

• How can we measure the performance of a feature
  matcher?

1
0.7
truepositiverate
0
1
false positive rate
0.1
23
ROC curve (Receiver Operator Characteristic)

• How can we measure the performance of a feature
  matcher?

1
0.7
truepositiverate
0
1
false positive rate
0.1

• ROC Curves
• Generated by counting current/incorrect
  matches, for different threholds
• Want to maximize area under the curve (AUC)
• Useful for comparing different feature matching
  methods
• For more info http//en.wikipedia.org/wiki/Recei
  ver_operating_characteristic

24

• Curva ROC

• Cut-off
• Área sob curva

• especificidade
• sensibiidade