Decision Support Systems I

1
(No Transcript)
2
Decision Support SystemsProbabilistic Principles
Edward H. Shortliffe, MD, PhD Department of
Biomedical Informatics Columbia University

• Biomedical Informatics A Course for Health
  Professionals
• Woods Hole Marine Biological Laboratory
• September 25, 2006

3
Biomedical Informatics Textbook (3rd edition)
Bio
Springer Verlag - 2006
4
Biomedical Informatics

• The scientific field that deals with the storage,
  retrieval, sharing, and optimal use of biomedical
  information, data, and knowledge for problem
  solving and decision making.

Biomedical informatics touches on all basic and
applied fields in biomedical science and is
closely tied to modern information technologies,
notably in the areas of computing and
communication.
5
Computer-Assisted Decision Support

• Examples of functionalities
• Generic information access tools (e.g., Medline,
  results reporting systems)
• Patient-specific consultation systems
• Advice regarding diagnosis
• Advice regarding optimal workup
• Advice regarding therapy or patient management
• Critiques reactions to users hypotheses
  regarding patients and their proper management
• Browsing tools that mix generic and
  patient-specific elements (e.g., electronic
  textbooks of medicine)
• Monitoring tools that generate warnings or advice
  as needed (advice as a byproduct of patient care)

6
Computer-Assisted Decision Support

• Examples of available methodologies
• Protocols and algorithms (clinical guidelines)
• Clinical databanks
• Mathematical models (often physiologic)
• Statistical pattern recognition and neural
  networks
• Bayesian statistics and Bayesian networks
• Decision analysis
• Artificial intelligence (expert systems)
• Syntheses of various techniques

7
Screening Test for Occult Cancer

• 100 patients with occult cancer 95 have "x" in
  their blood
• 100 patients without occult cancer 95 do not
  have "x" in their blood
• 5 out of every 1000 randomly selected individuals
  will have occult cancer

SENSITIVITY
SPECIFICITY
PREVALENCE
8
2 X 2 Table
100,000
If a patient has x in his blood, chance of
occult canceris 475 / 5475 8.7
9
Standard Terminology
True Positives (TPs)
False Negatives (FNs)
False Positives (FPs)
True Negatives (TNs)
Entire Population
10
Definitions
11
Positive Predictive Value Formula
(Sens)(Prev)
PV
(Sens)(Prev) (1-Spec)(1-Prev)
12
Detection of Prostatic Cancer by Solid-Phase
Radioimmunoassay of Serum Prostatic Acid
Phosphatase
13
Editorial

• The clear implication of the accompanying report
  is that mass screening on the basis of a blood
  test alone can reverse this gloomy experience of
  fatal delays in diagnosis of prostate cancer.

New England Journal of Medicine December 22, 1977
14
Medical Journal Advertisement to Physicians
15
Medical Journal Advertisement to Physicians
Posed by a Professional Model
16
Medical Journal Advertisement to Physicians
17
Advertisement

• (You should be aware of) a new blood test called
  the Male-P.A.P. test ... a new, more sensitive
  method that your physician can use to detect
  chemical signals of a cancerous growth in the
  prostate. ... And even though all lab tests
  must be ordered by a physician, we believe that
  you should know the facts.

New York Times, January 21, 1979
18
Sensitivity
of patients
of positive tests
sensitivity
Patients with prostate 113 79 70cancer
Stage I 24 8 33 Stage II 33 26 79 Stage
III 31 22 71 Stage IV 25 23 92
19
Specificity
of patients
of positive tests
specificity
Patients without prostate 217 13 94cancer
Normal controls 50 0 BPH 36 2 After total
prosta- 28 1 tectomy Other cancers 83 9 Misc.
GI disorders 20 1
20
Use As Screening Test

• Without rectal examination
• Sensitivity 70 Specificity 94
• Prevalence 33/100,000
• PV 0.41 (i.e., 1 in 244 subjects)
• With rectal examination
• Sensitivity 33 Specificity 94
• Prevalence 33/100,000
• PV 0.19 (i.e., 1 in 526 subjects)

21
When is the test useful for screening?

• Suppose patient has a nodule on rectal
  examination
• Sensitivity 79
• Specificity 94
• Prevalence 50 !!
• PV 93 (chance of cancer if acid phosphatase
  is positive)
• PV- 82 (chance that there is no cancer if
  acid phosphatase is negative)

22
Predictive Values in Patients with a Nodule
PAP
93
50
PAP-
18
23
Combining Tests For Screening

• If a prostate biopsy is now performed, it needs
  to be considered as another test.
• Specificity 100
• Sensitivity depends on talent and statistics of
  surgeon doing the procedure
• Prevalence is 50 if acid phosphatase has not
  been measured, but is 93 if acid phosphatase is
  positive and 18 if acid phosphatase is negative.

24
Sequential Testing
BX
100
PAP
93
50
BX
100
PAP-
18
25
Chance of Cancer after Negative Biopsy
Sensitivity of Biopsy
50 87 10
70 80 6
90 56 2
Acid Phosphatase positive (93 chance before
biopsy) Acid Phosphatase negative (18 chance
before biopsy)
26
Sequential Testing
BX
100
PAP
93
BX-
56
50
BX
100
PAP-
18
BX-
lt 2
27
BAYES THEOREM
OR
28
Typical Assumptions with theUse of Bayes' Theorem

• Completeness (for example, all men either have or
  do not have prostate cancer there are no other
  possibilities)
• Mutual exclusivity (for example, if a man has
  prostate cancer, he cannot simultaneously NOT
  have prostate cancer)
• Conditional independence (for example, acid
  phosphatase and a biopsy result ARE conditionally
  independent tests rectal exams and acid
  phosphatase may NOT be conditionally independent)

29
References

• Foti et al. Detection of prostate cancer by
  solid-phase radioimmunoassay of serum prostatic
  acid phosphatase. New England Journal of
  Medicine 2971357-1361 (1977)
• Watson, R.A. and Tang, D.B. The predictive
  value of prostatic acid phosphatase as a
  screening test for prostatic cancer. New
  England Journal of Medicine 303497-499 (1980)
• Berwick, D.M., Fineberg, H.V., and Weinstein,
  M.C. When doctors meet numbers. American
  Journal of Medicine 71991 (1981)

30
What is a Positive Test?

• All the analysis has assumed that it is clear
  whether a test is positive or negative
• In reality, many tests involve continuous values
  so that one result may be more positive than
  another
• How should one define the cut-off at which a test
  is judged to be abnormal?

31
Continuously Valued Variables
Result
32
Continuously Valued Variables

• Fewer false positives (more conservative)
• More false negatives
• Higher specificity
• Lower sensitivity

Normal cutoff
Result
33
Continuously Valued Variables
Result

• Fewer false negatives (more aggressive)
• More false positives
• Higher sensitivity
• Lower specificity

34
Receiver Operating Characteristic(ROC) Curves
Test B
Test A
True Positive Rate Sensitivity
False Positive Rate 1 - Specificity
35
The Importance of the Gold Standard

• Evaluating the value of a new test requires
  having some other method for determining truth
• Methods for determining truth are called gold
  standards
• Gold standards are often expensive, time
  consuming, uncomfortable, or risky
• Biopsies
• Major invasive procedures or surgery
• Autopsies
• Integrated opinions of super experts
• We often seek simple, inexpensive, rapid, and
  safe tests that can perform almost as well as the
  gold standard

36
Classification versus Planning

• Both are probabilistic reasoning activities
• Classification interpret data that may not
  deterministically characterize an object of
  interest
• Planning decision today makes assumptions about
  likely outcomes of actions and the desirability
  of those potential outcomes
• Issues recur frequently in biomedicine
• Classification under uncertainty characterizes
  much of the work in biology and in clinical care
• Classification diagnosis in clinical care
• Planning therapy or management in clinical
  care

37
Diagnosis vs Management

• Diagnosis What does this patient have?
• What is true about the world?
• Inherently a problem of probabilistic
  inferenceindirect observations from which
  unobservable causative explanations may be
  inferred
• The question does not involve resource commitments

38
Diagnosis vs Management

• Management What should I do for this patient?
• What test should I do next?
• How should I manage the patients condition?
• Combines probabilistic issues with value
  judgments (cost-benefit tradeoffs, using some
  metric of value)
• Such questions generally involve some commitment
  of resources (money, time, risk, discomfort,
  etc.)
• Of course, diagnosis and management are linked
  concepts

39
Decision Analysis

• 95 y/o male with a new lung nodule on chest xray
• High likelihood of cancer based on appearance
• Mayo Clinic (Minnesota) no surgery or other
  therapy given patients age
• Sloan-Kettering (New York) surgery to remove the
  portion of lung containing the tumor
• Patient wanted a third opinion from experts on
  complex decision making
• Went to Boston for evaluation by Decision
  Analysis service at Tufts-New England Medical
  Center

40
Possible Outcomes if Nothing is Done
41
Possible Outcomes ifSurgery is Performed
42
But Wait...! Are we ignoring another possible
approach?
43
Another option became clear...
44
Reference

• Moroff SV Pauker SG . What to do when the
  patient outlives the literature, or DEALE-ing
  with a full deck. Medical Decision Making, 1983
  3(3)313-38.
• DEALE Declining exponential approximation of
  life expectancy

45
Summary Comments Regarding Decision Support

• In informatics, what fundamentally guides us is
  the creation of systems that implement the
  approaches presented today, plus other methods
  for simulating and/or supporting clinical
  decision making
• Systems need to be viewed as implementations of
  fundamental theories and methods that is the
  source of their power

46
(No Transcript)