Prognostic models in the ICU

1
Prognostic models in the ICU

• From development to clinical practice

L. Minne, MSc. Dr. S. Eslami, PharmD Dr. D.A.
Dongelmans, MD Prof. Dr. S.E.J.A. de Rooij,
MD Prof. Dr. A. Abu-Hanna Dept. of Medical
Informatics Dept. of Intensive Care Academic
Medical Center Amsterdam, the Netherlands
Prof. Dr. E. de Jonge, MD Dept. of Intensive
Care Leiden University Medical Center Leiden, the
Netherlands
2
Use of prognostic models
1) Benchmarking
2) Decision-making
Observed mortality 25 15
Expected mortality 30 12
Estimates from prognostic model
SMR 0.83 1.25
3
Use of prognostic models
1) Benchmarking
2) Decision-making
Your probability to survive is -7.7631 (SAPS
II score 0.0737) (0.9971 (ln (SAPS II score
1)))
4
Barriers for use in clinical practice

• Lack of evidence for
• External validity
• Clinical credibility
• Impact on decisions and patient outcomes
• Selffulfilling prophecy
• Population level vs
• individual level

5
Overview of our research project

1. Identify prognostic models, their validity and
   use in clinical practice
2. Assess prognostic model behaviour over time
   effects on benchmarking
3. Assess clinicians predictions, (need for)
   prognostic models, their validity and impact in
   decision-making

6
Benchmarking Temporal validation
Upper control limit (usually at 3 sigma)
Mean value
Standardized Mortality Ratio
Lower control limit (usually at 3 sigma)
Time
7
Benchmarking Temporal validation
SMR gt 1 in 15 of the hospitals
8
Benchmarking Temporal validation
Effect of continuous updating (first level
recalibration)
p16
Data used for recalibration
p19
Data used for recalibration
Time
9
Benchmarking Temporal validation
SMR gt 1 in 35 of the hospitals ? effect on
quality of care assessment!
10
Decision-making Model development
Demography
Admission
Age Gender ...
Physiology Laboratory ...
Outcomes
Mortality (Length of Stay) (...)
During Stay
organ scores day1 organ scores day2 organ scores
day3
SOFA
11
Decision-making Model development
25
SAPS
Day 3
Day 2
Day 4
Day 1
Day 5
3
1
1
Renal
0
0
0
Hepatic
3
4
3
Circulatory
3
4
4
Respiratory
0
0
0
Neurological
0
0
0
Coagulation
9
9
8
12
12
SOFA score
d 3
12
Decision-making Model development
LP a0 a1SAPS a2admission_type a3day
A4number_of_readmissions b1 Pattern1 b2
Pattern2
Example at day 3 LP -9.3 0.005SAPS -0.0343
1.232 1.85 SOFAH,H 1.1 SOFAM,H,H
13
Decision-making Model performance
14
Decision-making Model performance
15
Decision-making The end-of-life decision-making
process

• Observation of multidisciplinary meetings ?
  poorly structured no clear guidelines
• Factors (implicitly) considered in decision
• Degree of organ failure
• Patient preferences
• Severity of illness
• Chance of cognitive limitations
• Wish to receive objective information

16
Conclusions and future work

• Decision-making process unstructured
• Possible role for mathematical models
• But insufficient evidence on their impact and
  external validation
• Before-after study to measure impact on
  decision-making

17
Any questions?
18
Decision-making Human predictions
Nurses Physicians
AUC 0.89 0.88
Var 6-7 7-8
Kappa 47.3-55.1