Clinical Decision Support: Miracle or Mirage ?

1
Clinical Decision Support Miracle or Mirage ?

• Dr Jeremy Rogers MRCGP
• Clinical Research Fellow
• Medical Informatics Group
• University of Manchester

2
Talk Outline

• Why we need it
• What does decision support mean ?
• Work so far
• Why we dont use it

3
Talk Outline

• Why we need it
• What does decision support mean ?
• Work so far
• Why we dont use it

4
Drivers for decision support

• Growth of medical knowledge
• Approx 100 articles were published in 1966 from
  RCTs
• Over 10,000 annually by 1995 (Chassin, 1998)
• The scarcely tolerable burden of information
  that is imposed taxes the memory but not the
  intellect (GMC 1993)
• Pressures to use knowledge
• Evidence based medicine
• National service frameworks
• Clinical Governance
• Cost e.g. 5.5M in 37 Days for one patient at
  Duke
• Post genomic individualised medicine

5
Drivers for decision support

• Public recognition of medical error
• IOM To err is human (2000) Crossing the
  quality chasm (2001)
• More people die from medical errors than from
  breast cancer or AIDS or motor vehicle accidents
• Jessica Santillan case17 year old who had a
  heart and lung transplant from a donor with an
  incompatible blood group in Feb 2003 at Duke,
  and died after a re-do 13 days later

6
Committee on Quality of Health Care in America

• (The American) health care delivery system is in
  need of fundamental change
• The current care systems cannot do the job
• Trying harder will not work
• Changing systems of care will

7
Talk Outline

• Why we need it
• What does decision support mean ?
• Work so far
• Why we dont use it

8
Kinds of decision

• Diagnosis
• Intervention
• Prognosis

9
Kinds of support

• Active vs Passive support
• Making specific suggestions one off, or
  continuing ?
• Critiqueing recorded actions screw-up detection
• Tweaking / filtering information display
• Intelligent image processing
• Reminders ? Alerts ?
• Decision support, or decision making ?
• Do we expect human to learn from device ?

10
Drowning in dataThe case for DS in display
filtering
EPR - Dr Kildare - 26th Oct 2000
John Doe
36 yrs
Engineer
Married, 2 children
Results
Appt
Letters
This Visit
Encounters
Code
Notes
Action
12.10.96 Coryza chest NAD reassure13.10.96
URTI wheezy amoxycillin20.10.96 Anxiety child
admitted to H reassure24.10.96 PEFR 300
10.11.96 PEFR 400 CXR requested12.11.96 CXR
Basal Consolidation erythromycin27.11.96
Chest clear 07.03.97 Depression death in
family paroxetine19.04.97 Gastoenteritis
reassure01.06.97 rpt Rx paroxetine18.10.97
Sick note 03.03.98 Viral URTI PEFR 350
salbutamol04.03.98 WCC NAD 30.06.98 PMR
report BP, ECG NAD 15.09.98 Eczema
hydrocortisone05.11.98 Depression
paroxetine03.01.99 Fibrositis trigger spot lwr
back ibuprofen17.02.99 Allergic Asthma PEFR
300 salbutamol21.03.99 Chest Inf L base
erythromycin07.10.99 Med4 anxious 26.01.00
Asthma Review Repeat Rx Salbutamol
Salbutamol inh 2 puff qds 1op
PEFR
550 l /min
Asthma
Influvac im BN 035679A4
Chest NAD. No Problems.
WCC
11
Drowning in dataThe case for DS in display
filtering
EPR - Dr Kildare - 26th Oct 2000
John Doe
36 yrs
Engineer
Married, 2 children
Results
Appt
Letters
This Visit
Encounters
Code
Notes
Action
12.10.96 Coryza chest NAD reassure13.10.96
URTI wheezy amoxycillin20.10.96 Anxiety child
admitted to H reassure24.10.96 PEFR 300
10.11.96 PEFR 400 CXR requested12.11.96 CXR
Basal Consolidation erythromycin27.11.96
Chest clear 07.03.97 Depression death in
family paroxetine19.04.97 Gastoenteritis
reassure01.06.97 rpt Rx paroxetine18.10.97
Sick note 03.03.98 Viral URTI PEFR 350
salbutamol04.03.98 WCC NAD 30.06.98 PMR
report BP, ECG NAD 15.09.98 Eczema
hydrocortisone05.11.98 Depression
paroxetine03.01.99 Fibrositis trigger spot lwr
back ibuprofen17.02.99 Allergic Asthma PEFR
300 salbutamol21.03.99 Chest Inf L base
erythromycin07.10.99 Med4 anxious 26.01.00
Asthma Review Repeat Rx Salbutamol
Salbutamol inh 2 puff qds 1op
PEFR
550 l /min
Asthma
Influvac im BN 035679A4
Chest NAD. No Problems.
WCC
12
Goal of support

• Influence outcome
• Good things more likely bad things less likely
• Outcomes
• Fatal events are only the tip of the iceberg
• Easiest to measure, and most dramatic, but.
• Non fatal events
• Side effects
• Sub-optimal treatment
• Inappropriate treatment
• Non harmful events
• Inefficiency Confusion
• Inappropriate resource consumption
• Bed stay
• Repeated re-investigation

13
Kinds of DS technology

• Statistical
• 93.467 of the time, things that quack and have
  webbed feet are ducks
• Model-based
• Its definitely a duck because you told me its
  mother was a duck
• Neural Networks
• Of all the things youve shown me so far, it
  looks most like the ones you said were ducks.

14
Talk Outline

• Why we need it
• What does decision support mean ?
• Work so far
• Why we dont use it

15
The Story so far

• Three decades of research into computer aids
  for medical decision making have resulted in
  thousands of systems and a growing number of
  successful clinical trials
• BMJ 1997315891 (4 October)

16
Projects past and present

• Acute Abdominal Pain (1972)
• Mycin (1977)
• Internist/QMR (1980s)
• DXPlain
• ILIAD
• Sophie
• Medical Logic Modules (Arden Syntax) (1992)

• ProFORMA
• Protégé
• PRODIGY (1997-)
• Prescribing Indicators
• Isobel (2000)
• NHS Direct (2000)
• Knowledge Coupling (PKC.com)

17
1970s 1980sBehold, the Oracle
18
Mycin Shortliffe 1970s, Stanford

• gt (mycin)
• ------ PATIENT-1 ------
• Patient's name Sylvia Fischer
• Sex female
• Age 27
• ------ CULTURE-1 ------
• From what site was the specimen for CULTURE-1
  taken? blood
• How many days ago was this culture (CULTURE-1)
  obtained? 3
• ------ ORGANISM-1 ------
• Enter the identity (genus) of ORGANISM-1 unknown
• The gram stain of ORGANISM-1 ?
• A GRAM must be of type (MEMBER ACID-FAST POS NEG)
• The gram stain of ORGANISM-1 neg
• Is ORGANISM-1 a rod or coccus (etc.) rod
• What is the AEROBICITY of ORGANISM-1? why
• It is known that
• 1) THE GRAM OF THE ORGANISM IS NEG
• 2) THE MORPHOLOGY OF THE ORGANISM IS ROD

• What is the AEROBICITY of ORGANISM-1? aerobic
• Is Sylvia Fischer a compromised host? yes
• Is Sylvia Fischer a burn patient? If so, mild or
  serious? why
• It is known that
• 1) THE SITE OF THE CULTURE IS BLOOD
• 2) THE GRAM OF THE ORGANISM IS NEG
• 3) THE MORPHOLOGY OF THE ORGANISM IS ROD
• Therefore,
• Rule 52
• If
• 1) THE BURN OF THE PATIENT IS SERIOUS
• Then there is weakly suggestive evidence (0.4)
  that
• 1) THE IDENTITY OF THE ORGANISM IS
  PSEUDOMONAS
• Is Sylvia Fischer a burn patient? If so, mild or
  serious? serious
• Findings for ORGANISM-1
• IDENTITY ENTEROBACTERIACEAE (0.800)
  PSEUDOMONAS (0.760)
• Is there another ORGANISM? (Y or N) Y
• ------ ORGANISM-2 ------

19
Abdominal Pain De Dombal (1972)

• A multicentre study of computer aided diagnosis
  for patients with acute abdominal pain was
  performed in eight centres with over 250
  participating doctors and 16,737 patients.
• Performance in diagnosis and decision making was
  compared over two periods a test period (when a
  small computer system was provided to aid
  diagnosis) and a baseline period (before the
  system was installed). The two periods were well
  matched for type of case and rate of accrual.
• The system proved reliable and was used in 75.1
  of possible cases.
• User reaction was broadly favourable.

20
Abdominal Pain De Dombal

• During the test period improvements were noted in
  diagnosis, decision making, and patient outcome.
  
• Initial diagnostic accuracy rose from 45.6 to
  65.3. The negative laparotomy rate fell by
  almost half, as did the perforation rate among
  patients with appendicitis (from 23.7 to 11.5).
  The bad management error rate fell from 0.9 to
  0.2, and the observed mortality fell by 22.0.
• The savings made were estimated as amounting to
  278 laparotomies and 8,516 bed nights during the
  trial period--equivalent throughout the National
  Health Service to annual savings in resources
  worth over 20m pounds and direct cost savings of
  over 5m pounds. Computer aided diagnosis is a
  useful system for improving diagnosis and
  encouraging better clinical practice. Br Med J
  (Clin Res Ed) 1986 Sep 27293(6550)800-4

21
Knowledge Couplers PKC.com
Larry Weed MD
22
Other successes

• Strong evidence suggests that some CDSSs can
  improve physician performance. Additional
  well-designed studies are needed to assess their
  effects and cost-effectiveness, especially on
  patient outcomes(Johnston 1994)
• Mothers receiving computer-generated reminders
  had 25 higher on-time immunization rate for
  their infants (Alemi, 1996)
• Decision support system was safe and effective
  and improved the quality of initiation and
  control of warfarin treatment by trainee
  doctors(BMJ 19973141252)
• Computerized physician order-entry reduced
  adverse drug events by 55 (Bates, 1998)
• 9 of redundant lab tests at a hospital could be
  eliminated using a computerized system (Bates,
  1998)
• 74 of the studies of preventive healthcare
  reminder systems and 60 of the evaluations of
  drug dosing models reported a positive
  impact(Trowbridge Weingarten, AHRQ, 2001)

23
..and some failures

• (PRODIGY) - No effect was found on the
  management of asthma or angina in adults in
  primary careBMJ 2002 325 941-944
• ..decision support system did not confer any
  benefit in absolute risk reduction or blood
  pressure control BMJ 2000320686-690
• Computerised decision support systems have great
  potential for primary care but have largely
  failed to live up to their promiseBMJ
  19993191281

24
My own failure Prescribing Indicators

• General Practice Repeat Prescribing
• Patients get more drug without seeing doctor
• typically, enough for 1-3 months
• 35 of population at any one time on repeat Rx
• Medication Review
• Accepted part of good clinical practice
• Requirement in NSF for Older People
• But signing authorities is daily batch process
• gt30 scrips per GP per day
• No time for careful review

25
What is Medication Review ? Indicators of
quality prescribing

• Cantrill et al 13 indicators
• Dose too high or too low?
• Course too long ?
• Expensive or useless drug ?
• Interaction with another drug ?
• Contraindicated ?
• By brand ?
• REASON FOR USE DOCUMENTED ?
• Manual system impractical
• Our project (2000-2002)
• computerise the indicators

26
Complex implementation..
Patient ID 4578 Medication DITA906
DISR10514B Problem List 183... (Oedema) 1B17..
(Depressed) G5732. (Paroxysmal Atrial
fibrillation) G73z0. (Intermittent claudication)
H3.... (Chronic obstructive pulm.dis.) 137S.. (Ex
smoker) 246... (O/E - blood pressure reading)
442... (Thyroid hormone tests) 44P... (Serum
cholesterol) 7L172. (Blood withdrawal for testing)
Ontology ID Product Rubric
345031(oral dig) DITA905 Digoxin 125 mcg tab
345031 DITA906 Digoxin 250 mcg tab
345031 DITA908 Digoxin 62.5 mcg tab
G57.. Cardiac dysrhythmias G573. Atrial
fibrillation and flutter G5730 Atrial
fibrillation G5731 Atrial flutter G5732
Paroxysmal atrial fibrillation G573z Atrial
fibrillation and flutter NOS
IDENT 9099269 MAIN digoxin PROPERTIES HAS_DRUG_
FEATURE physiological action WHICH_IS
process ACTS_ON heart HAS_DRUG_FEATURE
indication FOR treating ACTS_ON
supraventricular arrhythmia HAS_DRUG_FEATURE
indication FOR treating ACTS_ON atrial
fibrillation HAS_DRUG_FEATURE information
source IS_PART_OF interaction appendix
Indication Code Rubric
Atrial fibrillation 14AN. H/O atrial fibrillation
3272. ECG atrial fibrillation
3273. ECG atrial flutter
7936A IV pacer control of A Fib
G573. Atrial fibrillation / flutter
27
..and disappointing results

• Machine says there is no recorded indication in
  33 of prescribing events
• BUT high false positive rate 62
• gt it is wrong, most of the time
• Why ?

28
Of all alerts where machine says no indication
29
Problems with the oracle

• Painful data acquisition
• Exhaustive
• Includes exhaustive negative findings
• (which clinicians traditionally largely omit)
• Slow to use
• Poor support for clinical workflow
• Clinician is passive
• Infrequent recognised need

30
1990s More modest aspirations

• Narrow Domain systems
• ECG interpretations
• Arterial blood gas interpretation
• Predicting drug-drug interaction
• Alerts and Reminders
• Out of range test flagging
• But plans for the oracle are resurfacing in
  expectation of imminent EPR

31
Talk Outline

• Why we need it
• What does decision support mean ?
• Work so far
• Why we dont use it

32
You can lead a horse to water

• Three decades of research into computer aids for
  medical decision making have resulted in
  thousands of systems and a growing number of
  successful clinical trials
• Yet only a handful of applications are in
  everyday useBMJ 1997315891 (4 October)

33
Decision Support Systems in Use Today (2003)
http//www.openclinical.org/aisinpractice.html
34
Why ? the domain

• Rigid criteria difficult to apply in chaotic
  settings
• Medical data doesn't fit quantised definitions
• Even complex decision support algorithms require
  simplified and standardised inputs by users
• And descriptive data is very hard to quantise
• Rules are situation specific
• localising decisions to available resource is
  costly
• When are decisions actually made ?
• To be effective, system needs to be physically
  available in situation where decision is made

35
Why ? - the technology

• Highly mobile workforce vs highly static
  computers
• Slow computers
• Crude knowledge bases ? poor performance
• Lack of stats for bayesian approaches
• Crude KR technology for model-based
• Closed software architectures
• Cant integrate 3rd party DS modules with EPR

36
Why ? the law

• Medicolegal aspect of EPR
• Confidentiality consent
• HIPAA
• Medicolegal aspects of DS technology
• Responsibility for action rests with clinician
• Systems that are as effective as clinician
  overall no help if behaviour includes obvious
  clinical howlers
• Burden of recording why did not follow DS advice

37
Why ? the people

• Poor data quality
• Numerical data easy to obtain
• Much of medicine not numerical
• Inconsistent data entry

38
Data Quality(Frequency of recording per GP per
year)

• READ CODE Practice A Practice B
• Sore Throat Symptom 0.6 117
• Visual Acuity 0.4 644
• ECG General 2.2 300
• Ovary/Broad Ligament Op 7.8 809
• Specific Viral Infections 1.4 556
• Alcohol Consumption 0 106
• H/O Resp Disease 0 26
• Full Blood Count 0 838

39
Why? the people

• Poor data quality
• Numerical data easy to obtain
• Much of medicine not numerical
• Inconsistent data entry
• What happened to my clinical autonomy ?
• Interface issuesBMJ 19993181527-1531

• I know what Im doing
• Perception of infallibility
• 88 of the time users requested to bypass PRODIGY
  (Beaumont 1988)
• Reluctance to change clinical practice to fit the
  tool
• Weeds knowledge couplers
• Users intolerant of less than perfect
  performanceBMJ 2003326314

40
Why ? - money

• Through more improved choice of initial
  antibiotics to treat pneumonia, a group of
  mid-west hospitals decreased complications,
  mortality rates and hospital days and costs
• but hospital revenues also decreased as patients
  shifted from higher paying to lower paying DRGs.

• Improved management of diabetic patients through
  frequent e-mail communication can produce better
  outcomes and fewer visits
• but lower physician group revenues under fee
  for service payment.

41
Summary

• Research and commercial products pre-date IOM by
  almost 30 years
• Widespread adoption has not occurred even where
  results were positive
• Significant hurdles remain
• Legal
• Technical - EPR is harder than it looks
• Human factors