Computerized Clinical Decision Support Systems CDSSs

1
(No Transcript)
2
Computerized Clinical Decision Support Systems
(CDSSs)

• Questions to be addressed
• What do trials say about the state of CDSS
  evolution?
• How do CDSSs need to improve to bring greater
  success?
• How do trials need to be done to ensure fair
  evaluations? (Be alert as we go along!)

3
Effects of Computerized Clinical Decision
Support Systems on Practitioner Performance and
Patient Outcomes A Systematic Review
Amit Garg MD, Neill Adhikari MD, Heather McDonald
MSc, Patricia Rosas-Arellano MD,PhD, Phillip J.
Devereaux MD,, Joseph Beyene PhD, Justina Sam,
R. Brian Haynes MD, PhD Departments of Clinical
Epidemiology and Biostatistics, McMaster
University Departments of Medicine, McMaster
University, University of Toronto, and
University of Western Ontario Department of
Biostatistics and Epidemiology, University of
Western Ontario Ref Garg et al. Effects of
computerized clinical decision support systems on
practitioner performance and patient outcomes a
systematic review. JAMA 20052931323-38.
4

• Context Computerized Clinical Decision Support
  Systems
• Software designed to directly aid in clinical
  decision making in which characteristics of
  individual patients are matched to a computerized
  knowledge base for the purpose of generating
  patient specific assessments or recommendations.

Rules / Algorithms

• INPUT
• Patient characteristics
• Automated through EMR
• By extra research staff
• By existing health care staff
• By the patient
• By the practitioner

Computer

• OUTPUT
• Recommendations
• delivered to health
• care provider
• Directly by computer
• By pager
• By extra research staff
• By existing health care staff

• Outcomes
• Provider performance
• Patient outcomes

integrate into workflow
5
Examples of Clinical Decision Support Software
Alert Remind Critique Interpret Predict Diagn
ose Recommend
Highlight out of range serum potassium Remind
about need for hepatitis B vaccination Reject
med order when allergy present Interpret an
electrocardiogram Calculate risk for cardiac
disease Algorithm for ruling out fracture in
ankle injury Suggest new orders for active care
6

• Are these systems clinically effective?
• Hunt DL, Haynes RB et al. JAMA Oct 1998 1339
  46 (March 1998)
• 68 clinical studies (prospective cohort studies
  or RCTs),
• where addition of CDSS was compared to
  routine care
• results counting positive results on 50
  outcomes measured
• Improvement on practitioner performance
• In 14 of 19 (74) preventative care systems
• In 19 of 26 (73) active care systems
• In 9 of 15 (60) drug dosing systems

?
?
?
7

• Update Review to September 2004
• Data Sources
• We screened additional 3997 citations (MEDLINE,
  EMBASE,
• SciSearch, INSPEC, reference lists) in
  duplicate.
• We reviewed 226 full text articles in duplicate.
• 100 trials met our criteria
• Study Selection
• We included prospective cohort studies or RCTs
  where patient care
• with CDSS was compared to care without.
• CDSS used by health care professional to guide
  decision making.
• CDSS provided patient specific recommendations.

8
100 clinical studies of CDSS (1972 to 9/2004)
Number of trials increased with time 1 in
1970-74 4 in 1975-79 10 in 1980-84 13 in
1985-89 20 in 1990-94 26 in 1995-99, 26 in
2000-Sept 2004.
9
100 clinical studies of CDSS (1972 to 9/2004)

• Most trials conducted in US
• US 69
• UK 14
• Canada 5
• Australia 4
• Italy 2
• Austria, France, Germany, Israel, Norway, and
  Switzerland - each 1
• Methodological Quality
• 88 RCTs, 39 cluster, 24 reported power
  calculation, average methods score 7 out of 10
  (range 2 to 10)

10

• Characteristics of CDSS Studies how created
• Creators of CDSS are authors of the paper
  59
• Public source of funding reported for study
  69
• Graphical user interface
  15
• CDSS improved with pilot testing
  20
• CDSS a part of electronic medical record or
• computer order entry system
  46
• CDSS suggested new orders (vs. critiquing)
  68
• Feedback from CDSS at time of patient care
  78

11

• Characteristics of CDSS Studies where used and
  who used them
• CDSS used in academic centre
  68
• CDSS used in hospital inpatients
  40
• Practitioner was a physician (versus nurse etc.)
  92
• CDSS used by physicians-in-training
  43
• Practitioners trained in CDSS use
  27

12

• Characteristics of CDSS Studies INs and OUTs
• INPUT Clear who entered data into computer
  70
• - automated through EMR
  25
• - by staff paid by project funds
  19
• - by health care staff (nurses, clerks) or
  patients 25
• - by practitioner / decision maker
  31

13
Are CDSSs clinically effective?
14

• Did CDSS improve practitioner performance?
• Update 100 studies
• counting positive results on 50 outcomes
  measured

Examined in 97 studies, 63 cited improvement
(65)
?
?
?
15

• Did CDSS improve patient outcome?
• Update 100 studies

Examined in 52 studies, 7 cited improvement (13)
16
Reminder System 40 studies
Improved Practitioner Performance - 76 -
Improved Patient Outcome - 0 -

• Screening, counseling, vaccination, testing,
  medication use, or the identification of at-risk
  behaviors
• CDSS successes were typically demonstrated in
  ambulatory care, although one successful system
  was used in hospitalized patients

17
Disease Management Systems 37 studies
Improved Practitioner Performance - 62 -
Improved Patient Outcome - 19 -
Most are RECOMMENDATIONS. Range of problems,
for example - diabetes care - cardiovascular
prevention - incontinence in the elderly -
advanced directives - ventilator support -
infertility - corollary orders - reduce
unneeded health care utilization
18
Improved Practitioner Performance - 62 -
Improved Patient Outcome - 11 -
Drug Dosing 29 studies
aminoglycosides wafarin
dosing - - -
- - (coumadin) heparin
insulin

digoxin
- - TPN

lidocaine
- theophylline /
- - - -
- aminophylline multiple drugs
- - - -
Improved symptoms post tPA
HbA1C
Decreased length of hospital stay
19
Improved Practitioner Performance - 40 -
Improved Patient Outcome - 0 -
Diagnosis 10 studies
cardiac disease (EKG) -
- pediatric conditions
- depression, psychiatry - - -
- - acute abdo pain
- -
admission to CCU
20
Which factors influence the reporting of CDSS
success on practitioner performance? (Meta-regre
ssion)
21
Automated prompt to use CDSS
INPUT
integrate into workflow
Studies where users were automatically prompted
to use the system described better performance
compared to studies where users had to actively
initiate the system 73 vs. 49 success,
p0.02 unadjusted OR 2.8 (95 CI 1.2 - 6.6)
adjusted for methodological quality OR 3.0
(95 CI 1.2 7.1)
Compared to manual initiation, automatic
prompting may improve integration into
practitioner workflow, as well as provide better
opportunities to correct inadvertent deficiencies
in care.
22
Studies where the trial authors were also the
CDSS creators reported better performance
compared to those studies where the trialists
were independent of the CDSS development
process 72 vs. 43 success, p 0.008
unadjusted OR 3.4 (95 CI 1.4 to 8.4) adjusted
OR 3.0 (95 CI 1.3 8.3)

• Many reasons for this finding are possible
• developers enthusiasm for a created product
• better access to technical support and training
• need for on-site promotion and tailoring
• biases in assessing outcomes
• selective publication of successful trials.
• Most of the CDSSs in this review were home
  grown, and the importance of local champions to
  facilitate implementation cannot be
  underestimated.

23
To what extent should a CDSS be proven beneficial
before mass deployment?

• Clearly testing is required
• a CDSS can have unanticipated deleterious effects
  when used in patient care, including wasting time
  and resources
• If a CDSS creator/promoter claims health
  benefits, an RCT standard can be applied.
• Using such a standard, the majority of available
  systems are not yet ready for mainstream use.

24
To what extent should a CDSS be proven beneficial
before mass deployment?

• Most trials did not enroll enough patients for
  adequate statistical power to detect improvements
  in patient outcomes, even if in truth these
  improvements did exist.
• It would be terrific to have at least 2 or more
  adequately powered trials for a given CDSS, at
  least one by someone other than the developer.

25
To what extent should a CDSS be proven beneficial
before mass deployment?

• A CDSS is limited by the cumulative knowledge
  used to program its recommendations. It would be
  unrealistic to require repeat CDSS testing every
  time advances in the knowledge base become
  available.
• but keeping systems up to date is an issue.

26

• Cost Considerations
• Claims that CDSSs improve efficiency and reduce
  costs should be backed up by evidence
• the current supporting evidence is limited at
  best
• many studies suggested the CDSS was inefficient,
  requiring more time and effort from the user
  compared to paper

27
Conclusions for CDSS Review
Many CDSS evaluation studies report improved
practitioner performance. The effects on
patient outcomes remain understudied and, when
studied, inconsistent at present.
28
Areas for improvement

• Better interface design
• Faster response
• Better integration into work flow
• Based on processes that are
• strongly related to important patient outcomes
• often not done properly
• acceptable to practitioners

29
Questions?
30
(No Transcript)
31
Methodologic Issues in Testing CDSSs

• To reduce bias in testing

32
The intervention should be clearly defined

• Inputs
• Evidence base
• Logic
• Outputs
• Operators
• Ancillary interventions
• Place in work-flow

33
The question addressed should be clearly
specified

• PICO(T)
• Patients
• Intervention
• Comparator
• Outcomes
• Time

34
Allocation to study groups

• Participants should be randomly assigned to
  comparison groups with blinded allocation
• Usually cluster allocation is needed to avoid
  contamination at the level of the unit (eg,
  clinic, ward) or sometimes by provider (ie,
  clinician), rather than by patient

35
Comparability of study groups

• Baseline characteristics that are related to
  study outcomes should be balanced
• Computer skills of clinicians
• Clinical skills of clinicians
• Disease condition of patients
• Baseline differences should be adjusted in the
  analysis

36
Outcome measures

• Process measures
• Should be related to outcomes
• Should be reproducibly and objectively assessed
  or independently assessed
• Patient outcomes
• Should be as patient important as possible
• Should be objective or independently assessed
  (ie, blinded to study group)

37
Sample size

• Power should be adequate (gt80) to detect
  moderate effects for key process and outcome
  variables
• Cluster allocation needs to be taken into account

38
Follow-up

• Should be perfect or very close to it

39
Questions?
40
(No Transcript)