Personalized Prediction and Resolution of Clinician Information Needs

1
Personalized Prediction and Resolution of
Clinician Information Needs

• James J. Cimino, M.D.
• Departments of Biomedical Informatics and
  Medicine
• Columbia University

2
1984 1986 1988 1990 1992
1994 1996 1998 2000
2002 2004
3
Studying Information Needs

• Covell DG, Uman GC, Manning PR. Information needs
  in office practice are they being met? Ann
  Intern Med. 1985 Oct103(4)596-9.

4
Findings of Observational Studies

• Information needs occur often
• They are often unresolved
• Computer-based resources are underused
• Lack of knowledge of existence
• Lack of access
• Lack of navigational skills
• Perceived lack of time

5
Information Needs in Clinical Care
?
6
Clinical Information for Decision Support
7
Health Knowledge for Decision Support
8
(No Transcript)
9
Automated Clinical Decision Support
10
Clinical Decision Support Systems
11
Modes of Decision Support
What is the Information Retrieved?
Retrieval Uses Knowledge?
Retrieval Uses Clinical Data?
User Recognizes Need?
Decision Support Mode

• Patient Info Yes Maybe
  No Clinical Data

Knowledge Yes No No
Knowledge
Alerting No Yes
Yes Knowledge
?
12
Infobuttons
i
13
Information Needs of CIS Users

• Common tasks may have common needs
• System knows
• Who the user is
• Who the patient is
• What the user is doing
• What information the user is looking at
• So We may be able to predict the specific need
• User is sitting at a computer!
• So We may be able to get an answer automatically

14
1984 1986 1988 1990 1992
1994 1996 1998 2000
2002 2004
15
Unified Medical Language System

• The purpose of the UMLS is to improve the ability
  of computer programs to understand the
  biomedical meaning in user inquiries and to use
  this understanding to retrieve and integrate
  relevant machine-readable information for users.
• - Donald A.B. Lindberg 1986/1993

16
1984 1986 1988 1990 1992
1994 1996 1998 2000
2002 2004
17
First Attempt The Medline Button

• CIS (WebCISs predecessor) on mainframe
• BRS/Colleague (Medline) on same mainframe
• Get them to talk to each other
• Search using patient diagnoses and procedures

18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
First Attempt The Medline Button

• CIS (WebCISs predecessor) on mainframe
• BRS/Colleague (Medline) on same mainframe
• Get them to talk to each other
• Search using patient diagnoses and procedures

• Technical success
• Practical failure

23
1984 1986 1988 1990 1992
1994 1996 1998 2000
2002 2004
24
(No Transcript)
25
(No Transcript)
26
1984 1986 1988 1990 1992
1994 1996 1998 2000
2002 2004
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
1984 1986 1988 1990 1992
1994 1996 1998 2000
2002 2004
31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
Just in Time Education
37
Just in Time Education
1
MRSA
Understand Information Needs
38
Just in Time Education
2
Get Information From EMR
1
MRSA
Understand Information Needs
39
Just in Time Education
2
Get Information From EMR
1
MRSA
Understand Information Needs
3
Resource Selection
40
Just in Time Education
2
4
Get Information From EMR
Resource Terminology
1
MRSA
Understand Information Needs
3
Resource Selection
41
Just in Time Education
2
5
4
Get Information From EMR
Automated Translation
Resource Terminology
1
MRSA
Understand Information Needs
3
Resource Selection
42
Just in Time Education
2
5
4
Get Information From EMR
Automated Translation
Resource Terminology
6
1
MRSA
Querying
Understand Information Needs
3
Resource Selection
43
Just in Time Education
2
5
4
Get Information From EMR
Automated Translation
Resource Terminology
6
1
MRSA
Querying
Understand Information Needs
3
7
Resource Selection
Presentation
44
Research Issues

• What are the information needs?

45
Portable Usability Lab
Users Workstation
75 foot cable
Video Monitor
Converter Controller
Microphone
Video Converter
Headphones
VCR
Cassette Recorder
46
(No Transcript)
47
Morae screen capture
48
Research Issues

• What are the information needs?

49
Information Resource Use According to Log Files
50
Research Issues

• What are the information needs?

• Which context information is important?

51
Context-Specific Resource Use
52
Context-Dependent Information Needs
53
Research Issues

• What are the information needs?
• Which context information is important

• What resources can satisfy needs?
• How can retrieval be automated?
• What context data are used?
• How are the data translated?
• How are the data transmitted?

54
1984 1986 1988 1990 1992
1994 1996 1998 2000
2002 2004
55
Infobuttons vs. Infobutton Manager
56
(No Transcript)
57
(No Transcript)
58
(No Transcript)
59
(No Transcript)
60
(No Transcript)
61
(No Transcript)
62
(No Transcript)
63
(No Transcript)
64
1984 1986 1988 1990 1992
1994 1996 1998 2000
2002 2004
65
(No Transcript)
66
Infobutton Managers have been Deployed

• New York Presbyterian Hospitals WebCIS
• New York Presbyterian Hospitals Eclipsys
• LDS Hospitals HELP system
• New York State Psychiatric Institutes PSYCKES
• Regenstrief Medical Record System
• Partners Healthcare Systems Knowledgeliink
• Vanderbilt Universitys PC-POETS

67
(No Transcript)
68
(No Transcript)
69
Evaluation of the Columbia Universitys
Infobutton Manager at NYPH

• Use who, what, when, where, and why?
• How usable is it?
• How useful is it?
• What impact is it having?

70
Evaluation Modalities

• System log files
• Pop-up surveys
• On-line feedback
• E-mail surveys

71
Evaluation

• Log file analysis
• On-line survey
• Popup questionnaires
• E-mail survey

• Log file analysis 32 months of data

72
Results System Log Files

• March 2004-October 2006
• IM 3,009 users, 29,541 times

73
Infobutton Manager (IM) Usage
74
Resource Use by Context
75
Results System Log Files

• March 2004 - October 2006
• IM 3,009 users, 29,541 times

• HR 3,609 users, 155,718 times

• October 2006
• IM 281 users, 1,022 times (33/day)
• HR 708 users, 5,744 times (185/day)

76
Resource Use by Context - October
77
Finding a Resource or Question

• Users chose questions 48.7 of the time
• Diagnosis list 10.5-24.3
• Outpatient drug order review 23.0-63.3
• Inpatient drug order review 63.3-78.7
• Users chose at least one resource from HR page
  92.4 of the time (82.6-93.7, depending on user
  type and context)

78
Evaluation

• Log file analysis
• On-line survey
• Popup questionnaires
• E-mail survey

• Log file analysis 32 months of data

• On-line survey 108 comments

79
Results On-line Feedback

• 108 questions or topics submitted by users
• 54 (50) in Laboratory Results Review context
• 30 were about test interpretation
• 22 (20) in Inpatient Drug Order Review
• 15 were about side effects, dosage, cost, trade
  names, and interactions

80
Evaluation

• Log file analysis 32 months of data
• On-line survey 108 comments
• Popup questionnaires
• E-mail survey

• Popup questionnaires 195/3,642 (5.4)

81
Pop-up Survey Responses
Q1 Easy to use Q2 Got answer Q3 Helpful
82
Evaluation

• Log file analysis 32 months of data
• On-line survey 108 comments
• Popup questionnaires
• E-mail survey

• Popup questionnaires 195/3,642 (5.4)

• E-mail survey 73/1,228 (5.8)

83
E-mail Survey Responses
Q2 Infobuttons are easy to use Q3 Presence of
question on the list Q4 Speed of answer Q5
Helpful answer Q6 Positive effect on patient
care decisions 1 (strongly positive) to 5
(strongly negative)
84
Results Pop-Up and E-Mail Surveys

• Pop-Up E-Mail
• Easy to use 83 92
• Question on list gt50 of time 89
• Answered question 69
• Useful 77
• Helpful gt50 of time 90
• Positive effect on care 74
• Specific cases of improved care 14
• (40 of negative responses due to Lab Manual)
• (Students were the least positive)

85
Summary of Impact

• Over 29,000 additional accesses to resources
  (19)
• Positive impact at least half the time 74
• 14 respondents identified one or more specific
  situations in which patient care improved

86
Lessons Learned

• Use of the IM has been more gradual than desired
• User fails to select a question half the time
• Is the IM failing to anticipate the information
  need?
• Example - questions sugested during lab review
• Housestaff did not select a question 57.8 of
  time
• Online survey 4/11 dealt with reference ranges
• IM almost always provides a link to lab manual
• Is the user unable to find questions on the IM
  page?
• Solutions
• Education
• Add questions
• User interface

87
(No Transcript)
88
Conclusions

1. Context-specific access to health knowledge
   resources has been successfully accomplished
2. Impact on patient care decisions has been
   positive
3. Need to improve user interface for navigation
4. Increased resource use should result in
   clinicians making better informed patient care
   decisions

89
Acknowledgments

• This work is supported by NLM grant R01LM07593
• Evlauation plan
• Vimla Patel
• Sue Bakken
• Leanne Currie
• Beth Friedman
• Programming Jianhua Li
• Log files Rick Gallagher