eHealth: Innovations and Issues

1
eHealth Innovations and Issues

• Lecture 9
• Computerized Clinical Decision Support
• Norm Archer

2
Agenda

• Definitions
• Medical procedure for patients
• History of medical DSS
• Consumer DSS
• Clinical DSS
• Treatments for diseases conditions
• Example use of DSS
• Excerpts from Andre Kushniruk presentation on
  usability testing of DSS

3
Definitions

• Decision support system
• A computer system designed to assist users in
  decision-making activities.
• Decision support systems in healthcare
• Management DSS - Computer technologies which
  allow providers to collect and analyze data in
  more sophisticated and complex ways. Activities
  supported include case mix, budgeting, cost
  accounting, clinical protocols and pathways,
  outcomes, and actuarial analysis.
• Clinical (CDSS) - form a significant part of the
  field of clinical knowledge management
  technologies through their capacity to support
  the clinical process and use of knowledge, from
  diagnosis and investigation through treatment and
  long-term care.
• Consumer DSS - help consumers become more aware
  and responsible for their own healthcare through
  a patient-centred approach, while cooperating
  with medical professionals who make informed
  treatment decisions.

4
Definitions

• Disease
• A disease is an abnormal condition of the body or
  mind that causes discomfort, dysfunction, or
  distress to the person afflicted or those in
  contact with the person. Sometimes the term is
  used broadly to include injuries, disabilities,
  syndromes, symptoms, deviant behaviours, and
  atypical variations of structure and function,
  while in other contexts these may be considered
  distinguishable categories.
• Differential diagnosis
• Determination of which one of two or more
  diseases or conditions with similar symptoms is
  the one from which the patient is suffering.
• Correct diagnosis leads to the choice of the most
  effective treatments that will alleviate the
  disease or condition

5
Medical Procedure for Patients

• 1. Ill patient has appointment with physician, or
  visit to emergency room or clinic
• 2. Physician takes patient history, symptoms,
  examines patient
• 3. Physician
• Makes preliminary determination of disease
• Writes prescription for medication to combat the
  illness, or
• Refers patient to specialist, or
• Orders medical tests
• If patient sent to specialist - orders additional
  tests and either reports back to physician or
  takes patient under care for treatment or surgery
• If further tests ordered or specialist reports
  back, physician makes determination of illness
  and writes prescription for treatment
• Best practice information very helpful for
  physician or specialist in treating disease after
  it has been identified

6
History of Medical DSS

• See http//www.openclinical.org/dss.html
• Early history
• 1970s and 80s, development of rule-based expert
  systems INTERNIST, MYCIN, PIP, ONCOCIN
• Some were commercialized as DSS
• E.g. DXPlain and QMR
• Demo of DXPlain at http//www.openclinical.org/dm_
  dxplain.html

7
Consumer DSS (Eysenbach 2000)

• Using self-analysis decision support tools,
  patients may be able to attain a healthy balance
  between self reliance and seeking professional
  help, by balancing of the need for face to face
  interaction with provision of virtual
  interaction.
• Focus group evaluations of HouseCall (for
  example) have shown that the program is easy to
  use and that consumers like using technology at
  home to investigate health issues and like
  participating in solving their medical problems.
• Obviously, such systems do not and cannot
  replace visits with physicians they can,
  however, make such encounters more productive,
  for both doctor and patient. They may also help
  to triage patients. For consumers, the aim of
  such support systems would not be to make
  definitive diagnoses or to propose treatment but
  to answer simple questions such as do I need to
  see a doctor? or to alert patients to potential
  drug interactions or other health risks.
• The main challenge in developing comprehensive
  systems for consumers is that little is known
  about how patients interact with computer based
  informatics tools and how they digest and act on
  information.

8
Online Consumer DSS

• General
• YourDiagnosis (U.S. 12.50 to 15.00)
  http//www.yourdiagnosis.com/start.htm
• The Analyst (U.S. 25.00 to 77.00)
  http//www.diagnose-me.com/?pagemain
• MyElectronicMD (free) http//myelectronicmd.com/st
  ep1.php
• EasyDiagnosis (Subscription cost)
  http//easydiagnosis.com/
• Cancer
• NexProfiler (free, operated by NexCura)
  http//www.cancerfacts.com/DefaultSecure.asp
• Cardiology
• NexProfiler (free, operated by NexCura)
  http//health.discovery.com/jump/nexcura/heart_pro
  filer.html
• Other diseases
• HealthCommunities (free, operated by NexCura)
  http//www.healthcommunities.com/

9
Clinical DSS for Physicians

• Medexpert/WWW is a medical knowledge base server
  links to many online DSS http//medexpert.imc.ak
  h-wien.ac.at/start.html
• Example MedTechUSA (annual 1050 U.S.
  subscription) medical reference tools for a
  variety of diseases and conditions (includes
  handheld device versions) http//www.medicalamazon
  .com/

10
Treatments for Diseases and Conditions

• List of diseases and treatments
• http//www.surgerydoor.co.uk/medical_conditions/
• Information on diseases and treatments
• http//www.mayoclinic.com/
• And many others

11
Example 1

• Symptoms
• Tingling in upper right arm
• At irregular intervals, when walking, computer
  work, driving, etc.
• Especially when looking upwards (e.g. changing a
  light bulb in the ceiling)
• Tingling stops when right hand is placed at back
  of neck in stretching motion
• Problem began August 2005 while swimming
• No or little loss of strength and use of right
  arm
• Patient chart and history
• Gender
• Age
• Pre-existing health problems episode of carpal
  tunnel syndrome about 1990 (cleared up over a one
  year period after ergonomic adjustment of
  computer work environment)
• Current medications
• Life style (exercise, diet)
• Psychological and physical (work, home)
• Medical test results from EMG (electromyography)
  compared right left arms.

12
Example 2

• Symptoms
• Episodes or sudden onsets for 10 to 30 minutes
  of
• Dizziness
• Sweating
• Blurred vision
• Excessive saliva
• Déjà vu
• Episodes have been occurring irregularly for
  several years
• Patient chart and history
• Gender
• Age
• Parental health histories
• Pre-existing health problems
• Mental condition
• Current medications
• Life style (exercise, diet)
• Psychological and physical (work, home)
• Recent visits to foreign countries
• Medical test results

13
Excerpts from From Laboratory Usability Testing
to Televaluation of Web-based Information
Systems
2003 Presentation by André W. Kushniruk, Ph.D.
Director, Information Technology Program,
Faculty of Arts York University, Toronto,
Ontario, Canada
14
Needs-satisfaction curve of information technology
Excess Functionality
Performance Required by Typical User
Unfilled Technology Need
Technology is good enough User Experience
Dominates
High Technology
Human-Computer Interaction
Transition Point Technology delivers basic need
15
Motivation HCI Issues in Health Informatics

• Problems with information systems in health care
• Lack of acceptance of systems
• Poor usability
• Failure to support work practices
• Introduction of errors
• Inadvertent changes in workflow
• Issues related to human-computer interaction
  (individual and group/social level) may be
  single-most important barrier to successful
  implementation of systems in health care

16
Evaluation in Health Informatics

• Summative Evaluation - need for assessment of
  whether systems meet the needs of users, are safe
  and effective
• Formative Evaluation need for assessment of
  systems throughout their development
• Traditional development approach classic
  waterfall development cycle
• Newer approach rapid prototyping involving
  continued user input and testing

17
From Laboratory to Real-worldAnalysis and
Evaluation (Kushniruk,2001)
A Continuum of Studies

• LABORATORY
• Fixed usability lab
• Experimental tasks
• - think aloud
• - cognitive task
• analysis

• NATURALISTIC
• Virtual
• usability lab
• Analysis of Web-
• based systems
• - Data mining

• Simulations
• E.g. simulated
• doctor-patient
• interviews

18
A Continuum of Approaches to Evaluation Along the
SDLC
1. Planning (needs analysis) -workflow
analysis -job analysis -analysis of decision
making -interviews
2. Analysis (requirements) -interviews
-questionnaires -focus groups -video analysis
-cognitive task analysis
3. Design
4. Implementation (programming)
5. Support (maintenance)

• usability testing
• -usability
• inspection
• -design walk-
• throughs

-outcome-based evaluations -randomized
clinical trials -summative evaluations
-usability testing -code inspections -software
unit testing
Figure 1. The systems development life cycle
(SDLC) in relation to
evaluation methodologies.
19

• Study 2 Study of use in diabetes clinic over
  six month period naturalistic approach
• Interviews (pre and post) 16 clinic staff
• Usability testing with subset of subjects
• Training recorded as well
• Logging of all system use
• Study of contents of paper and computer records

20
Study 2 Results

• More irrelevant information in paper records
• Overall less information recorded in computer
  based records
• For corresponding records, CPR version contained
  25 less information.
• Fewer diagnoses recorded in CPR for matched
  records - typically only single primary diagnosis
• Change in reasoning -- from hypothesis driven
  to screen driven

21
Changes in Reasoning

• Data-directed (paper records)
• Problem-directed (CPR)
• Problem-directed (paper records)
• Lasting change in reasoning patterns, even when
  CPR removed (effects of and effects with)

22
Diagnostic Reasoning Using Paper Record
Multiple Hypotheses
Patient Data
Diagnostic Reasoning Using CPR
Patient Data
Hypotheses
23
Residual Effects of CPR Use
of Record Contents
24
10 fold decrease in usability problems

• Use of same method as above for improving
  prototype CPR system
• 9 subjects video recorded using the system
• Number of problems and type of problems
  identified from coding scheme
• System refined based on the testing
• 9 new subjects run
• Found
• Dramatic decrease in number of errors
• From 19 per session on average in first testing
  to 1.9 in second testing!

25
Usability and the WWW (Kushniruk et al., 2001)

• Objective to adapt usability testing to the WWW
• how are people using health care sites?
• Do they get information they want from particular
  sites?
• what problems do they have?
• How are Web-based guidelines used?
• Remote tracking of Web users
• Remote video-based usability testing

26
Evaluation of Usability of Web-Based Health Care
Information Systems

• Varied users who interact from various locations
• Less able to conduct controlled evaluative
  studies
• Current state-of-the-art
• track user actions (e.g. clicks) - tells what
  they do, but not why
• on-line questionnaires/feedback forms - often not
  filled in, limited questions
• interviews - problem that users often do not know
  what they do

27
Questions in the Evaluation of e-Health
Information Systems

• What type of information do e-Health consumers
  want?
• Is the information provided useful, helpful?
• How to collect useful data from large number of
  subjects remotely?
• How to integrate data from multiple sources?
• How to analyze such data from varied data sources
  to discover usage patterns?

28
Objectives

• To collect psychologically rich and useful data
  on a large scale
• Methods for automatically collecting usability
  data at point of system use
• identify patterns of usage of interest to
  automatically collect data about
• Analysis tools and discovery tools
• Automatically identify patterns of usage from
  merge of data collected
• Integration of multi-method data collection and
  analysis
• To answer both specific and generic questions
  regarding use and usability of Web-based health
  systems

29
Remote evaluation of System Usage
1. Video Based Usability Testing - from
laboratory to remote
4. Tracking User Actions - System Usage
Database (log files)
Interact via WWW
Consumer Information System
User
3. E-mail (to evaluators)
2. Interviews - from phone to electronic
5. On-line Questionnaire Data
(triggered forms)
Kushniruk, Patel, Patel, Cimino, 2001
30
Example Evaluation of a Patient Clinical
Information System (PatCIS)

• Over the WWW patients can
• Review their own medical data (e.g. laboratory
  results
• Enter their data (e.g. blood glucose levels)
• Receive advice
• Receive educational information
• Subjects recruited from private practices in New
  York state
• Followed over one year
• Thousands of accesses

31
Screen of a patient clinical information system
(PatCIS) showing data review function
32
Evaluation Questions

• What features of such systems are most used by
  patients, Why?
• What features are least used and why?
• Are there usability issues that need to be
  resolved?
• How does use of such systems affect the
  doctor-patient interaction?
• Can patients comprehend information presented?
• Does use of these types of systems affect
  decision making and disease management?

33
Results

• Function Usage
• Most frequently accessed function was Review of
  Laboratory Data
• Accessed by patients at least once in the
  majority of the sessions
• Review of Reports was second most frequently
  accessed function
• Other functions (advice, education and data
  entry) were used sparingly

34
Analysis of User-System Interactions

• Function Usage (number percent of
  accesses)
• ADVICE DATA ENTRY DATA REVIEW
  EDUCATION TOTAL
• .3 4
  93 3 100
• Majority of accesses by patients for Data
  Review
• Laboratory details
• Reports admit/discharge, cardiology, radiology
• Discovery of patterns of usage related to
  both
• demographic and medical data
• Most used and useful for patients with specific
  illnesses chronic illness (e.g. diabetes)
• Patients liked the system since they felt greater
  ownership
• Physicians liked the system as it streamlined
  their limited face-to-face visits with patients
  (patients had often reviewed their data prior to
  the interview)

35
Excerpts from interviews with Patients
Communication is less in the way of getting
information now, and more in the way of
discussing treatment options and agreeing on a
course of action, so to me its more efficient
than the old way I look for trends in my
medical data and if I see something I can contact
the doctor to see whats going on, what we can
do, change meds or whatever
36
Excerpts from Interviews with Physicians
Right now most of the communication takes place
during the ten or fifteen minute visit and if I
throw a lot of information at the patient about
their condition or what I want them to do, its
very hard for them to absorb all that. It
(PatCIS) gives them a chance to go back and look
at things about their health record that they can
then ask better questions about in the limited
time that we have during the visit. Its another
channel of communication
37
Figure 5a. Resource page showing links to
clinical guidelines available from within a
computer-based patient record system.
38
Figure 5b. Form to assess clinicians reason for
accessing a guideline (which appears when the
user selects a guideline from the resource page).
39
(No Transcript)
40
Implications

• Analysis of results led to guideline designers to
  modify
• the format and amount of information contained
  in guidelines
• Now includes text OR easy to read graphical
  representations (maps)
• Studying impact of changes

41
Summary

• Need for range of approaches for assessing HCI /
  usability
• Usability is critical to success of health care
  information systems
• Usability will be recognized as a major success
  and marketing factor
• Consumer expectations for usability will increase

42
Future Directions

• Extension of methods for qualitative coding of
  data
• Development of newer portable and virtual
  approaches
• Usability engineering methods applied throughout
  system development
• User needs analysis (before system development)
• During requirements gathering, design,
  implementation
• In general, the earlier the better

43
References

• Eysenbach, G. (2000). Recent advances Consumer
  health informatics. British Medical Journal, 320,
  1713-1716.
• Kushniruk, A. W. (2003). Human-computer
  interaction in health informatics From
  laboratory usability testing to "televaluation"
  of Web-based information systems (pp. 57 -
  Powerpoint Presentation). Toronto, ON York
  University.
• Schwitzer, G. (2002). A review of features in
  Internet consumer health decision-support tools.
  Journal of Medical Internet Research, 4(2), e11.

44
eHealth Innovations and IssuesEnd Lecture
9Computerized Clinical Decision Support Norm
Archer, Ph.D.archer_at_mcmaster.cExt. 23944