Evaluating Medical DeviceUse Safety

1
Evaluating Medical Device-Use Safety

• Todd R. Johnson, PhDAssociate Professor
• The University of Texas School of Health
  Information Sciences Houston andCenter of
  Excellence in Patient Safety Research and
  Practice

2
Device-Use Error

• An error resulting from the way a device is used,
  rather than a device malfunction.

3
Example Infusion Pump

• Order Deliver 130.1 ml/hr
• Nurse programs pump by pressing
• 1 3 0 . 1
• Device ignores decimal point keypress for numbers
  greater than 99.9
• 1301 appears on display

4
Effect of Device-Use Errors

• 82 of all preventable errors involving
  anesthesia devices were due to device-use errors
  (Cooper et al., 1978)
• Patients may be 3 to 10 times more at risk due to
  device-use error than to device failure (Grant,
  1998)

5
Sources of Device-Use Errors
Devices Design
Task
Use Environment
User
6
How Can We Select Devices That Minimize the
Chances of Device-Use Error?
7
Results of Purchasing Study at 3 Sites

• Very few alternative devices considered
• Device users were not direct participants in the
  purchasing decisions
• Evaluation process was often used to justify a
  decision
• Device evaluation was often limited to technical
  safety
• Usability testing for assessing device safety was
  not conducted at any of the sites.

8
Major Recommendations

• Involve all device stakeholders in the selection
  process
• Directly involve active device users
• Analyze strengths and weaknesses of existing
  device
• Set explicit safety goals
• Consider a wide range of alternative devices
• Explicitly evaluate device-use safety
• Ask for manufacturers human factors engineering
  plan and results
• Consider the cost of device-use errors

9
Challenge

• Challenge
• Device-use safety is best evaluated by usability
  experts, butusability experts are rarely
  available to purchasing committees.
• Solution
• Develop methods for use by healthcare
  professionals

10
Methods for Evaluating Device-Use Safety
11
What are good dimensions for evaluating
device-use safety?

• Must be
• Informative to healthcare professionals
• Allow clinically meaningful comparisons of devices

12
Classic Usability Dimensions are Too Abstract

• Speak the users language
• Visibility of system status
• User control and freedom
• Be consistent and follow standards
• Design to prevent errors
• Recognition rather than recall
• Design for flexibility and efficiency of use
• Aesthetic and minimalist design
• Help users recognize, diagnosis, and recover from
  errors
• Provide relevant, task oriented, easy to access
  and search, help and documentation

13
Classic Usability Results May Not be Very Helpful
for Clinicians
14
Solution The UPCARE model

• Developed by Kaye, North, and Peterson
• Designed to help describe and analyze device use
  problems
• Uses device-based dimensions familiar to clinical
  users

15
UPCARE

• Unmet user needs
• Perception
• Cognition
• Actions
• Results
• Evaluation

16
Modification of UPCARE for Device-Use Evaluation

• UPC Simplified from 38 to 23 usability items
• ARE not relevant to device-use evaluation

17
Usability Dimensions

• System set-up and configuration
• 1. System set-up is easy and efficient
• 2. Start up procedure is intuitive and efficient
• User-device interaction
• Perception
• 3. Device displays, labels, or markings are
  clearly visible
• 4. Text, number, or status indication is easy to
  locate in the visual display

18

• Information interpretation
• 1. Labeling on device or overall device
  configuration clearly shows identity, operation,
  or use
• 2. Input, output, level, or calibration values
  conform to established standards
• 3. Device data are sufficient for user to
  diagnose patient situation or adjust treatment

19

• Feedback
• 1. Feedback to user actions is timely, adequate,
  and easy to understand
• 2. Device status, including normal operation,
  critical change and device failure, is adequately
  indicated
• 3. Battery (or charge) end-of-life condition is
  clearly indicated
• Error-correction
• 4. Device provides safe default/fail-safe mode
• 5. In case of error, device allows user to make
  corrections easily and quickly
• 6. In case of error, device allows user to
  immediately stop device action or process

20
Device-Use Survey

• Uses the same dimensions
• Used after user tests or clinical trials
• Neutral items with semantic differentials

21
Combining Results

• Nearly every recommended evaluation method uses
  the same dimensions
• Exception Error reports
• Spreadsheet combines cost, technical specs, and
  device-use safety for a variety of devices

22
(No Transcript)
23
(No Transcript)
24
Current Status

• Guidelines are nearly complete
• Testing with mock purchasing groups will begin
  soon
• Future work
• Test use of each method with healthcare
  professionals

25
Acknowledgments

• Collaborators
• University of Texas Health Science Center at
  Houston
• Jiajie Zhang, PhD, James P. Turley, RN, PhD
• Xiaozhou Tang, MD, Juliana Brixey, MSN, MPH, RN
• Danielle Paige, Zhihua Tang, PhD, Kelly
  deChermont
• Columbia University
• Vimla L. Patel, PhD, DSc
• Alla Kesselman, PhD
• Mark Graham, PhD
• This project was supported by grant number PO1
  HS11544 from the Agency for Health Care Research
  and Quality