User Centered Design of Medical Systems

1
User Centered Design of Medical Systems

• James A. Larson, PhD
• Intel Corporation

2
Headline

• Medical Usability How to Kill Patients
  through Bad Design
• 1 Nielsen, Jacob Alertbox, April 11,
  2005, http//www.useit.com/alertbox/20050411.html
  
• 2 Koppel, R. Metlay, J.P., Cohen, A..
  Abaluck, B. Localio, A.R. Kimmel, S.E. and
  Strom, B.L. Role of Computerized Physician
  Order Entry Systems in Facilitating Medication
  Errors, Journal of the American Medical
  Association. Vol. 293, No. 10 (2005)

3
User errors

• Poor readability
• Font difficult to read
• Misleading default values
• Dosage vs medication units
• Memory overload
• Multiple screens to see all of patients
  medications
• Data description errors
• tomorrow

4
CSE 566/666 Course objective

• Learn how to be effective at gathering relevant
  user-based information, integrating it into the
  development process, and creating innovative,
  usable, and error-free medical systems.

5
Textbooks

• US Food and Drug Administration. Do It by
  Design An Introduction to Human Factors in
  Medical Devices. http//www.fda.gov/cdrh/humfac/d
  oitpdf.pdf
• Michael E. Wiklund, Stephen B Wilcox. Designing
  Usability into Medical Products. 2004. CRC Press,
  ISBN 0849328438

6
Other information

• Spring quarter
• Tuesdays, tentatively 1130-230, CHH 12173
• Syllabus
• http//www.larson-tech.com/class.htm
• Instructor
• James A. Larson
• jim_at_larson-tech.com

7
Involve users in every phase of the design,
implementation, deployment

• Health care professionals
• Patients
• Administrative and clerical personal

• Hardware
• Software
• System

8
Medical System Lifecycle
9
Medical System Lifecycle
1. Identify Problem
Shadowing Modeling
2. Establish Requirements
5. Deploy
Users
Interviews Questionnaires
Monitoring
Focus Groups Wizard of Oz Experiments
Controlled Testing
4. Quality Assurance and User Acceptance
3. Evaluate And Refine Prototypes
10
Organization charts
11
Floor plans
12
Shadow Perspective Users

• Sometimes called Ethnography study
• Goal Identify new applications that will be
  useful to users
• Record everything a perspective user does for
  several hours
• Alternative video tape perspective user
• Analyze user activities to determine how
  automation can help user
• Make it easier for user to perform an existing
  task
• Enable user to perform a new task not previously
  possible

13
Activity log

• 330 Patient arrived at clinic
• 331 Checked in with receptionists showed
  insurance card
• 332 reads old National Geographic magazine
• 348 admitted to examining room
• 349 blood pressure, pulse, temperature taken
• 351 asked what prescriptions taken

14
Artifacts

• Copies of formsboth paper and electronics
• Pictures of work aids

15
Analyze data

• What can be done to automate existing tasks?
• Frequently performed tasks
• Critical tasks
• What new tasks can automation enable?

16
Medical System Lifecycle
1. Identify Problem
Shadowing Modeling
2. Establish Requirements
5. Deploy
Users
Interviews Questionnaires
Monitoring
Focus Groups Wizard of Oz Experiments
Controlled Testing
4. Quality Assurance and User Acceptance
3. Evaluate And Refine Prototypes
17
Story Board

• Goal describe usage modelhow a new
  application will be used

18
End Users Conceptual Model

• NounsThe names of application objects and
  their relationships
• VerbsHow the user may manipulate application
  objects and relationships

Patient contact data
Update, save
Medical history
Create, update, save
19
GUI design

• User participates in design

20
Medical System Lifecycle
1. Identify Problem
Shadowing Modeling
2. Establish Requirements
5. Deploy
Users
Interviews Questionnaires
Monitoring
Focus Groups Wizard of Oz Experiments
Controlled Testing
4. Quality Assurance and User Acceptance
3. Evaluate And Refine Prototypes
21
Wizard of Oz Testing

• Term comes from the Wizard of Oz story in which
  an ordinary man hides behind a curtain pretending
  to a powerful Wizard

22
Wizard of Oz Testing
On a scale of 1 to 10 with ten being the best,
how do you feel today?
7
User
Wizard

• Developer and perspective user talk with each
  other using telephones
• Developer pretends to be the computer (The
  Wizard)
• User believes that he/she is interaction with a
  real speech application
• Debug dialog flow before it is implemented

23
Focus Group

• Several perspective users review several
  prototypes
• Express opinions about each prototype

24
Medical System Lifecycle
1. Identify Problem
Shadowing Modeling
2. Establish Requirements
5. Deploy
Users
Interviews Questionnaires
Monitoring
Focus Groups Wizard of Oz Experiments
Controlled Testing
4. Quality Assurance and User Acceptance
3. Evaluate And Refine Prototypes
25
Two types of usability tests
Is the application useful?
Is the application enjoyable?
Performance
Preference
Measure users likes and dislikes
Measure what the user actually accomplished
Validate that users enjoyed the application and
will uses it again
Validate that users achieved success
26
Performing usability tests
Is the application useful?
Is the application enjoyable?
Performance
Preference
User answers survey questions after using the
application
Capture data via logging
Scoring program analyzes log file
27
Data Collection
Log Files
Users
Report Generation
Survey Interview
Performance Reports
Perference Reports
28
Performance Metrics
29
Preference Metrics
30
Preference Metrics (Open-ended Questions)

• What did you like the best about the
  voice-enabled application? Do not change these
  features
• What did you like the least about the
  voice-enabled application? Consider changing
  these features
• What new features would you like to have added?
  Consider adding these features in this or a later
  release.
• What features do you think you will never use?
  Consider deleting these features.
• Do you have any other comments and suggestions?
  Pay attention to these responses. Callers
  frequently suggest very useful ideas.

31
Score Summary

• How easy was it for you to understand how to use
  this system?
• Very easy 60 (303 Users)  
• Easy   28 (142
  Users)
• Neither easy nor difficult  8 (41
  Users)
• Difficult   3 (16 Users)

32
Medical System Lifecycle
1. Identify Problem
Shadowing Modeling
2. Establish Requirements
5. Deploy
Users
Interviews Questionnaires
Monitoring
Focus Groups Wizard of Oz Experiments
Controlled Testing
4. Quality Assurance and User Acceptance
3. Evaluate And Refine Prototypes
33
Monitoring

• Requests for help log
• Discussion forums
• User surveys
• Site visits

Quickly react to trouble spots as soon as they
occur.
34
Medical System Lifecycle
35
Bottom Line

• Involve the right users at each stage of design,
  implementation, and deployment
• Perform a variety of tests with a variety of
  subjects to identify potential problems
• Leave money in the budget for repairing
  flaws/recalls

36
Questions

• ?