Usability Testing of a Medical Library Portal

1
Usability Testing of a Medical Library Portal

• Anamarija Roic-Hristovski, M.D., M.Sc.Central
  Medical Library
• Iztok Humar, B.Sc., M.Sc.Faculty of Electrical
  Engineering

2
Presentation Overview

• Introduction
• CMK portal
• Case study goals
• Usability studies
• Methods
• Results
• Conclusions

3
Introduction

• A Web portal or gateway aggregates library
  resources and services through a single access
  point.
• Medical library Web interfaces have become
  increasingly complex.
• There is a fundamental need for the systematic
  evaluation of the usability of these Web based
  systems.

4
CMK Portal

• CMK decided to extend its Web site with
  information portal
• CMK portal based on MyLibrary system at NC State
  University
• Customisation to the needs of CMK users was
  needed
• MyLibrary functionality is enabled

5
MyLibrary CMK functionality

• Consideration of the CMK organisation, role and
  function
• Slovene-English communication with MyLibrary
• Customisation of text and content
• Personalization of users profiles
• Slight modification of the portal design to suit
  the rest of the CMK Web site

6
(No Transcript)
7
Case Study Goals

• Formal usability case study for the evaluation of
  the pilot portal
• Review of applications of usability testing to
  library Web sites
• Selection of the most appropriate usability
  methods
• Application of selected methods to evaluate CMK
  portal

8
Usability usability testing

• ISO defines usability as the extent to which a
  product can be used by specified users to achieve
  specific goals with effectiveness, efficiency,
  and satisfaction in a specified context of use
• In the world of WWW this means that the interface
  of a portal must be easy to learn, remember and
  use and must lead to as few errors as possible
• In usability testing, users are observed while
  using the portal, to perform given task or
  achieve a set of predefined goals

9
Most common methods for evaluating the web site
usability

• Objective performance
• Subjective user preferences
• Experimental evaluation
• Direct observation

10
Think aloud method

• The most systematic and valid direct observation
  method
• The users are asked to perform specific task
• A relatively small sample size (5-6 users) is
  needed for identifying the majority of the
  usability problems
• During their work, they are required to verbalize
  their problems
• User verbalizations and keystrokes are recorded
  on audio tapes, observations are written to
  notebooks

11
Think aloud method properties

• This method provides answers
• How easy a user can learn and use the system
• How flexible the system is
• Why users have made mistakes
• Critiques of this method goes to
• Presence of the experimenter
• disturbance for the user
• The need for verbalization
• depends on the users character

12
Methods- I

• Think aloud method was selected as the most
  appropriate
• Designing the test
• 12 test exercises
• Instructions to observer
• Tape-recording of sessions
• Observations written to notebooks

13
Methods II

• Test participants
• Before instruction
• 3 students
• 3 faculty
• After instruction
• 3 students
• 3 faculty

14
Usability Testing of MyLibrary_at_CMK

• Login to MyLibrary_at_CMK portal!
• On your personal page, select the library links
  that you intend to use at your work. Return to
  MyLibrary!
• On your personal page, select the university
  links that you intend to use at your work. Return
  to MyLibrary!
• On your personal page, select the quick searchers
  that you intend to use at your work. Return to
  MyLibrary!
• On your personal page, select the databases that
  you intend to use at your work. Return to
  MyLibrary!
• On your personal page, select the e-texts that
  you intend to use at your work. Return to
  MyLibrary!
• Identify all the resources about AIDS available
  in MyLibrary! Return to MyLibrary!
• Find all books from the field of dermatology,
  available at the Central Medical Library! Return
  to MyLibrary!
• Find journal Neurology! Return to MyLibrary!
• Find a journal article about headache! Return to
  MyLibrary!
• Find new books from the field of public health of
  the last three months! Return to MyLibrary!
• Switch the language for communication with
  MyLibrary!

15
Results - ISuccess in Performing Tasks
16
Results - IITime Needed to Performing Tasks
17
Results IIISummary of Usability Problems

• Note on length of password
• Inconsistent performance of buttons
• Complaints about registration
• Unclear terminology
• The role and content of MyLibrary
• The role and content of OPAC
• The role and content of databases
• No idea how to find an article
• No distinction between the title of an article
  and the title of an journal

18
Conclusions

• Students performed tasks much better than faculty
• More familiar with internet and concept of
  digital library
• Role of education
• No influence on success rate
• Less time needed for finishing the tasks
• Identification of main usability problems

19
Further Work

• Redesign of library portal considering results of
  usability study
• New usability study to verify appropriateness of
  changes
• Updating the content of portal
• Public use of portal