Case study usability evaluation

1
Case study - usability evaluation

• Howell Istance

2
ELISE II - Web- based Image Retrieval System

• This allows users to enter search terms which
  relate to descriptions of images stored in image
  collections e.g. sunflower

3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
Methods proposed for usability evaluation of
ELISE II

• Structured expert reviewing
• Use of cut-down version of cognitive walkthrough
  technique
• Observation of users completing standard tasks
• Studied via a number of user trials
• Extended in-situ testing
• Use of ELISE II in normal work during 1 month
• Participant keeps diary of system usage

9
User trials pilot testing

• Used 2 graphics designers
• Insufficient scope in the tasks used to test the
  advanced search and the browse facilities
• Requires additional tasks which require these to
  be used

10
Participants in evaluation trials

• Limerick 5 librarians, 1 student
• ULB 10 librarians
• DMU 2 graphics designers (pilot), 2 lecturers, 2
  research students
• VA 1 researcher, 2 cataloguers, 2 curators
• Tilburg 1 librarian, 2 cataloguers, 1 manager

11
Procedures for user trials

• introduction to the trial and its purpose
• initial interview to establish background and
  nature of image use
• initial exploration phase
• formal explanation of the functions
• search for last image actually used
• Searches for 3 sample images
• post-trial interview

12
Macro Usability Issues

• How well does the system support the user in
  formulating a query?
• Are the retrieval results presented in such a way
  that enables the user to comprehend the relevance
  of the results to his search needs?
• Is the user supported in making advanced queries?
• Can the image retrieval system be integrated into
  the working environment of the user?

13
Search strategies

• Most began with restrictive search (several
  terms and boolean operators), removed terms
  successively of increase number of hits, only if
  no hits found then different generic term used
• Few began with generic term and added terms to
  reduce hits
• Very few used browse facility to check search
  term
• Many tried to minimise numbers of record
  retrieval in view of long wait

14
Support in formulating query

• Level of information about individual
  collections, type of images and search terms
• Truncation not supported
• Language of collections not clear
• Order of precedence in Boolean expression
  evaluation
• collections visible to user or one large virtual
  collection?

15
Assessing relevance of search results

• Order of image retrieval unclear
• Detail of retrieved images
• Local copies of image records
• Decision to buy image records

16
(No Transcript)
17
Support for Advanced search

• Understanding meaning of index fields
• Information about individual collections
• Complexity of search expressions

18
Integration with users normal working environment

• Speed of image retrieval and system reliability
• Saving retrieved images
• General usefulness

19
Micro usability issues

• Consistency
• Appropriate user support
• Minimal user input
• Reduce user memory load
• Feedback
• Flexibility
• Navigational support

20
Navigation

• Mechanisms inconsistent
• For example..
• No obvious means of returning to the simple
  search page from the search results page
• Clicking on buttons containing the numbers of
  hits to view the record list is not obvious
• Requires consistent mechanism to move query
  search view sequence,
• Consistency in object used, in labelling, in
  location

21
System feedback

• System messages are
• too small in size
• appear in a non-prominent part of the page
• language and content is often very obscure
• e.g ERROR HTTP return error Broker not
  found?HTTP/1.1 500 Internal Server error..
• empty return contact the administrator -
  meaningless to most users

22
Login Facility

• Location of command not prominent
• Feedback about outcome (success or failure) is
  poor
• Dialogue box persists on-screen with no visible
  change during login attempt
• User encouraged to quit rather than logging off
• Have separate login page

23
(No Transcript)