Usability testing for library catalogs

1
Usability testing for library catalogs

• October 25, 2001
• Nicole Hennig, Web Manager
• libraries.mit.edu
• libraries.mit.edu/barton

2
Thank you

• Tracy Gabridge
• Librarian for Civil Environmental Engineering
• led the HTML customization team

3
Details available ...

• http//macfadden.mit.edu9500/
• webgroup/usability2001/barton/test1/
• overview.html

4
Outline

• 1. background
• 2. the tests
• 3. problems solutions
• 4. future directions

5
1. Background
6
6 month process

• January - June 2001
• old system GEAC Advance
• new system ExLibris ALEPH

7
Web OPAC project teams

• web OPAC team
• - public service librarians
• - circulation staff
• - processing staff
• - cataloger
• - web manager

8
Web OPAC project teams

• HTML customization team
• same as previous, plus
• - systems office staff
• - programmer

9
Bibliography

• on handout
• includes background on display and interface
  design of library catalogs

10
Background research

• a lot of research on OPAC design available
• but not based on observing users
• or usability testing
• library system vendors are not following basic
  good design principles

11
Who makes design decisions?

• we have more control now that we can customize
  HTML screens
• the vendors need to practice good design in
  building the system

12
A work in progress

• libraries.mit.edu/barton
• more rounds of testing and improvements are
  coming later in the spring

13
Usable design goals

• every page is self-explanatory
• self-teaching interfaces

14
Will it apply?

• some things are specific to ExLibris systems
• many things are general - could apply to any
  OPAC

15
General principles

• success summary
• http//macfadden.mit.edu9500/
• webgroup/usability2001/barton/
• test2/success.html

16
2. The tests
17
The test

•  we had already done extensive usability testing
  while redesigning our web site

18
Latest thinking has changed

•  1999
• Large test, 30 users, timed people
• - quantitative
• 2001
• - More frequent, smaller tests, 5-6
• people at a time
• - qualitative

19
The test

•  1/2 hour long
• 10 questions
• think out loud

20
The test

•  observer takes detailed notes
• train observers to not answer how it was
  supposed to work until end of the test
• each observer tests 2 people
• (2 week time frame)

21
Designing questions

• easy, basic tasks that a first-time user should
  be able to accomplish
• real-world tasks
• (give them a real article citation)

22
Designing the questions

• no need to obsess over perfect, scientific
  questions
• you will learn plenty from watching people use
  the catalog

23
The questions

• 1 - 5 known items
• 6 - 10 general research
• complete list
• http//macfadden.mit.edu9500/webgroup/
• usability2001/barton/test1/questions.html

24
The questions

• test the questions
• get the bugs out
• print out the questions
• in large type

25
Who we learned from

• Washington State University
• Janet Chisman, et al.
• Usability Testing A Case Study
• College Research Libraries
• Nov. 1999

26
What we learned

• multi-part questions
• - if user cant complete first part, observer
  does it so they can try second part

27
What we looked for

• features that were confusing or unclear
• aspects of the system that worked well

28
The tests

• test 1 test 2
• Who 7 students 3 students
• 3 library staff 4 library staff
• 4 disabled
• Catalogs our old web catalogBarton (6) 1st draft
  of
• McGill MUSE (2) new Barton
• Boston College QUEST(2) screens
• Dates Jan. 22 - Feb. 1, 2001 May 21 - June 1
• Successes 4 of 10 tasks 7 of 10 tasks

29
3. Problems solutions
30
Problem 1

• people usually picked the default choices or
  the first choices without thinking much about it
• (not always the best strategy for their search)

31
Example
people used first box, ignored second
32
Solution
Default choice is keyword. This casts a broad net
for those who forget to make a choice.
33
Problem 2
 Difference between browse keyword search not
clear
34
Example
?
?
35
Solution
No need to know difference between keyword and
browse search. Combined in one menu.
36
Problem 3

• it wasnt clear how to input a search string
• (people used initial articles, authors first
  name first, thought they had to type the entire
  title)

37
Example

• carefully typed complete title, with article
• The Journal of the American Chemical Society

38
Examples far away
39
Solution

• include examples and instructions of how to
  input data near the search box and in the search
  menu

40
Examples for each type
Example changes when menu changes.
41
Examples for each type
Example changes when menu changes.
42
Grouping

Group different title searches, author searches,
and subject searches together.
43
Problem 4

• very busy screens with many buttons were
  overwhelming for people

44
Example
45
Solution
 Present choices only where needed Group
navigation links in ways that make sense
46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
Problem 5

• it was difficult to find clickable URLs for
  electronic titles

50
No URL on brief results
51
Better URLs showing
52
Problem 6
not enough information on brief results screen
to choose the most relevant titles (especially
for subject searching) see David Thomas
article in bibliography
53
keyword women scientists
54
includes subject headings
55
Problem 7

• pop-up windows caused confusion

56
Solution

• minimize number of pop-up windows
• use only in cases where its handy
• to have the previous screen in the
• background

57
Problem 8
 title you input isnt at the top
58
Solution
 add marker (wish list)
59
Problem 9

• holdings info for serials was very confusing to
  everyone

60
Example
Do we have Dec 13, 2000 issue?
61
Example
62
Example
people dont understand open date range v.1
(1879)-
63
Not easy to fix
NISO standard way the data is input
limitations of the system
64
Example
65
A better holdings display
66
A better holdings display
67
Problem 10

• back buttons or back links didnt behave as
  expected (a problem with frames)

68
Coming soon

• no frames version due from ExLibris soon
• telnet version next year
• minimal javascript

69
What worked well?

• - hyperlinked author names
• - hyperlinked subject headings
• people found and used these very successfully

70
(No Transcript)
71
Self-teaching interfaces
 For difficult searches where you need to
combine fields in a specific way - design
screen so user doesnt need to know - it just
does the right thing
72
(No Transcript)
73
Users with disabilities

• 2 blind users one used Jaws, one used
  Window Eyes
• 1 user with dyslexia
• 1 user without use of hands, used
• pencil in fist to type, and large trackball

74
Users with disabilities

• these users had same problems and successes as
  everyone else
• (but the problems were magnified)
• everything took longer

75
Users with disabilities

• solutions that help everyone help disabled
• solutions that help disabled help everyone

76
Categories of problems
problems we can fix by changing the HTML
changing the tables in the database adding
custom programming changing our indexing
decisions changing cataloging
practice problems that only the vendor can fix
77
Other problems
 many other problems not mentioned here today
are described on our web site
78
4. Future directions
79
Latest usability research
 most large web sites have hundreds of
usability problems continuous rounds of
testing are necessary to find and fix all
problems better to begin with user-centered
design
80
Latest usability research
User Interface 6 East, Cambridge, MA Oct.
2001 Proceedings available http//www.uie.com
81
Still to test
Not so basic features, like  email/save/print
your bookshelf advanced searching  complex
limiting etc.
82
Cycles of testing
frequent small tests test your solutions
informal tests with handful of people
83
Future directions
share information compile guidelines  influ
ence vendors
84
Future directions
 are other libraries testing web OPACs?
 contact me to share test results Nicole
Hennig hennig_at_mit.edu