Make it Happen: Library Website Usability Testing

1
Make it Happen Library Website Usability Testing

• Nina McHale
• Assistant Professor, Web Librarian
• Auraria Library
• November 10th, 2006

2
Our Agenda

• Why usability?
• Three phases planning, testing, results
• Testing example Auraria Library home page Fall
  2006 usability testing
• Create your own test plan!

3
Why Usability?

• Continuous assessment and improvement
• The Google problem
• The user isnt broken.
• -Karen Schneider
• Learning to think like your user the reverse
  teachable moment

4
Reverse teachable moment, example 1
5
Reverse teachable moment, example 2

• Why did you design LexisNexis and InfoTrac to
  look different?

6
Testing Phases

• Planning
• Goal
• Method
• Staff
• Subjects
• Testing
• Materials
• Procedure
• Practice

• Results
• Collate data
• Identify problems
• Prioritize problems
• Recommend/implement fixes
• Write report

7
Phase 1 PlanningGoal What to test, and why?

• A group of related web pages, i.e., subject
  guides
• Your OPAC/ILS
• Your interlibrary loan system
• Your OpenURL resolver

• to inform a redesign of those pages
• to make a vendor/product decision
• to improve out of the box interface screens
• to clarify the process of finding the full text
  of an article

8
Method How to test it?

• Card sorting/affinity mapping
• Focus groups
• Surveys and Interviews
• Task-based testing
• Prototyping
• For a more complete list of methods, see James
  Homs Usability Methods Toolbox

9
Staff Whos going to be involved?

• Identify staff willing to participate in the
  various phases of testing (planning, testing
  process, evaluating results)
• Include staff from multiple areas of the library
• Consider recruiting non-library staff to test
• Reduced anxiety for test subjects
• More critical/honest feedback from subjects
• May be required in some settings
• Suggestions friends groups, volunteers, students

10
Subjects Whom to test it on?

• Strive for a representative sample of your
  patrons
• How many subjects is enough?
• Provide an incentive valuable to them
• Gift certificate (book store, Starbucks)
• USB drive
• Food
• Spontaneity versus scheduling
• Advertising
• Notice on website
• Signage in library

11
Caveat for Academic Librarians the IRB

• Federal law requires that any institution
  receiving funding from the Department of Health
  and Human Services formally review any study
  using human subjects
• Institutional Review Board (IRB)
• Plan ahead
• Online course (2-4 hours)
• Submit written proposal and paperwork (sample)
• Allow time for the IRB to review your project
• Allow time for revision, if required by the IRB
• Exempt status versus full review

12
Phase 2 Testing

• Materials
• Procedure
• Practice

13
Phase 2 Testing

• Materials
• Space a separate room/classroom
• Office supplies (pens, paper, flip chart,
  scissors, Post-Its, etc.)
• Computer or video camera setup (w/software such
  as Camtasia if desired)
• Lo-tech is effective, cheap, and easy
• Hi-tech is kewl, but

14
Phase 2 Testing, ctd.

• Procedure and practice
• Determine procedure
• Write scripts and create notetaker forms
• Streamline working with subjects
• Ensures consistency among testing sessions
• Do a dry run that approximates the real test
  situation as closely as possible
• Revise your test procedure and documents as
  necessary

15
Phase 2 Testing, ctd.

• Some thoughts about recording test sessions
• Subjects can be videotaped, or software such as
  Camtasia can be used to record sessions
• Useful for revisiting test sessions or as
  evidence in the results phase
• Note Academic IRBs generally will NOT extend
  exempt status to any project that involves
  videotaping your subjects allow extra time for
  full IRB review if you feel strongly about
  recording

16
Phase 3 Results (now what?)

• Collate data collected from all subjects into one
  document
• Identify problems common among subjects
• Prioritize problems two ranking systems
• Recommend fixes
• Implement fixes
• Written report

17
Phase 3 Results, ctd.

• Collate data from all subjects into one document
• Type of document will depend upon the type of
  data collected
• Survey/interview responses
• Mock-ups of a proposed home page
• Recorded sessions of patrons performing tasks
• Usually a spreadsheet with accompanying chart
• Georgetown example
• task-based testing
• 4 users
• Identify problem areas common among subjects (the
  single collated document simplifies this process)

18
Phase 3 Results, ctd.

• Prioritize problems with a pre-established
  ranking system
• Two examples (Source Barnum, p. 270)
• Rubin
• Unusable
• Severe
• Moderate
• Irritant
• Dumas and Redish
• Level 1 prevents completion of a task
• Level 2 creates significant delay and
  frustration
• Level 3 has a minor effect on usability
• Level 4 subtle problem points to a future
  enhancement

19
Phase 3 Results, ctd.

• Recommending fixes
• Involve staff from multiple areas of the library
  in the discussion of how to resolve problems
• If you have a pre-established web advisory body,
  you may wish to begin with them
• Distribute data collation document prior to
  discussion sessions, if possible
• Implementing fixes
• Delegate tasks as appropriate
• For complex problems, plan steps accordingly

20
Phase 3 Results, ctd.

• Write a report
• Communicates your findings
• Documents an extensive process
• Record everything from your initial goal through
  methodology and results
• Report can be written before all changes are made
• Barnum Document positive findings
• Good news!
• To ensure that things that arent broken dont
  get fixed in the future

21
Auraria Library UsabilityPhase 1 Planning, Goal

• A more user-friendly library home page
• Current issues
• Use of jargon how to make links meaningful?
• Too many links (30)
• Outdated look and feel

22
Phase 1 Planning, Method

• Based closely on a study conducted at the
  University of Central Florida
• Terms and phrases on the librarys home page are
  chopped up into cards for users to group and
  then arrange on a flip chart

23
Phase 1 Planning, Staff

• Web Librarian
• Two Metropolitan State College of Denver students
  enrolled in the course COM 3625 Usability Testing
• Standing library Web Advisory Committee

24
Phase 1 Planning, Subjects

• Undergraduate and graduate students from all
  three Auraria institutions
• Goal is 3-4 students from each institution (per
  Nielsens recommendation)
• Incentive 128 MB USB hard drive, customized with
  the librarys logo
• Advertising library signage and note on library
  home page

25
Phase 2 Testing, materials

• Flip chart
• Copies of library terms and phrases on small
  slips of paper
• Scotch tape
• Pens/markers
• Blue dot stickers
• Room 130

26
Phase 2 Testing, Procedure/Practice

• Web Librarian and students did a dry run
• One student completed the process while the other
  facilitated
• Procedure improvements
• Scotch tape versus glue sticks
• Allowing users blank cards to create their own
  terms/wording

27
Phase 3 Results

• Flip chart home page mock ups
• Data will be collated a la the University of
  Central Florida method
• Web Advisory Committee will provide
  recommendations to Web Librarian about the
  revised home page

28
Create your own test plan!

• Goal What do you want to improve upon?
• Be specific! NOT the library web site
• Method Which makes sense in light of your goal?
• Survey/interview card sort task-based focus
  group?
• Staff
• Given the goal and method, who should be
  involved?
• Ensure that multiple departments are represented
• Subjects
• Whom and how many will you use to test?
• How will you recruit a representative sample?
• Academic librarians do you need/have IRB
  approval?

29
Creating your plan testing

• Where will you conduct the test?
• Will you recruit non-library staff to help?
• What kinds of information will you include in
  your script?
• Will you need office supplies?
• Describe your test procedure.

30
Creating your plan results

• What type(s) of data will you have after the
  testing sessions?
• How will you analyze/present the data, and to
  whom?
• Once problems are identified, how will you
  proceed?
• Who will implement the necessary fixes to
  finalize the project?

31
Web Resources

• Jakob Nielsen, Useit.com
• Alertbox (regular column email subscription
  available)
• James Hom, The Usability Methods ToolBox

32
Books

• Carol Barnum, Usability Testing and Research. New
  York Longman, 2002.
• Elaina Norlin and CM! Winters, Usability Testing
  for Library Web Sites A Hands-on Guide. Chicago
  ALA, 2002.

33
Articles

• All articles are from Computers in Libraries,
  October 2005, Volume 25, Issue 9
• Frank Cervone, What Weve Learned From Doing
  Usability Testing on OpenURL Resolvers and
  Federated Search Engines, 10-24.
• Janet Ballas, Does Your Library Pass the Web
  Site Usability Test? 36-39.
• Heather Cunningham, Designing a Web Site for One
  Imaginary Persona That Meets the Needs of Many,
  15-19.

34
Questions? Comments?

• Nina McHale
• nina.mchale_at_cudenver.edu
• Presentation slides, handouts, and supporting
  materials are available online
• http//carbon.cudenver.edu/nmchale/usability/