Usability Engineering

1
Usability Engineering

• Mary Czerwinski
• Microsoft Research

2
Agenda

• Why usability test?
• What is usability testing?
• An example
• Toward user-centered design
• Other resources

3
Why usability test?

• Cost savings (well documented, see Neilsen, 1993)
• Not always directly visible (PSS calls, resales,
  product returns, distributed productivity
  benefits to user, sw development costs)
• Competitive market--user expectations
• Political demands
• Help might not

4
What is usability?

• Useful - Does it do what is needed? (teach, find,
  manage , calibrate/extend the hw, escape)
• Usability
• Is it easy to learn?
• Is efficient to use?
• Do no or only few errors occur?
• Is it easy to remember?
• Desirability - Is it fun to use? Do you want to
  keep using it again and again?

5
Usability in your product cycle the
earlier the better!
Planning

• Establish usability goals
• Field research--tasks
• Cognitive modeling
• Competitive testing
• Participatory design
• UI design guidelines
• Applied research
• PSS communication
• Roundtables
• Low fidelity prototyping
• Focus groups

• Competitive testing
• Field testing
• PSS communication

6
Ideal software

• Software features should be discoverable
• Users can easily identify the controls or
  features that match the tasks they want to do
• Users can quickly identify what the software lets
  them do EVEN if they didnt previously know this
  action was possible--principle of gradual
  disclosure

7
Ideal software

• Software features should be learnable
• If the feature is new or unfamiliar, the software
  (and on-line help) should contain information
  that guides the user through learning what the
  feature is for and how it works
• Software features should be memorable
• After time spent on other activities, the user
  should be able to return to a feature and use it
  correctly

8
Research and ideal software

• Research on human cognitive abilities
• attention visual perception
• memory learning
• Research on human-computer interaction
• applied research task-oriented studies
• heuristics for software design
• Literature is a great resource for studies!

9
Basic Cognitive Principles
Memory

• Associations are built by repetition
• Scaffold model - more likely to remember items
  that have many associatons
• Recognition is easier than recall
• Working memory has small capacity (time size)
• Long-term memory has large capacity (time size)

10
Attention

• Attention is a resource - gets divided between
  the different senses, different tasks
• Automatic well-learned processes dont require
  very much attention which means we can
  concentrate on new items
• Context can
• provide information
• make observer focus on one part of the display
• prime an observer so theyre biased towards what
  you want them to see

11
Basic Cognitive Principles
Visual Perception

• We excel at pattern recognition
• We automatically try to organize visual displays
  - gestalt principles
• Motion, contrast, color, outlining can provide
  salience
• Also, motor control (becoming more important)

12
Basic Cognitive Principles
Memory, Attention, and Visual Perception
Dynamically Interact
PERCEPTION
What is this feature? Does it match the
task? Recognition Pull info from memory Feedback
ATTENTION

• MEMORY

Motor Control
13
Early focus on users...

• Feature Prioritization, Task Analysis
• Participatory Design, Focus Groups
• Heuristic Evaluation, Design Guidelines
• Advisory Panels/Design Buddies
• Field Studies
• Cognitive Modeling exercises
• Competitive Testing
• Work with PSS

14
Toward a user-centered design

• Modeling customers activities (even mental ones)
• Understand activities, then create a solution
• A way to share information as a team
• Generating multiple solutions
• Developing usability goals
• Measuring against clear, quantifiable goals

15
Usability Goals Table
16
Usability goals--data analysis

• Collecting data video, protocols, subjective
  ratings and objective observations debrief
• Averages times, error time, of trials before
  success, of experimenter interventions,
  subjective ratings, of task interrupts,
  completed
• Usability issues with of Ss
• Look for patterns and lines of converging evidence

17
Recruiting participants

• Need to find external users that match your
  target population
• External users preferred to avoid bias
• Work with marketing to understand user
  characteristics and screen for them
• Screen via phone or Internet
• Background questionnaire at beginning of study
  session
• NDA, video consent

18
Getting users comfortable

• Greet the users promptly
• Offer beverage, washroom
• Give tour of lab, explain cameras,
  confidentiality
• Emphasize its the software design thats the
  issue, not the users expertise
• Adjust chair, table heights--ergo concerns
• Be enthusiastic, encouraging, listen!

19
During the study session

• Record EVERYTHING (and videotape)
• Every keystroke, comment (use video)
• Too hard to know ahead of time whats important
  or a trend
• Dont give too much help (3 levels) and
  record 1. Encouragement (Youre doing great!)
• 2. Hint (Have you looked under Find?)
• 3. Walk them through the task (1st go here, then
  here)

20
At sessions end.

• User Satisfaction Questionnaire (Chin, Diehl
  Norman or your own)
• Debrief
• Remind of key successes, failures and review
• Disclose total purpose of the study
• Provide gratuity and thank participant
• Escort out of the building

21
Interpreting results

• Be quick and back up all of your usability issues
  with data (quantify it if possible)
• Check your recommendations out with the
  team--some might not be possible
• Recommend well thought-out UI changes treat them
  like bugs
• Retest to see if the changes fixed the problems

22
Usability Issues Table

• Table contains the issue , its severity, the Ss
  that experienced the issue, its description, and
  its recommendation
• Optionally, you can status--e.g., did group
  agree to fix the issue and how, when?
• Another alternative is to use your corporations
  bug reporting tools

23
Golden Rule(s)

• List of top priority issues to team in 24 hours
  from last subject
• Report with recommendations out in 1 week
• Provide screen shots of tested UI and the
  recommended design solution
• This implies learning a quick and dirty drawing
  or prototyping tool
• (Gives you more influence and shows you care
  about fixing the problem)

24
Development stage design, test redesign

• Not traditional waterfall model
• Developing low-fi/hi-fi prototypes
• Broad and shallow
• Narrow and deep
• Testing with a few users
• Redesigning based on feedback
• Testing again

25
Drawing conclusions

• Usability issues and recommendations
• Updated usability goals table
• Important to mark specifics down and publish for
  archival purposes
• Usability issues should be tracked with PSS if
  unresolved

26
Toward beta.

• Identify usability showstoppers before ship
  fit and finish (e.g., audio tweaks, text)
• Competitive benchmarking
• Prioritize usability enhancements for next
  version
• Field research to understand real usage of
  products in context and usability opportunities

27
Cautions about lab testing

• Doesnt usually tell you what to
  design--structured user visits and interviews do
• YOU set the tasks, the design, and the analysis
• Best case performance
• Look for patterns of behaviors--the usability
  issues with the UI design

28
Accessibility issues

• Include users with disabilities during the design
  phase--Lighthouse for the Blind, etc.
• Also available--Design guidelines for accessible
  products

29
Important considerations...

• Ethical treatment of Ss, consent forms and NDAs
• Statistical power and significance
• Guided exploration v. free discovery, learning v.
  initial use
• Validity, reliability, and generalizability
• Objectivity

30
Resources Books

• Rubin, J. 1995. Handbook of Usability Testing.
• Dumas, J. Redish, J. 1993. A practical guide
  to usability testing
• Hellander, M. (Ed.) 1988 1998. Handbook of
  human computer interaction I II.
• Hix, D. Hartson, H. 1993. Developing user
  interfaces
• Nielson, J. 1993. Usability engineering
• Preece, J. 1995. Human-computer interaction

31
Resources URLs

• http//www.hfes.org
• SIGCHI - http//www.acm.org/sigchi/
• HCI Index - http//is.twi.tudelft.nl/hci/
• HCI Resources on the Net -
• http//www.ida.liu.se/labs/aslab/groups/um/hci/
• HCI-BIB - http//www.cmd.uu.se./html/hcibib.html/

32
Questions?