Overview of Usability Testing

1
Overview of Usability Testing

• SE 452, Software Verification and Validation
• Mark Ardis, RIT

2
Outline

• Goals
• Limitations
• Methods
• Environments

3
General Goals

• Ensure creation of systems that are
• easy to learn and use
• satisfying to use
• provide value to the user

4
Benefits of Usability Testing

• Create historical record of benchmarks
• Minimize cost of service calls
• Increase sales
• Provide competitive edge
• Minimize risk

5
Limitations of Testing

• Always an artificial situation
• Tests do not prove that the product works
• Participants in tests are rarely fully
  representative of target population
• Testing is not always the best technique to use

6
Basic Method

• Form a hypothesis
• Choose participants
• Conduct test
• Analyze results

7
Types of Tests

• Exploratory
• Assessment
• Validation
• Comparison

8
Exploratory Tests

• Explore preliminary design concepts
• Use a prototype of system
• User follows a simple scenario
• User should "think aloud"
• Test monitor asks opinion of user during test

9
Assessment Tests

• Evaluate how successfully a design has been
  implemented
• Use the implemented system (or part of it)
• User performs specific tasks
• Quantitative measures are collected

10
Validation Tests

• Compare product to a benchmark
• Use a completed system
• Users perform specific tasks
• Quantitative data is collected for comparison
  with benchmark

11
Comparison Tests

• Determine which of 2 versions is better
• May be used at any stage of development
• Side-by-side comparison of 2 approaches
• Should propose radically different approaches

12
First Cartoon of the Day
13
Example User Profile
14
Attributes of Participants (1/3)

• Personal History
• Age
• Gender
• Attitude toward computers
• Left or right-handed
• Learning style
• Attitude toward high tech

15
Attributes of Participants (2/3)

• Education
• Highest grade completed
• Subjects studied
• Major
• Computer Experience
• Total time using computers
• Frequency of use
• Types of systems used

16
Attributes of Participants (3/3)

• Product Experience
• Total time used
• Frequency of use
• Types of tasks performed
• Experience with other brands
• Occupation History
• Current and past job titles
• Training classes taken
• Time with company

17
Problems to Avoid

• Purchaser vs. User population
• Selecting only the best users
• Self selection
• Attributes that cluster together

18
Second Cartoon of the Day
19
Test Environment

• 4 different choices
• Each has advantages and disadvantages
• Consider
• Test Monitor
• Observers
• Participant

20
Simple Single-Room
21
Modified Single-Room
22
Electronic Observation Room
23
Classic Testing Lab