CMC/CC A Usability Evaluation

1
CMC/CC AUsability Evaluation

• Master IK, CIW, MMI
• L.M. Bosveld-de Smet
• Course 4 mon. 02/10/06 16.00-18.00

2
Usability Evaluation
3
Design of interactionDix et al. (2004)
What is wanted
Analysis
Design
Usability Evaluation
Implement and deploy
Prototype
4
Design process of usable interactive systems

• Interaction design process
• Complex
• Iterative
• Never complete
• User-centered design
• Within software engineering framework
• Design rules
• Implementation support
• Evaluation techniques
• Universally accessible designs
• Provision of user support

5
3 related concepts

• Usability measure of success of a product
• User-Centered Design design involving user
  participation
• Usability Engineering
• Whole process ensuring usable interactive systems
• Process aiming at systems fit for the user, the
  task, the environment
• Process implying user participation during
  software development
• Process committing itself to design-evaluate-redes
  ign development cycle
• In search for objective measures for user
  interface

6
Usability different views

• Same intuitions vs. Different approaches
• Different views on
• Definition of usability
• Usability attributes
• Importance of user-centered design
• When and how to deal with user participation
• How to build usable systems
• How to measure usability

7
Usability Attributes Shackels view

• A usable product is one that users find
  satisfactory for the tasks for which it was
  designed.
• Good design for usability depends upon achieving
  successful harmony in the dynamic interplay
  between user, task, system, and environment.
• Usability 4 usable criteria
• Learnability
• Effectiveness
• Attitude
• Flexibility

8
Usability Attributes ISO definition

• Usability is the effectiveness, efficiency and
  satisfaction with which specified users can
  achieve specified goals in particular
  environments.
• Effectiveness accuracy and completeness with
  which users achieve specific goals
• Efficiency accuracy and completeness of goals
  in relation to resources expended
• Satisfaction comfort and acceptability of the
  system

9
An aside User Interface Standards

• Disadvantages
• Constrain design
• Stagnate innovation
• Describe principles, do not provide solutions
• Become quickly obsolete
• Advantages
• Define good practice
• Affect attitudes w.r.t. software development

10
User-Centered Design Karats view

• Karat (1996) UCD Quality or Quackery?
• UCD is an iterative process whose goal is the
  development of usable systems, achieved through
  involvement of potential users of a system in
  system design
• I suggest we consider UCD a nice fluffy little
  catch phrase. It captures a commitment that the
  usability community supports that you must
  involve users in system design while leaving
  fairly open how this is accomplished

11
UCD a more objective view

• UCD can be accomplished through the application
  of Usability Engineering
• Design should centre on users
• End-user should be consulted
• Needs of end-users should be considered

12
HCI in software process

• Software engineering
• Usability engineering
• Iterative design practices
• Design rationale

13
Waterfall model
Requirements Specification
Architectural design
Detailed Design
Implementation and Unit testing
Integration and Testing
Operation and Maintenance
14
Usability Engineering

• Usability specification as part of requirements
  specification
• Iteration and testing
• List of usability measurement requirements
• (Whiteside, Bennett and Holtzblatt, 1988)
• Time to complete a task
• Ratio of successes to failures
• Time spent in errors
• Number of commands used
• Frequency of help and documentation use

15
Iteration in waterfall model
Requirements Specification
Architectural design
Detailed Design
Implementation and Unit testing
Integration and Testing
Operation and Maintenance
16
Usability Engineering LifecycleFaulkner (2000)

• Know the user
• Know the task
• User requirements capture
• Setting usability goals
• Design process
• Apply guidelines, heuristics
• Prototyping
• Evaluation with users
• Redesign and evaluation with users
• Evaluation with users and report

17
Role of prototyping
Design
OK?
Prototype
Evaluate
Done!
Not OK?
Redesign
18
Iterative design and prototyping

• Three main approaches to prototyping
• Throw-away prototyping within requirements
  specification
• Incremental prototyping within the life cycle
• Evolutionary prototyping throughout the life cycle

19
Throw-away prototyping
Preliminary Requirements
Build Prototype
Evaluate Prototype
Adequate?
no
Final Requirements
yes
20
Evolutionary prototyping
Build Prototype
Req
Arch
Det
Evaluate Prototype
Impl
Int
Operation and Maintenance
21
Usability Evaluation (UE)

• Methodologies for measuring usability aspects of
  systems user interface and identifying specific
  problems (Dix et al. 1998 Nielsen 1993)
• Should occur throughout design life cycle
• Common activities
• Capture
• Analysis
• Critic
• There is a wide range of UE techniques
• Each technique has its own requirements
• Different techniques uncover different usability
  problems

22
Taxonomies of UE (1)

• Formative vs. Summative
• Analytical vs. Empirical
• Expert analysis vs. User participation
• Analytic methods
• Review methods
• Model-based methods
• vs.
• Experimental methods
• Observational methods
• Query methods

23
Taxonomies of UE (2)

• Automated vs. Non-automated
• Ivory and Hearst (2001)
• Testing
• Inspection
• Inquiry
• Analytical modeling
• Simulation

24
Expert Analysis
25
Controlled Experiment
26
User participation
27
Important factors choice UE

• Stage in cycle at which UE is carried out
• Style of UE
• Level of subjectivity or objectivity of UE
  technique
• Type of measures provided
• Information provided
• Immediacy of response
• Level of interference implied
• Resources required

28
Analytic UE techniques
Cognitive Walkthrough Heuristic Evaluation Review based Model based
Stage throughout throughout design design
Style laboratory laboratory laboratory laboratory
Objective? no no as source as source
Measure qualitative qualitative as source qualitative
Information low level high level as source low level
Immediacy N/A N/A as source N/A
Intrusive? no no no no
Time medium low low-medium medium
Equipment low low low low
Expertise high medium low high
29
Experimental and query UE techniques
Experiment Interviews Questionnaire
Stage throughout throughout throughout
Style laboratory lab / field lab / field
Objective? yes no no
Measure quantitative qualitative / quantitative qualitative / quantitative
Information low/high level high level high level
Immediacy yes no no
Intrusive? yes no no
Time high low low
Equipment medium low low
Expertise medium low low
30
Observational UE techniques
Think aloud Protocol analysis Post-task walkthrough
Stage implementation implementation implementation
Style lab / field lab / field lab / field
Objective? no no no
Measure qualitative qualitative qualitative
Information high/low level high/low level high/low level
Immediacy yes yes no
Intrusive? yes yes no
Time high high medium
Equipment low high low
Expertise medium high medium
31
Monitoring UE techniques
Eye tracking Physiological measurement
Stage implementation implementation
Style laboratory laboratory
Objective? yes yes
Measure quantitative quantitative
Information low level low level
Immediacy yes yes
Intrusive? no yes
Time medium/high medium/high
Equipment high high
Expertise high high
32
Vocabulary application

• Task analysis
• Feasability study
• Design representation

33
Feasibility study

• Why is system needed?
• How will system help to improve user task
  performance?
• Are there critical processes that need to be
  supported by system?
• What are the technical implications?
• Can system be produced within given budget?
• Is there a timescale for the development of the
  system?

34
Strategies for representing design

• Storyboards
• State transition diagrams
• Simulations
• Scenarios
• Rapid prototyping
• Wizard of Oz

35
Storyboard
36
State Transition Diagram