What sells usability?

1
What sells usability?

• Nigel Bevan
• nigel.bevan_at_serco.com
• Serco Usability Services
• and University of York
• UK

2
Why is usability important?

• The individual components of usability make
  business sense
• Effective success in achieving goals
• Efficient productivity, staffing, waiting time
  in line
• Satisfied willingness to use the system
• Risk reduced business risk

3
Potential cost benefits

• Product sales can be increased as a result of the
  usability of the product
• Improving the competitive edge by marketing the
  product or service as easy to use
• Increasing the number of customers satisfied with
  the product who will make repeat purchases and
  recommend the product to others
• Obtaining higher ratings for usability in product
  reviews
• Employers can benefit from easier to use systems
  in the following ways
• Faster learning and better retention of
  information
• Reducing task time and increased productivity
• Reducing employee errors that have to be
  corrected later
• Reducing employee errors that impact on the
  quality of service
• Reducing staff turnover as a result of higher
  satisfaction and motivation
• Reducing time spent by other staff providing
  assistance when users encounter difficulties
• E. Suppliers and/or employers can benefit from
  reduced support and maintenance costs in the
  following ways
• Reducing support and help line costs
• Reducing costs of training
• Reducing maintenance costs

• Development costs can be reduced by
• Producing a product that has only relevant
  functionality
• Detecting and fixing usability problems early in
  the development process
• Reducing the cost of future redesign or radical
  change of the architecture to make future
  versions of the product more usable
• Minimizing or eliminating the need for
  documentation
• Redesigning web sites to increase revenue, not
  just to change the image
• Reducing the risk of product failure
• E-commerce sales can be improved by increasing
  the number of web site customers who will
• Be able to find products that they want
• Find supplementary information easily (e.g.
  delivery, return and warranty information)
• Be satisfied with the web site and make repeat
  purchases
• Trust the web site (with personal information and
  to operate correctly)
• Not require any support, or use the web site for
  support rather than calling the support center
• Recommend the site to others
• Support and increase sales by other channels

4
So why is usability often ignored?

• Believed to add to project risk
• Additional time and cost
• Outside area of experience
• No established process for usability
• Not specified in requirements
• Not my responsibility
• Customers dont ask for it
• Intangible
• Every user wants something different

5
Barriers to uptake

• What motivates an organisation (or project) to
  invest in usability?
• The barriers, perceived
• Increased cost
• Increased development time
• Therefore increased risk?

6
Main motivator for investing in usability

• The requirements/RFP mentions ease of use
• What is the least we can do to address this?
• This project has a problem with usability
• Usability can help solve it ie risk reduction
• Senior usability convert
• Usability is important to our business success
• We realise that usability is important for
  success
• Usually for consumer products and web sites

7
Selling the concept?

• Usability can
• improve your productivity
• Improve your sales
• Improve the quality of your product
• Reduce the risk of product failure
• Improve your process
• Drive innovation (IBM)
• Improve affecive/emotional design (Arnie)
• Usability helps design products to match user
  needs (Arnie)
• Usability is acknowledged good practice

8
The importance of user requirements

• Standish Group found that
• 15 of projects failed
• 51 were partially successful
• Main causes were poor user requirements
• 13.1 Incomplete requirements
• 12.4 Lack of user involvement
• 10.6 Inadequate resources
• 9.9 Unrealistic user expectations
• 9.3 Lack of management support
• 8.7 Requirements keep changing
• 8.1 Inadequate planning
• 7.5 System no longer needed

9
One number for usability

• Jeff Sauro has shown that it can be done
• Sauro and Kindlund, UPA 2005
• Making Sense of Usability Metrics Usability and
  Six Sigma
• Satisfaction questionnaire can provide a single
  number
• But the weighting is context-dependent?
• The individual components of usability make
  business sense
• Effective success in achieving goals
• Efficient productivity, staffing, waiting time
  in line
• Satisfied willingness to use the system
• Risk reduced business risk

10
But, to quote Allen Cooper

• If you were a plastic surgeon, would you go down
  the road knocking on doors, and telling the
  person who comes to the door
• I can see you are ugly but I am a plastic
  surgeon and I can help you!

11
Tailor the message for the audience

• No one size fits all?
• Adapt the message to the audience?
• But how do you know what the audience wants to
  hear?
• Need to analyse the business environment and the
  motivators and constraints for the gatekeeper
• Tailor the message to the context of use. What
  matters to the gate keeper?
• Increased revenue?
• Decreased costs?
• Reduced risk of failure of the product?
• None of these no sell!

12
(No Transcript)
13

• Essential prerequisites for usability

14
Human centred design process for interactive
systems ISO 13407 (1999)
1. Plan the human centred process
Meets requirements
2. Specify the context of use
5. Evaluate designs against user requirements
3. Specify user and organisational requirements
4. Produce design solutions
15
Essential activities to achieve usability
PlanProcess
SpecifyContext of Use
DesignSolutions
SpecifyRequirements
Evaluate against Requirements
System lifecycle
feasibility
requirements
design
implement
release
4. Usability requirements 5. Evaluate existing
system
10. Collect feedback
6. Prototyping 7. Style guide
8.Evaluation 9. Usability testing
www.usabilitynet.org/trump/ucdmethods
16
ISO TR 18529 Human-centred lifecycle processes
descriptions

• HCD.1 Ensure HCD content in system strategy
• HCD.2 Plan and manage the HCD process
• HCD.3 Specify the user and organisational
  requirements
• HCD.4 Understand and specify the context of use
• HCD.5 Produce design solutions
• HCD.6 Evaluate designs against requirements
• HCD.7 Introduce and operate the system
• Can be used to assess organisational capability
  and identify needs for process improvement

17
ISO TR 18529 Human-centred lifecycle processes
descriptions

• Understand and Specify User and Organizational
  Requirement and Context of Use
• 2.1 - Clarify and document system goals
• 2.2 - Analyze stakeholders
• 2.3 - Assess risk to stakeholders
• 2.4 - Identify, document and analyze the context
  of use
• 2.5 - Define the use of the system
• 2.6 - Generate the stakeholder, user and
  organizational requirements
• 2.7 - Set usability objectives

18

• What is usability?

19
ISO/IEC 9126 Software quality characteristics

• 1991
• Functionality
• Efficiency
• Reliability
• Usability
• Maintainability
• Portability

20
ISO/IEC 9126-1 Software Product Quality Model
(1991)
functionality
reliability
accuracy suitability interoperability security
maturity fault tolerance recoverability availabili
ty
efficiency
usability
understandability learnability operability attract
iveness
time behaviour resource utilisation
maintainability
portability
adaptability installability co-existence replaceab
ility
analysability changeability stability testability
21
ISO 9241-11 Guidance on Usability (1998)
Usability The extent to which a product can be
used by specified users to achieve specified
goals with effectiveness, efficiency and
satisfaction in a specified context of
use Effectiveness The accuracy and
completeness with which users achieve specified
goals. Efficiency The resources expended in
relation to the accuracy and completeness with
which users achieve goals. Satisfaction The
comfort and acceptability of use
22
Usability model
personalenvironment
goals
user
user satisfaction
interactiontasks
physical environment
output effectiveness efficiency
social and organisationalenvironment
product
technical environment
23
ISO/IEC 9126-1 Software Product Quality Model
functionality
reliability
accuracy suitability interoperability security
maturity fault tolerance recoverability availabili
ty
efficiency
usability
understandability learnability operability attract
iveness
time behaviour resource utilisation
maintainability
portability
adaptability installability co-existence replaceab
ility
analysability changeability stability testability
24
Quality in use

• The extent to which a product used by specific
  users meets their needs to achieve specific
  goals with effectiveness, productivity, safety
  and satisfaction in specific contexts of use
• Effectiveness
• The extent to which users are able to achieve
  specified goals with accuracy and completeness in
  a specified context of use
• Productivity
• The extent to which users are able to expend
  appropriate amounts of resources in relation to
  the effectiveness achieved in a specified context
  of use
• Satisfaction
• The extent to which users are satisfied in a
  specified context of use
• Safety/Risk
• The extent to which risk of harm to people,
  business, software, property or the environment
  is acceptable in a specified context of use

25
How do you know a consumer product is usable?

• Cant tell from appearance in the shop
• Difficult to tell from a demonstration
• By the time you find out it is too late
• So no incentive for suppliers

26
Plan for Usability Assurance Statements

• 1990
• Planned that ISO 9241-11 would provide Usability
  Assurance Statements
• Usability test results giving effectiveness,
  efficiency and satisfaction in a particular
  context of use

27

• But not everyone wants their usability assured

28
US computer company opposed usability assurance
statements

• The results could be misinterpreted
• They did not wish to be obliged to produce such
  statements
• US computer company around the world would lobby
  for vote no votes
• So usability assurance statements were dropped

29
Common Industry Format for usability test reports

• NIST initiative in 1998
• National Institute of Standards and Technology
• Suppliers provide standard test reports to
  purchasers
• Suppliers included
• IBM, Microsoft, HP, Sun, Oracle, Compaq
• Purchasers included
• Boeing, Northwest Mutual Life, State Farm
  Insurance, Fidelity, Kodak
• Reports provided in confidence
• Could permit comparisons

30
Common Industry Format usability tests

• Objectives
• Raise the profile of usability in procurement
• Encourage suppliers to work more closely with
  purchasers to understand user needs
• A common format for sharing usability data
• Reduce uncontrolled overhead costs of usability
  problems
• Enable purchasers to compare usability

31
CIF motivation

• Boeing
• We traditionally have had little visibility of
  how usable a product will be or how much training
  and support users will need. This has made it
  difficult to compare products, to plan for
  support, or estimate total cost of ownership.
• US WEST
• US WEST has been actively participating in
  the CIF initiative and will clearly benefit
  from the results of this effort such as a
  standard testing process for usability, a
  standard specification for reporting usability
  tests, and other techniques to enable us to
  partner more effectively with our vendors.
• State Farm Insurance
• We have found it difficult to identify software
  products that meet our needs without contributing
  to excessive overhead, increasing support costs,
  or negatively impacting employee productivity or
  morale. If successful the CIF initiative should
  result in the development of better, more usable
  software for all of industry.

32
How it should work

• Purchaser requests supplier to provide a CIF
  report
• initially as part of agreed trials
• The supplier may already be planning a usability
  test
• may have to adapt the test plan to be suitable
  for CIF report
• or
• The supplier asks the purchaser to carry out the
  test
• or
• The supplier asks a third party to carry out the
  test
• The test results help the purchaser decide
• whether to purchase
• whether to request changes

33
Report format - technical

• Test objectives
• Participants
• Total number and user groups
• Key characteristics and capabilities expected
• How selected with the key characteristics and
  capabilities?
• Context of evaluation
• Task scenarios and task data used
• Any performance criteria used
• Physical and computer environment

34
Report format - results

• Design
• Independent and control variables
• Measures used
• Any training and assistance given
• Task instructions
• Effectiveness
• completely and correctly achieving their goals
• Separate data where assistance was given
• Efficiency
• Mean time taken to complete the task
• Standard deviation (and standard error)
• Satisfaction
• Optional own scales and/or standard questionnaire

35
Common Industry Format Standard

• ANSI/NCITS 354
• ISO 25062

36
Why are so many consumer products difficult to
use?

• Difficult to assess usability when purchasing
• Corporate purchasers want usability reported in
  the Common Industry Format
• Consumers need a label or statement

37
ISO 20282 Ease of operation of everyday products
Part 1 Context of use and user characteristics

• Context of use
• Are the main goals identified?
• Are actions on other equipment influential?
• Are environmental factors relevant?
• Is privacy or the social environment relevant?

38
User characteristics

• Psychological and social characteristics
• Do cognitive abilities influence ease of
  operation?
• Do knowledge, experience and habit influence ease
  of operation?
• Do cultural differences influence ease of
  operation?
• Does literacy influence ease of operation?
• Does language influence ease of operation?
• Physical and sensory
• Are body dimensions relevant for ease of
  operation?
• Are strength and biomechanical abilities
  relevant?
• Are visual abilities relevant for ease of
  operation?
• Are auditory abilities relevant for the ease of
  operation?
• Does handedness influence the ease of operation?
• Categories
• Does age influence the ease of operation?
• Does gender influence ease of operation?

39
(No Transcript)
40
ISO CD 20282-2 Test method

• A summative test method for measuring usability
  based on
• ISO 9241-11 (Guidance on usability)
• Common Industry Format for usability test reports
• Identify intended context of use and user groups
• Measures
• Success rate (ease of operation)
• Task time (efficiency)
• Satisfaction scale

41
ISO TS 20282-2 Test method for usability of
everyday products

• Measures
• Success rate (effectiveness)
• Task time (efficiency)
• Satisfaction scale
• Criteria
• 80 of the intended users can successfully
  complete the main tasks without assistance

42
A manufacturer can use ISO 20282-2 to provide
evidence of the ease of operation of a product
for marketing purposes

• A manufacturer wants to demonstrate that a
  videorecorder is very easy to program without
  instructions.
• A stratified sample of 50 people who have
  purchased a videorecorder with the intention of
  programming it to record programmes are recruited
  who are representative in age, education and
  ownership of brands of videorecorder owners.
• Each session is expected to take a total of 5-10
  minutes, and 2 videorecorders are set up with TVs
  for programming.
• Each person is given asked to program the
  videorecorder to record a specified programme in
  a TV magazine. It is expected that most people
  will use the Videoplus code, but direct
  programming on the channel and time is also
  acceptable.
• The test result is given as the measured success
  rate together with a confidence interval. To be
  95 confident that 80 of the user population can
  program the videorecorder, 45 of the 50 people
  tested will have to be successful.

43

• Not everyone wants their usability assured
  (again)

44
German computer company opposes the standard

• The results could be misinterpreted
• We do not wish to be obliged to carry out such
  tests
• Remove all reference to consumer products
• Lobbied for for industry association to vote no
• Tried to raise procedural obstacles to prevent
  the standard progressing

45

• We nearly forgot about requirements!

46
Common Industry Format
Supplier
Consumer
Usability requirements
Usability test report
Purchase decision
CIF
47
CIF Requirements

• Context of use
• Users
• Key characteristics and capabilities of each user
  group for which requirements are provided
• Goals
• Main goals for each group with scenarios of use
• Equipment technical environment
• Computer hardware and software
• Physical and social environments
• Scenarios of use for most important goals
• Training scenarios
• Usability measures
• Effectiveness unassisted completion rate
• Efficiency mean time to achieve goals
• Satisfaction mean score on a satisfaction scale
• Relative user efficiency (optional)

48
CIF Requirements

• Context of use
• Users
• Key characteristics and capabilities of each user
  group for which requirements are provided
• Goals
• Main goals for each group with scenarios of use
• Equipment technical environment
• Computer hardware and software
• Physical and social environments
• Scenarios of use for most important goals
• Training scenarios
• Usability measures
• Effectiveness unassisted completion rate
• Efficiency mean time to achieve goals
• Satisfaction mean score on a satisfaction scale
• Relative user efficiency (optional)

49
Example Usability measures

• Task 1 Download SITE guard software
• Effectiveness 90 unassisted task completion
  rate.
• Efficiency 10 minutes of user time.
• Satisfaction Total SUMI score of 50.
• Task 2 Burn SITE guard software to DVD and
  launch installer
• Effectiveness 90 unassisted task completion
  rate.
• Efficiency 15 minutes of user time.
• Satisfaction Total SUMI score of 50.
• Task 3 Prepare SITE guard software for staged
  install and launch installer
• Effectiveness 70 unassisted task completion
  rate.
• Efficiency 25 minutes of user time.
• Satisfaction Total SUS or SUMI score of 50.

50
What can YOU do about it?Usability Professionals
Association

• World Usability Day
• 14 November 2006
• Usability Body of Knowledge

51
World Usability Day
52
(No Transcript)