Hueristic evaluation

1
Hueristic evaluation
2
What is usability?
3
The evaluation process

• Identify values
• Set goals, objectives
• Operationalize goals measurable criteria
• Assess performance against criteria, goals
• Revise criteria, goals? values?

4
Kinds of usability criteria

• Usefulness task-related
• Functionality
• Content
• Integration with tasks, tools, activities
• User productivity
• Speed
• Ease of learning
• Ease of use
• Cognitive effort tasks, activities
• cognitive friction resistance encountered by
  human intelligence when it engages with complex
  set of rules that change.
• Quality of user experience
• perceptions, feelings, opinions

5
InfrastructureLeigh Star

• A practical match among routines of work
  practice, technology, and organizational and
  technical resources
• Transparent invisibly supports tasks usable
  systems disappear, by design and by habit.
• Learned as part of membership in a community of
  practice
• Links with conventions of practice
• Never transparent to everyone
• Becomes visible upon breakdown
• Is fixed in modular increments, not all at one or
  globally
• Layered and complex
• Change takes time and negotiation, and adjustment
  with other aspects of the system

6
Sources of usability criteria

• Organizational goals
• Pre-existing, general heuristics and guidelines
• Research
• Convention
• Consensus of experts
• Legal requirements (e.g., accessibility)
• Empirical
• User and task analysis
• Competitor sites (assumption their goals apply
  to your site, your users, also)

7
Types of measures

• Performance metrics
• Speed of response, availability, errors (e.g.
  dead links)
• User assessment
• opinion, perception, feeling
• Observable user behavior
• Time spent, user errors, operations performed
• Need to be cautious about drawing inferences,
  e.g. time spent on a page

8
Performance Metrics

• Only 5.3 percent of visitors could load the
  NYTimes' home page within 30 seconds on Nov 12,
  after the plane crashed on Long Island.
  MSNBC.com had 63 percent availability and
  response times of 26 seconds for those who could
  get a page during the hour following the crash..
  ..
• New York Times Digital served almost 13 million
  page views on Nov. 12, about 3 million more than
  its daily average. Following the crash, the
  company pulled ads and rearranged its page
  content so that text and Web links would load
  first. Prior to Nov. 12, it added 50 percent more
  server capacity following its brownout of Sept.
  11, and reconfigured its load balancers for
  better efficiency.

9
Evaluating usability

• Identifying values on the measures/criteria (how
  much?)
• Setting goals for levels of performance,
  interpreting values (how good?)
• It's difficult to believe the Times would find
  it OK if 95 percent of its print subscribers
  didn't get their newspapers in a timely way, or
  that MSNBC would conclude it had served customers
  well if 37 percent couldn't receive a clear
  broadcast signal.

10
Sources of data

• Unobtrusive
• Monitoring, observation
• Obtrusive, intentional
• Questioning
• Testing
• Opportunistic
• User feedback, questions, suggestions, complaints

11
Uses of heuristics and guidelines

• Competitive testing
• Design guidance
• Heuristic evaluation
• Embodying lessons learned for future design
  guidance

12
Heuristic evaluation

• The evaluators
• Who, how many
• Sources of heuristics, criteria
• Existing sets of guidelines, heuristics
• Customization for specific applications
• Applying the heuristics
• Advantages, disadvantages of heuristic evaluation

13
Heuristic evaluation Method

• Multiple evaluators
• Experts (in heuristic evaluation)
• Representative users
• How many?
• Nielsens rule of thumb 3 to 5
• Empirical rules of thumb
• keep going until you dont find much thats new
  and critical
• Have several (3-5 or more) representing each kind
  of use/user
• Inter-rater reliability consistency x (similar)
  raters
• Apply heuristics
• Individually
• Collective debriefing
• Determine severity
• Make recommendations for improvement

14
Nielsens basis - of evaluators
15
Heuristic evaluation Sources of Heuristics

• Pre-established
• Nielsen
• Other guidelines
• accessibility
• Customized
• For this kind of application (web, homepages)
• For this specific application
• Your users, their uses
• Mixed

16
Nielsens heuristics

• Visibility of system status
• Match between system and the real world
• User control and freedom
• Consistency and standards
• Error prevention
• Recognition rather than recall
• Flexibility and efficiency of use
• Aesthetic and minimalist design
• Help and documentation

17
Bruce Tognazzinis principles

• Anticipation
• Autonomy
• Color Blindness
• Consistency
• Defaults
• Efficiency of the User
• Explorable Interfaces
• Fitts's Law size and distance
• Human-Interface Objects

• Latency Reduction
• Learnability
• Limit Tradeoffs
• Metaphors
• Protect the Users Work
• Readability
• Track State
• Visible Interfaces

18
Guidelines and checklists

• How do you ensure usable sites, interfaces,
  software?
• Guidelines as presenting means toward end of
  usability
• E.g. web guidelines from http//usability.gov/guid
  elines/index.html
• Design guidelines
• Page length, layout font graphics
• Design process guidelines
• Set performance and/or preference goals
• Create and evaluate prototypes
• Usability guidelines
• Content/content organization navigation
  download time
• Accessibility

19
Web usability some relevant characteristics of
the web

• Uses, users may be hard to define
• Multiple and varied users, contexts, technology,
  and so on
• Uncontrolled
• Must design for naïve and experienced users
• The web itself as the context of use
• Users may be infrequent users who readily move to
  another site
• User expectations formed by other sites
• Users Web-specific concerns
• Privacy, security
• Users may not know who you are
• Demonstrating credibility

20
Nielsens113 Design Guidelines for Homepages

• Determining Homepage Content
• Vertical Industry Segments
• Communicating the Site's Purpose
• Communicating Information About Your Company
• Content Writing
• Revealing Content Through Examples
• Archives and Accessing Past Content
• Links
• Navigation
• Search
• Tools and Task Shortcuts
• Graphics and Animation
• Graphic Design
• UI Widgets

• Window Titles
• URLs
• News and Press Releases
• Popup Windows and Staging Pages
• Advertising
• Welcomes
• Communicating Technical Problems and Handling
  Emergencies
• Credits
• Page Reload and Refresh
• Customization
• Gathering Customer Data
• Fostering Community
• Dates and Times
• Stock Quotes and Displaying Numbers

21
Homepage Design Statistics (what people are used
to what others have found useful)

• Download Time
• Basic Page Layout
• Page Width
• Liquid Versus Frozen Layout
• Page Length
• Frames
• Fundamental Page Design Elements
• Logo
• Search
• Navigation
• Footer Navigation
• Site Map
• Routing Pages
• Splash Pages

• Frequent Features
• Sign In, About Us, Contact Info, Privacy
  Policy, Job Openings, Help
• Graphics and Multimedia
• Pictures, ALT Text , Music, Animation
• Advertising
• Typography
• Body Text and Background Colors
• Link Formatting

22
Rules of thumb based on research about users

• Guideline Design for monitors with a screen
  resolution of 800 x 600 pixels.
• Comments There is a definite trend in monitor
  design to go from screen resolutions of 800 x 600
  pixels to screen resolutions of 1024 x 768
  pixels. Five studies of screen resolutions were
  reviewed. Two of the studies reported that the
  largest number of users (53) were using screen
  resolutions of 800 x 600 pixels (27 were using
  1024 x 768). However, three of the studies
  reported that the largest number of their users
  (43) were using screen resolutions of 1024 x 768
  pixels (only 24 were using 800 x 600 pixels).
  Only about 7 of users are using 640 x 480
  pixels, and about 13 are using higher
  resolutions (1280 x 1024, 1600 x 1200, etc.)
• http//usability.gov/guidelines/softhard.htmlthre
  e

23
Users II

• Guideline Design for connection speeds of 56
  kilobytes per second (kbps).
• Comments Sixty percent of users use a 56 kbps
  connection speed or slower. The remaining users
  have faster connection speeds (ISDN, DSL, Cable,
  T1, etc.). Actual connection speeds are about 38
  lower than modem speed capability. This means
  that users with a 56 kbps connection actually
  have a connection averaging about 35 kbps. If you
  have data indicating that most, if not all, of
  your users have slower or faster connection
  speeds than 56K, determine what is appropriate.
• http//usability.gov/guidelines/softhard.htmlone

24
Weighting categories to differentiate by
importance

• Different from severity this is importance of
  feature/characteristic, not severity of problem
• http//www.weinschenk.com/tools/rate_scale.asp
  combines severity and weighting

25
Specialized Heuristics e-Commerce and Order
Forms

• From http//www.weinschenk.com/tools/online_checkl
  ist.asp
• Shows total cost
• Shows itemized costs
• Shows product names and/or descriptions
• Allows the user to change the quantity easily
• Provides an option to save an order and complete
  it later
• Provides details on any other charges on the
  order
• Provides details on shipping options and charges
• Provides shortcuts for repeat visitors to make
  transactions faster
• Allows users to easily move from the order form
  to shopping
• and back again
• Provides security information
• Provides users with an alternate offline way of
  ordering
• Allows users to view and/or change previous
  orders
• Does not require users to register before a
  purchase

26
Severity ratings

• Nielsen
• Based on frequency, impact, persistence
• Possible severity rating scale
• 0 not a problem
• Cosmetic need not be fixed unless time
  available
• Minor low priority
• Major high priority
• Catastrophe fix before release

27
Heuristic evaluation plusses and minuses

• Benefits
• Low resource requirements
• Usually find many problems fairly quickly
• Easy to repeat in iterative design
• Easy to communicate
• Usually easy to get agreement on a basic set of
  heuristics
• Face validity
• Limits
• Can be superficial
• Focuses on easily-seen problems harder to find
  more subtle problems associated with in-depth
  use, repeated use
• Can be deceptive assumption that evaluation has
  been more complete and thorough than it has been
• How similar to users are experts? How expert are
  users?
• How appropriate are the heuristics to THIS site?
• Tends toward a short list of heuristics
• Trade-offs among heuristics, the fixes needed?

28
Some key points from class discussion

• We need to differentiate among official
  standards, how people generally do things, and
  expert opinion
• Usability in a changing environment what people
  are used to, their technology and expectations,
  are continually evolving
• Heuristics need to be customized to goals,
  context
• Most guidelines are solutions to problems have
  to ask what is the underlying rationale, goal
• Trade-offs among different goals, heuristics
  often have to be made

29
(No Transcript)