Metrics for Heuristics

1
Metrics for Heuristics

• Proving the
• Information Architects Case
• Andrea Wiggins

2
What Information ArchitectsWant to Know

• Context for making design decisions
• Validation of heuristic assumptions
• Understand why visitors come to the site what
  they seek

3
Everybody Loves Rubinoff

• UX audit quantifies subjective measures
• Offers structure for comparing properties of the
  site
• Completely customizable, use strategically
• In a perfect world
• Analyst IA work together to set key performance
  indicators (KPI) and measurable heuristics
• Each independently evaluates the site on the same
  points
• They compare the IAs heuristics to user data for
  validation
• They work with the client to set before-and-after
  measures to prove value for the entire redesign
  project

4
Rubinoffs Four Categories

• Branding
• Engaging, memorable brand experience
• Value of multimedia graphics
• Functionality
• Server response time technical errors
• Security privacy practices
• Usability
• Error prevention recovery
• Supporting user goals tasks
• Content
• Navigation site structure
• Search referrals

5
Branding Memorable, Engaging Experiences

• Ratio of new to returning visitors, set target
  KPI specific to site business goals
• Track trends over time and in relation to
  cross-channel marketing
• Average visit length, in time pages viewed
• Depth or breadth of visits
• Segment new and returning visitors to examine
  visit trends for different audiences

6
Adding Value with Multimedia Graphics

• Flash AJAX require deciding upon what to
  measure and programming custom parameter strings
  into the application
• Problematic multimedia logging is
  action-oriented, analysis tools are page-oriented
• Plan to include measures when designing
  multimedia applications to prove value
• Compare clickthrough rates for clickable graphics
  to rates for standard navigation links

7
Server Response Time Technical Errors

• Response time is a default log field, easy to
  measure
• Check at peak load time to make sure site is
  responding quickly enough
• Monitor the rate of 500 (server) errors this
  should be an extremely low number

8
Security Privacy Practices

• A matter of design for measurement, not
  measurement of design considerations for
  designing a site that will be measured
• Privacy best practices
• Give a short, accurate, easy to understand
  privacy statement and stand by your word
• True first-party cookie
• Security best practices (from an IA/analytic
  POV)
• SSL encryption on any transactional forms lead
  generation, ecommerce, surveys
• Secure file transfer for restricted access to
  raw web analytic data password restrictions at
  minimum.

9
Error Prevention Recovery

• Percentage of visits experiencing 404 and 500
  errors errors should be
• Percentage of visits including an error that end
  with an error
• Where do 404 errors occur?
• Use to build a redirect page list to ensure
  (temporary) continuity of service to bookmarked
  URLs
• Path/navigation analysis how did users arrive at
  404? What did they do after?
• User errors

10
Supporting User Goals Tasks

• Scenario/conversion analysis
• Define tasks and procedures
• Examine completion rates, step by step, intervals
  overall
• A to B, B to C, C to D A to C, B to D A to D
• Look at leakage points
• Where did they go when they left the process? Did
  they come back later?
• Shopping cart analysis
• Keep in mind that users shop online for offline
  purchases
• Do behaviors suggest a need for a tool like a
  shipping calculator or product comparison?
• Online form completion
• Field-level tagging helps maximize data capture

11
Content AccessibilityNavigation Site Structure

• Pogo-sticking jumping back forth between
  content or hierarchy levels
• Need a comparison tool, cant identify product
  not enough detail at the right level of site
  hierarchy or step of the purchase decision
  process
• Compare page-level traffic statistics for larger
  trends, broad navigation analysis the usual
  statistics
• Path analysis on navigation tools (by type) to
  pinpoint navigation and labeling problems
• Extensive use of supplemental navigation
  indicates need for updates to global navigation

12
Content Value Mining Search Referrals

• Popularity value? What about findability?
• Compare the contents value (against similar
  content) with proportions of returning visitors,
  average page viewing length, external referrals -
  especially search referrals
• Bookmark or WOM/viral referrals to content pages
• No referrer (and not a spider or bot)
• Content page is first page of visit
• High proportion of returning visitors indicates
  valued content
• Search log analysis what do your users value?
• Does user query language match site contents?
  Are users searching for panties when youre
  selling pants?

13
Validate the Match Between the Site the Real
World

• My favorite heuristic from Jakob Nielsen
• More ways to use search log analysis
• Does user vocabulary match site vocabulary?
• Do different audiences have different
  vocabularies, and does the site support them?
• Brand measurement returns
• product and industry terminology usage
• accuracy of brand queries spelling, inclusion
  of competitors brands, advertising slogans
• Did users find what they expected? How many
  visits end on search results? Null results are
  revealing.

14
Language Validation
15
Thanks! Any questions?