Asking Users and Experts

1
Asking Users and Experts

• Bobby Kotzev
• Adrian Sugandhi

2
Outline

• Asking Users
• Interviews
• Questioners
• Asking Experts
• Inspections
• Walkthroughs

3
Interviews, Survey, or Focus Group?

• Interview
• Take significant time and need more resources
• Survey
• Large data set
• Focus group
• Seeking multiple point of views in a shorter
  period of time

4
Preparing for InterviewsThings to be aware

• Be alert to unconscious biases
• Honesty
• Social desirability
• Prestige response bias
• Sensitive or highly personal topics survey?
• Consider the interviewee as he/she is Doing you
  a favor

5
Preparing InterviewsOutcome Analyses

• Outcome Analyses
• Plan outcome-based interviews
• Associate questions with the goals
• Capture desired outcomes
• Differentiate between outcome and solution
• Organize the outcomes
• Rate the outcomes for importance and
  satisfactions
• Importance (Importance-Satisfaction)
  Opportunity
• Use the outcomes to jump-start innovation

6
Preparing InterviewsPre-interview

• Identify the objective of the study
• Select type of interviews
• Medium of interview (in person/phone)
• Decide how you will analyze the data
• Write the questions
• Avoid long questions (Brevity)
• Avoid compound sentences
• Avoid using jargon
• Avoid leading questions
• Avoid Biases
• Clarity
• Avoid future prediction questions
• Avoid inaccessible topics
• Test your questions

7
Preparing InterviewsPlayers in the Activity

• Participants
• 6-10 of each user type (diversity)
• Interviewers
• Ensure participants understand the questions
• Must be skilled
• The note-taker
• Interviewers can focus more on body language
• The videographer
• Whenever possible, record the interview session

8
Conducting Interviews Sections

• Introduction
• 5-10 Minutes
• Warm-up
• 5-10 Minutes
• Main section
• 85-100 Minutes
• Cool off period
• 5 Minutes
• Closing

9
Preparing for Interviews Additional Preparations

• Run pilot study
• Be professional
• Dress similarly and appropriately
• Prepare informed consent
• Check and familiarize yourself with recording
  equipment
• Record answers exactly

10
Interviews Types

• Interview Types
• Open-ended
• Unstructured
• Structured
• Semi-structured

11
Unstructured Interviews

• A conversation that focus on a particular topic
• Open questions
• Can generate rich data which the interviewer
  havent thought about
• Generates a lot of unstructured data
• Impossible to replicate
• Hard to analyze

12
Unstructured Interviews Prepare

• Have an agenda
• Be prepared to follow new lines
• Pay attention to ethical details
• Respond with sympathy but make sure to avoid
  putting ideas in the users head
• Analyze data as soon as possible after the
  interview

13
Structured Interviews

• Predetermined questions, similar to a
  questionnaire.

14
Semi-Structured Interviews

• Starts as a structured interview but can change
  direction and inject open-ended questions
  depending on responses
• Avoid preempting answers
• Probes Do you want to tell me anything else
  about

15
Interviewers Role

• Do not interrupt
• Keep on track
• Unstructured interview easy goes off-track
• Silence is golden
• permission to provide more detail
• Remain attentive
• Asking the tough questions
• Wait until you develop rapport
• Using examples
• Watch for generalities
• Do not force choices

16
Interviewers Role (contd)

• Watch for markers
• Key events to probe for more rich information
• Select the right type of probes
• Know when to move on
• Reflecting
• Summarize, reword, or reflect responses
• Empathy and antagonism
• Transitions
• Transition smoothly from one topic to another

17
Monitoring the Relationships with the
Interviewee/Participant

• Watch participants body language
• Nervous, tenses go to easier question or
  restate purpose/motivation of the study
• Fighting for control
• Ask yourself why one refusing your questions
• Hold your opinions
• Dos and Donts

18
Interview Data Analysis and Interpretation

• Must be analyzed shortly after each interview
• Categorizing
• Affinity diagram
• Qualitative analysis tools
• Look for patterns

19
Interview Communicate the Findings

• Over time
• By topic
• By participant
• Vehicles for communicating the results
• Summarized poster
• Identify follow-up activities based on the
  results
• Table of recommendations (uncovered issues and
  next steps)

20
Questionnaires

• Good, established technique for collecting
  demographic data and users opinion
• Can be Closed and Open.
• Can be distributed to large number of people

21
Questionnaires Designing

• Start with basic demographic data
• Make questions clear and specific
• When possible ask closed questions and offer a
  range of answers
• Include no-opinion option
• Ordering matters
• Avoid complex and compound questions
• Use intuitive, consistent scaling
• Avoid jargons
• Provide clear instructions how to complete the
  questioner

22
Questionnaires Question Types

• Checkboxes
• Likert Scales
• Semantic differential scales

Gender Maleq Femaleq
23
Questionnaires Administering

• Well designed
• Include stamped self addressed envelope
• Provide a short version
• Contact and respondents trough a follow-up, mail,
  e-mail phone
• Offer incentives like payments

24
Questionnaires Online Forms

• Pros
• Reach large audience
• Response is fast
• No postage costs
• Data is already in a electronic format and can be
  imported easily in DB
• Errors can be corrected easily and fast
• Cons
• Obtaining random sample is difficult web users
  represent only a certain demographic.
• Participants are self selecting proclaimed
  nonscientific

25
Questionnaires Online Forms (cont)

• Construct Online questioners based on their paper
  forms
• Run pilot studies

26
Asking Experts

• Experts will be power users but also could be
  involved with the development of similar,
  successful products so they have an deeper
  understanding.

27
Asking Experts Inspections

• Heuristic evaluations
• evaluate user-interface against a predetermined
  guidelines and principles

Visibility of system status Are users kept
informed about what is going on? Is appropriate
feedback provided within reasonable time about a
user action? User Control and freedom
Are there ways of allowing users to easily escape
from places they unexpectedly find
themselves in? Consistency and standards
Are the ways of performing similar actions
consistent?
28
Asking Experts Inspections (cont)

• Different heuristics are needed for different
  products, devices and software
• Using heuristics and experts can reveal issues
  fast and inexpensively 5 experts can reveal 75
  of issues - graph
• Performing heuristic evaluation
• Spend 1-2 hours with the product.
• At least two passes one to get the feel, second
  to walk trough the interface and address all
  aspects of usability.
• Have a specific task in mind when evaluating
  functional products

29
Asking Experts Inspections (cont)

• Heuristic evaluation of websites
• Internal consistency
• Simple dialog
• Shortcuts
• Minimizing the user memory load
• Preventing errors
• Feedback
• Internal locus of control
• Layout
• Internal consistency

30
Asking Experts Problems

• Reporting issues which are not there (Bill
  Bailey 2001)

31
Asking Experts Walkthroughs

• Walk through a task and to notice problems
• Cognitive walkthroughs simulating the user
  problem solving process for a task
• Figure if the users will know what to do, how to
  do it and weather the action was correct or not.
• Record assumptions, side issues and summaries of
  results
• Pluralistic walkthroughs
• Users developers and usability experts get
  together to step through a scenario

32
Summary

• Interviews
• Fresh input
• Questionnaires
• Reach the masses
• Experts
• Easy way to discover most problems