User Interface Evaluation

1
User Interface Evaluation

• Usability Inquiry Methods
• http//jthom.best.vwh.net/usability/
• http//www.cs.umd.edu/zzj/UsabilityHome.html

2
Usability Inquiry Methods

• Usability experts learn about the users likes,
  dislikes, needs, etc. of the system through
• Observation
• Verbal questioning
• Written questioning
• Widely used in practice.
• Different methods have different costs, but in
  general, this is relatively cheap.

3
Usability Inquiry Methods

• Contextual Inquiry
• Field Observation
• Questionnaires
• Interviews
• Focus Groups
• Logging Actual Use

4
Contextual Inquiry

• Contextual Inquiry is also known as proactive
  field studies.
• Applicable Stages
• Requirements Design
• Personnel
• Usability Experts, approximately 1.
• Developers, 0.
• Users, 2.

5
Contextual Inquiry

• Usability Issues Covered
• Effectiveness No
• Efficiency No
• Satisfaction No
• Quantitative Data is NOT collected.
• Can NOT be conducted remotely.
• Can be used on any system.

6
Contextual Inquiry

• What is it?
• Before designing the system, the expert(s) visit
  the users workplace and question them.
• This should occur before any design has been done.

7
Contextual Inquiry

• How can I do it?
• Determine who your users are.
• Go visit them where they work.

8
Contextual Inquiry

• How can I do it?
• Talk to them about the system
• How do they currently do their job?
• How would you like to do your job?
• What do you like about the current system/method?
• What dont you like about the current
  system/method?
• http//jthom.best.vwh.net/usability/context.htm
• http//www.cs.umd.edu/zzj/FieldStu.htm

9
Field Observation

• Applicable Stages
• Test Deployment
• Personnel
• Usability Experts, approximately 1.
• Developers, 0.
• Users, 2.

10
Field Observation

• Usability Issues Covered
• Effectiveness Yes
• Efficiency No
• Satisfaction Yes
• Quantitative Data is NOT collected.
• Can NOT be conducted remotely.
• Can be used on any system.

11
Field Observation

• What is it?
• Usability experts observe users in the field
  using the system/product.

12
Field Observation

• How can I do it?
• Go to the users workplace and simply observe.
• Things to look for
• What is the users mental model?
• Are the users using it the way you expect?
• You dont want them to know you are evaluating
  them.
• http//www.cs.umd.edu/zzj/FieldObs.htm

13
Questionnaires

• Applicable Stages
• Any stage of development (depending on the
  questions)
• Personnel
• Usability Experts, approximately 1.
• Developers, 0.
• Users, 2.

14
Questionnaires

• Usability Issues Covered
• Effectiveness Yes
• Efficiency No
• Satisfaction Yes
• Quantitative Data is NOT collected.
• Can be conducted remotely.
• Can be used on any system.

15
Questionnaires

• What is it?
• Written lists of questions that you distribute to
  your users.

16
Questionnaires

• How can I do it?
• Develop a list of questions on paper, web, email,
  etc. and give the questionnaire(s) to the users.
• The users will answer the questions and return
  the questionnaires to you.
• http//jthom.best.vwh.net/usability/question.htm
• http//www.acm.org/perlman/question.html

17
Interviews

• Applicable Stages
• Any stage of development (depending on the
  questions)
• Personnel
• Usability Experts, approximately 1.
• Developers, 0.
• Users, 1.

18
Interviews

• Usability Issues Covered
• Effectiveness Yes
• Efficiency No
• Satisfaction Yes
• Quantitative Data is NOT collected.
• Can be conducted remotely.
• Can be used on any system.

19
Interviews

• What is it?
• You interview users and ask them questions.

20
Interviews

• How can I do it?
• Develop a list of questions for the users.
• Meet with the users, individually.
• Ask them the questions and log the responses
• Written and/or taped

21
Interviews

• How can I do it?
• Interview Tips
• Clearly define this is an interview.
• Ask open ended questions to get the user talking.
  Yes-No questions are bad.
• Begin with less demanding topics and progress to
  more difficult topics.
• Dont ask questions to support your belief or
  hypothesis.
• Do not answer your own questions.
• Do not agree or disagree remain neutral.

22
Interviews

• How can I do it?
• Probes used to encourage the subjects to
  continue speaking, or to guide their response in
  a particular direction
• Addition Probe
• Encourages more information or clarifies certain
  responses from the test users. Either verbally or
  nonverbally the message is, "Go on, tell me
  more," or "Don't stop."

23
Interviews

• How can I do it?
• Reflecting Probe
• Uses a nondirective technique, encourages the
  test user to give more detailed information. The
  interviewer can reformulate the question or
  synthesize the previous response as a
  proposition.
• Directive Probe
• Specifies the direction in which a continuation
  of the reply should follow without suggesting any
  particular content. A directive probe may take
  the form of "Why is the (the case)?"

24
Interviews

• How can I do it?
• Defining Probe
• Requires the subject to explain the meaning of a
  particular term or concept.
• http//jthom.best.vwh.net/usability/surveys.htm
• http//www.cs.umd.edu/zzj/Interview.htm

25
Focus Groups

• Applicable Stages
• Test Deployment
• Personnel
• Usability Experts, approximately 1.
• Developers, 0.
• Users, 6.

26
Focus Groups

• Usability Issues Covered
• Effectiveness Yes
• Efficiency No
• Satisfaction Yes
• Quantitative Data is NOT collected.
• Can be conducted remotely with email, bulletin
  boards, conference calls, etc.
• Can be used on any system.

27
Focus Groups

• What is it?
• A group of users are gathered to talk about the
  system.
• The expert acts as the moderator.
• Should conduct more than 1 focus group.

28
Focus Groups

• How can I do it?
• Bring a group of users together and begin.
• Collect data
• http//www.cs.umd.edu/zzj/FocusGro.htm

29
Logging Actual Use

• Applicable Stages
• Test Deployment
• Personnel
• Usability Experts, approximately 1.
• Developers, 0.
• Users, 6.

30
Logging Actual Use

• Usability Issues Covered
• Effectiveness Yes
• Efficiency Yes
• Satisfaction Maybe
• Quantitative Data is NOT collected, in most
  cases.
• Can be conducted remotely.
• Can be used on most systems.

31
Logging Actual Use

• What is it?
• The computer automatically collect usage data.
• You could ask the user to log their usage, but
  thats not practical.

32
Logging Actual Use

• How can I do it?
• Usability Logger
• http//www.usabletools.com/
• Automatic capture of keyboard, mouse, etc.
• VideoCam and other products.

33
Logging Actual Use

• Facts On Logging Actual Use
• You know exactly what the user is doing.
• You dont know why, but you do know what, when,
  where.
• You dont know how the user feels.
• http//www.cs.umd.edu/zzj/Logging.htm

34
Usability Inquiry Methodsmost useful

• Contextual Inquiry
• Field Observation
• Questionnaires
• Interviews
• Focus Groups (least useful)
• Logging Actual Use