User Interface Evaluation - PowerPoint PPT Presentation

1 / 56
About This Presentation
Title:

User Interface Evaluation

Description:

During the task, the user will ask the expert questions about the product ... Neither is an expert. The blind leading the blind. ... – PowerPoint PPT presentation

Number of Views:34
Avg rating:3.0/5.0
Slides: 57
Provided by: juaneg
Category:

less

Transcript and Presenter's Notes

Title: User Interface Evaluation


1
User Interface Evaluation
  • Usability Testing Methods
  • http//jthom.best.vwh.net/usability/
  • http//www.cs.umd.edu/zzj/UsabilityHome.html

2
Usability Testing Methods
  • Caring out experiments to find out specific
    information about a design and/or product.
  • Basis comes from experimental psychology.
  • Uses statistical data methods
  • Quantitative and Qualitative

3
Usability Testing Methods
  • During usability testing, users work on specific
    tasks using the interface/product and evaluators
    use the results to evaluate and modify the
    interface/product.
  • Widely used in practice, but not appropriately
    used.
  • Often abused by developers that consider
    themselves to be usability experts.
  • Can be very expensive and time consuming.

4
Usability Testing Methods
  • Performance Measurement
  • Thinking-aloud Protocol
  • Question-asking Protocol
  • Coaching Method

5
Usability Testing Methods
  • Co-discovery Learning
  • Teaching Method
  • Retrospective Testing
  • Remote Testing

6
Performance Measurement
  • Applicable Stages
  • Design, Code, Test Deployment
  • Personnel
  • Usability Experts, approximately 1.
  • Developers, 0.
  • Users, 6.

7
Performance Measurement
  • Usability Issues Covered
  • Effectiveness Yes
  • Efficiency Yes
  • Satisfaction No
  • Quantitative Data is collected.
  • Can NOT be conducted remotely.
  • Can be used on any system.

8
Performance Measurement
  • What is it?
  • Used to collect quantitative data.
  • Typically, you will be looking for benchmark
    data.
  • Objectives MUST be quantifiable
  • 75 of users shall be able to complete the basic
    task in less than 30 minutes.

9
Performance Measurement
  • How can I do it?
  • Define the goals that you expect users to perform

10
Performance Measurement
  • How can I do it?
  • Quantify the goals
  • The time users take to complete a specific task.
  • The Ratio between successful interactions and
    errors.
  • The time spent recovering from errors.
  • The number of user errors.
  • The number of commands or other features that
    were never used by the user.
  • The number of system features the user can
    remember during a debriefing after the test.
  • The proportion of users who say that they would
    prefer using the system over some specified
    competitor.

11
Performance Measurement
  • How can I do it?
  • Get participants for the experiments
  • Conduct very controlled experiments
  • All variables must remain consistent across users
  • http//www.cs.umd.edu/zzj/PerfMeas.htm

12
Performance Measurement
  • Problems With Performance Measurement
  • No qualitative data.

13
Thinking-aloud Protocol
  • Applicable Stages
  • Design, Code, Test Deployment
  • Personnel
  • Usability Experts, approximately 1.
  • Developers, 0.
  • Users, 4.

14
Thinking-aloud Protocol
  • Usability Issues Covered
  • Effectiveness Yes
  • Efficiency No
  • Satisfaction Yes
  • Quantitative Data is NOT collected.
  • Can NOT be conducted remotely.
  • Can be used on any system.

15
Thinking-aloud Protocol
  • What is it?
  • Technique where the participant is asked to
    vocalize his or her thoughts, feelings, and
    opinions while interacting with the product. .

16
Thinking-aloud Protocol
  • How can I do it?
  • Select the participants, who will be involved?
  • Select the tasks and design scenarios.
  • Ask the participant to perform a task using the
    software.

17
Thinking-aloud Protocol
  • How can I do it?
  • During the task, ask the user to vocalize
  • Thoughts, opinions, feelings, etc.
  • http//www.cs.umd.edu/zzj/ThinkAlo.htm

18
Thinking-aloud Protocol
  • Problem With Thinking-Aloud Protocol
  • Cognitive Overload
  • Can you walk chew gum at the same time?
  • Asking the participants to do too much.

19
Question-asking Protocol
  • Applicable Stages
  • Design, Code, Test Deployment
  • Personnel
  • Usability Experts, approximately 1.
  • Developers, 0.
  • Users, 4.

20
Question-asking Protocol
  • Usability Issues Covered
  • Effectiveness Yes
  • Efficiency No
  • Satisfaction Yes
  • Quantitative Data is NOT collected.
  • Can NOT be conducted remotely.
  • Can be used on any system.

21
Question-asking Protocol
  • What is it?
  • Similar to Thinking-aloud protocol.
  • Instead of participant saying what they are
    thinking, the evaluator prompts the participant
    with questions while using the system.

22
Question-asking Protocol
  • How can I do it?
  • Select the participants, who will be involved?
  • Select the tasks and design scenarios.
  • Ask the participant to perform a task using the
    software.

23
Question-asking Protocol
  • How can I do it?
  • During the task, ask the user to questions about
    the product
  • Thoughts, opinions, feelings, etc.
  • http//www.cs.umd.edu/zzj/Question.htm

24
Question-asking Protocol
  • Problem With Thinking-Aloud Protocol
  • Cognitive Overload
  • Can you walk, chew gum talk at the same time?
  • Asking the participants to do too much.
  • Added pressure when the evaluator asks questions.
  • Can be frustrating on novice users.

25
Coaching Method
  • Applicable Stages
  • Design, Code, Test Deployment
  • Personnel
  • Usability Experts, approximately 1.
  • Developers, 0.
  • Users, 4.

26
Coaching Method
  • Usability Issues Covered
  • Effectiveness Yes
  • Efficiency No
  • Satisfaction Yes
  • Quantitative Data is NOT collected.
  • Can NOT be conducted remotely.
  • Can be used on any system.

27
Coaching Method
  • What is it?
  • A system expert sits with the participant and
    acts as a coach.
  • Expert answers the participants questions.
  • The evaluator observes their interaction.

28
Coaching Method
  • How can I do it?
  • Select the participants, who will be involved?
  • Select the tasks and design scenarios.
  • Ask the participant to perform a task using the
    software in the presence of a coach/expert.

29
Coaching Method
  • How can I do it?
  • During the task, the user will ask the expert
    questions about the product
  • http//www.cs.umd.edu/zzj/Coaching.htm

30
Coaching Method
  • Problem With Coaching Method
  • In reality, there will not be a coach present.
  • This is good for creating a coaching system, but
    not for evaluating the interface.

31
Co-Discovery Learning
  • Applicable Stages
  • Design, Code, Test Deployment
  • Personnel
  • Usability Experts, approximately 1.
  • Developers, 0.
  • Users, 6.

32
Co-Discovery Learning
  • Usability Issues Covered
  • Effectiveness Yes
  • Efficiency No
  • Satisfaction Yes
  • Quantitative Data is NOT collected.
  • Can NOT be conducted remotely.
  • Can be used on any system.

33
Co-Discovery Learning
  • What is it?
  • Two test users attempt to perform tasks together
    while being observed.
  • They are to help each other in the same manner as
    they would if they were working together to
    accomplish a common goal using the product.
  • They are encouraged to explain what they are
    thinking about while working on the tasks.
  • Thinking Aloud, but more natural because of
    partner.

34
Co-Discovery Learning
  • How can I do it?
  • Select the participants, who will be involved?
  • Select the tasks and design scenarios.
  • Ask the participants to perform a task using the
    software.

35
Co-Discovery Learning
  • How can I do it?
  • During the task, the users will help each other
    and voice their thoughts by talking to each
    other.
  • http//www.cs.umd.edu/zzj/Codiscov.htm

36
Co-Discovery Learning
  • Problem With Co-Discovery Learning
  • Neither is an expert
  • The blind leading the blind.

37
Teaching Method
  • Applicable Stages
  • Design, Code, Test Deployment
  • Personnel
  • Usability Experts, approximately 1.
  • Developers, 0.
  • Users, 4.

38
Teaching Method
  • Usability Issues Covered
  • Effectiveness Yes
  • Efficiency No
  • Satisfaction Yes
  • Quantitative Data is NOT collected.
  • Can NOT be conducted remotely.
  • Can be used on any system.

39
Teaching Method
  • What is it?
  • You have 1 participant use the system.
  • Ask the participant to teach a new novice
    participant how to use the system.

40
Teaching Method
  • How can I do it?
  • Select the participants, who will be involved?
  • Select the tasks and design scenarios.
  • Ask the 1st participant to perform a task using
    the software.
  • Ask the 1st participant to teach a new
    participant.

41
Teaching Method
  • How can I do it?
  • Observe their interactions.
  • http//www.cs.umd.edu/zzj/Teaching.htm

42
Teaching Method
  • Problem With Teaching Method
  • Neither is an expert
  • The blind leading the blind.
  • Possible to discover some interesting things
    about the learn-ability of your interfaces.

43
Retrospective Testing
  • Applicable Stages
  • Design, Code, Test Deployment
  • Personnel
  • Usability Experts, approximately 1.
  • Developers, 0.
  • Users, 4.

44
Retrospective Testing
  • Usability Issues Covered
  • Effectiveness Yes
  • Efficiency Yes
  • Satisfaction Yes
  • Quantitative Data can be collected.
  • Can NOT be conducted remotely.
  • Can be used on any system.

45
Retrospective Testing
  • What is it?
  • A videotape of the session is observed by the
    usability expert and the participants.

46
Retrospective Testing
  • How can I do it?
  • Select the participants, who will be involved?
  • Select the tasks and design scenarios.
  • Use one of the usability testing methods that we
    have discussed.
  • Videotape the session.

47
Retrospective Testing
  • How can I do it?
  • Review the videotape with the users.
  • http//www.cs.umd.edu/zzj/Retrospe.htm

48
Retrospective Testing
  • Problem With Retrospective Testing
  • Extremely time consuming!

49
Remote Testing
  • Applicable Stages
  • Design, Code, Test Deployment
  • Personnel
  • Usability Experts, approximately 1.
  • Developers, 0.
  • Users, 5.

50
Remote Testing
  • Usability Issues Covered
  • Effectiveness Yes
  • Efficiency Yes
  • Satisfaction Yes
  • Quantitative Data can be collected.
  • Can be conducted remotely.
  • Can be used on any system.

51
Remote Testing
  • What is it?
  • The participants are separated from the
    evaluators.
  • No formal observation.
  • No usability lab.

52
Remote Testing
  • How can I do it?
  • Give the product/software to participants.
  • Collect information about how they use your
    software/product.
  • Methods
  • Same-Time Different Place
  • Different-Time Different Place

53
Remote Testing
  • How can I do it?
  • Lotus Video Cam, Look_at_Me, SnagIt
  • Usability Logger
  • http//www.usabletools.com/
  • Journaled Sessions
  • http//www.cs.umd.edu/zzj/Retrospe.htm

54
Remote Testing
  • Problem With Remote Testing
  • The evaluator is not there.
  • Cant observe facial expressions.
  • Great for Web based systems.

55
Usability Testing Methods
  • Select the method that works best for you.
  • Select the method that fits your implementation.
  • Be thorough during your experiments.
  • The more data, the better.

56
Usability Testing Methods
  • Hawthorne Effect
  • The tendency for people to improve their
    performance after any change when they know their
    performance is being studied.
Write a Comment
User Comments (0)
About PowerShow.com