Usability testing

1
Usability testing

• The Think-aloud test

Presented by Connor Stroomberg
2
Two papers

• Jakob Nielsen, 1994
• Estimating the number of subjects needed for
  thinking aloud test
• Van Den Haak et al., 2003
• Retrospective vs. concurrent think-aloud
  protocols testing the usability of a online
  library catalogue

3
Definition

• The think aloud method A simplified thinking
  aloud test involves having a test user operate an
  interface to perform a set of pre-defined tasks
  while being asked to think out loud. By
  Listening in on the users thoughts, the
  experimenter can pinpoint misconceptions and
  other usability problems as they occur in the
  interaction (Nielsen, 1994).

4
Nielsen paper

• A lot of developers know about the think-aloud
  method (21) but few use it (6).
• Why ?
• To complex and expensive
• The need for highly trained specialists
• Special equipment and laboratory
• Nielsen however says
• There is no need for these expensive and complex
  tools and specialists if we are only concerned
  with qualitative results.
• Because the method can be used at various steps
  in de design process qualitative results will
  have a higher impact on the end product.

5
The Nielsen study

• two set of experiments using the same think-aloud
  method.
• The people doing the user testing were two groups
  of students, three hours of training.
• People doing the tests 24 and 30 different
  students.
• Two applications being tested, a commercial word
  processor and a piece of shareware software.

6
Results
7
Results 2
8
Results 3
9
Nielson Conclusion

• The models says that only four to six
  participants are needed.
• This will give 75 of the usability problems.
• Major problems are more likely to occur during
  testing than minor ones.

10
van den Haak et al.

• Is think-aloud really a good measure for
  usability testing ?
• In what way should we conduct a think-aloud test
  ?

11
van den Haak et al.

• Retrospective vs. concurrent think-aloud
• That are the benefits of doing retrospective
  think-aloud ?
• That are the drawbacks ?

12
benefits of retrospective think-aloud

• Decrease in reactivity
• Recording of working times per task
• Possibility for reflection by test subjects
• Cross culture (?)

13
Drawbacks of retrospective think-aloud

• Duration
• Biased accounts
• Maybe
• Equipment ?
• Expertise ?

14
Van der Haak et al. Methode

• Online library catalogue.
• 40 participants.
• 2 groups ? Concurrent vs. retrospective
• Questionnaires

15
Results
16
Results
17
Results
18
Disscusion

• Limited contribution of verbalization in CTA
  (Nielsen does not mention this)
• Enhance and impede task performance
• Interaction between task difficulty, degree of
  verbalization and task performance CTA

19
Conclusion van der Haak

• CTA and RTA are equivalent, but clearly different
  evaluation methods.
• Task difficulties need to be controlled in CTA,
  however this may not always be possible .