Data-collection techniques

1
Data-collection techniques
2
Contents

• Types of data
• Observations
• Event logs
• Questionnaires
• Interview

3
Two types of data
Usability Measures
Use/problem Descriptions
4
Observations

• Observe in real-time or from video recordings
• Can be used both for measures and descriptions

Usability Measures
Use/problem Descriptions
5
http//www.mangold.de/english/intoverview.htm
6
Measures from observations

• Particularly suitable for measuring effectivity
  and efficiency
• How many tasks were performed?
• How quickly was the task performed?
• Were the goals achieved?
• How many errors were made?

Effectivity
Efficiency
7
Use/problem descriptions from observations

• Description of breakdowns
• Describe situations when there is a breakdown
  in the users interaction with the product
• Description of use
• Describe how the user actually used the product

8
Event logs

• Logging of the users interaction with the
  product
• Typically collected automatically
• Can be used both for measures and descriptions

Usability Measures
Use/problem Descriptions
9
http//zing.ncsl.nist.gov/WebTools/WebVIP/overview
.html
10
http//zing.ncsl.nist.gov/WebTools/FLUD/example-pr
etty.html
11
http//zing.ncsl.nist.gov/WebTools/VisVIP/overview
.html
12
http//zing.ncsl.nist.gov/hfweb/proceedings/etgen-
cantor/
13
http//zing.ncsl.nist.gov/hfweb/proceedings/etgen-
cantor/
14
http//zing.ncsl.nist.gov/hfweb/proceedings/etgen-
cantor/
15
http//www.mangold.de/english/patternexample.htm
16
http//zing.ncsl.nist.gov/WebTools/VisVIP/overview
.html
17
Measures from event logs

• Suitable for measuring effectivity and efficiency

Effectivity
Efficiency
18
Use/problem descriptions from event logs

• Visualise interaction patterns
• (Quantify use)

19
Questionnaire

• Self-made or validated
• Most useful for collecting measures

Usability Measures
20
Self-made or validated?

• Designing your own questionnaire allows you to
  address issues of particular interest however,
  it is difficult to create good items
• Validated questionnaires (e.g. SUMI) have
  typically gone through extensive testing. Thus,
  test items are generally good, but they may not
  address the issues you are particularly
  interested in.

21
http//www.acm.org/perlman/question.cgi?formNAU
22
http//www.acm.org/perlman/question.cgi?formASQ
23
http//www.acm.org/perlman/question.cgi?formPUEU
24
Measures from questionnaires

• Can be used for measuring all attributes, but is
  particularly suitable for measuring satisfaction
  we are interested in the users subjective opinion

Satisfaction
25
Interview

• Structured, semi-structured, open
• Most useful for use/problem descriptions

Use/problem Descriptions
26
Use/problem descriptions from interviews

• Information about previous experiences and
  knowledge that might explain the users
  performance
• Obtain explanations
• the users preferences
• Breakdowns
• errors

27
Student tasks

• What data-collection techniques do you think are
  suitable for testing your product?
• What do you believe are the strengths and
  weaknesses of the technique?