Protocol Analysis

1
Protocol Analysis

• IE 8541 / HUMF 8001

2
Protocol Analysis

• Protocol record. A record may be in the form
  of a written record, an audio tape, video tape or
  some other form.
• A verbal protocol is simply a record of
  everything that is said.
• Verbal Protocol Analysis is a technique for
  studying complex cognitive tasks like decision
  making, planning and design.

3

• Verbal protocols
• Are particularly useful for observing internal
  cognitive processes that are primarily verbal in
  nature.
• Provides cognitive traces of problem solving.
• Will not provide a complete picture of all
  cognitive actions. Some may not be verbalized.

4
Types of Protocols

• There are many additional types of protocols
  which may be useful in observing problem solving
  and decision making behaviors
• Body movement protocols
• Eye movement protocols
• Key stroke protocols
• Etc.
• A developer must consider what is most
  appropriate for studying that task of interest.

5
Verbal Protocol Analysis Procedure

• A subject is given all materials they would
  typically use to perform a particular type of
  task, this may be as little as paper and pencil,
  or it may include computer tools and special
  equipment.
• The subject is given a problem description.
• The subject is asked to think aloud as they
  read the problem description and solve the
  problem.
• The experimenter should not prompt the subject
  with questions other than ones designed to remind
  the subject to keep talking, like What are you
  thinking?
• Everything the subject says is recorded.

6
Uses of Protocol Analysis in DSS development

• To observe internal verbal cognitive processes,
• To get a big picture idea of how people
  approach problems in a given problem domain,
• To understand at a detail level how people solve
  problems of a given type a protocol provides a
  cognitive trace of problem solving,
• To identify specific problem areas in which
  people might benefit from DSS assistance.
• To provide insights during an evaluation of how
  and why a DSS has changed problem solving
  performance.

7
Example Exercise

• Identify 2 volunteers,
• Have one read problem,
• The other solves the problem,

8
Example Exercise Three Extra Terrestrial Monsters

• Only one globe may be transferred at a time
• If a monster is holding two globes, only the
  larger of the two may be transferred
• A globe may not be transferred to a monster
  holding a larger globe.

9

• 3 Extra Terrestrial Monsters (by Marlene, age 8,
  January, 2007)