Think aloud protocol

1
Think aloud protocol
2

• Think a loud
• It is a research method in which the subject is
  asked to talk aloud, while solving a problem and
  this request is repeated if necessary during the
  problem-solving process thus encouraging the
  subject to tell what he or she is thinking.

3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
Criteria for selecting subjects

• Both subjects and tasks must thus be selected
  that the effect of possible disruptive effects of
  thinking aloud is minimized. Subjects must be
  selected randomly because results must be
  generalized over all persons of this kind. Two
  important properties of subjects with regard to
  the applicability of the think aloud method are
  the degree of expertise and verbalization skills.

8
Selecting problems

• Certain tasks are less suited because they
  involve non-verbal information or because speed
  is inherent in the task for example tasks that
  involve verbal communication (for example, air
  traffic control) are in their original form not
  suited for the think aloud method. The subjects
  should not be able to solve the problem in an
  automated manner. The task should be difficult
  enough. Also within a task area that is suitable
  for example, solving physics problems or
  architectural design.

9
Setting

• The first thing to do when one wants to get a
  subject to think aloud is to make sure that the
  setting is such that the subject feels at ease.
  The subject should be settled comfortably the
  situation must be focused on the task . The
  experimenter should interfere as little as
  possible with the thought process to avoid
  influencing its course.

10
(No Transcript)
11
Instructions

• Instructions about the task at hand should be
  given as customary. The instruction on thinking
  aloud is quite simple. The essence of the
  instruction is performing the task and saying out
  loud what comes to your mind. Sometimes the task
  they have to perform will also ask for a practice
  phase. Then it is efficient to use that phase to
  practice thinking aloud. In general it is wise to
  look for a task which is not too different from
  the target task.

12

• Example of instruction
• 1.Please solve the following problems and while
  you do so, try to say everything that goes
  through your mind.
• 2. I will give you a problem. Please keep
  talking out loud while solving the problem.
• 3.You will in a moment receive a design task. You
  are asked to perform this task in the way you are
  used to go about a commission in your daily
  practice.It is important that you say aloud
  everything that you think or do in designing.

13
Example of practice phase

• A bottle of milk costs 5. The milk costs 4.50
  more than the bottle. How much does the bottle
  cost?

14
Behavior of the experimenter and prompting

• When the subject is working on the task, the role
  of the experimenter is a restrained one.
  Interference should only occur when the subject
  stops talking. Then the experimenter should
  prompt the subject by just saying Keep on
  talking. It is a hard job for the experimenter
  he is inclined to correct the subject when going
  wrong or help the subject along when stuck this
  should be avoided, thus experiencer should avoid
  unnecessary interference.

15
(No Transcript)
16
(No Transcript)
17
Recording

• The session is usually recorded on audio- or
  video-tape. It may be wise to include the
  instruction and the practicing phase, in order to
  be able to check afterward whether the procedure
  was performed correctly.

18
Transcription of the protocol

• After the session has been recorded, it has to be
  transcribed. Typing out complete protocols is
  inevitable to be able to apply reliable coding
  procedures. Transcribing a protocol usually means
  typing it out as verbatim as possible. Typing out
  protocols is a tedious and time-consuming task in
  this case it is possible just to look at
  transcription note down a code every time an
  information source is mentioned, although direct
  coding from audio tape instead of transcribing
  coding the transcription seems attractive in term
  of efficacy it is often not a method to be
  recommended.

19
Review

• Reviewing the protocol with the subject can
  provide very useful additional information.
  Protocols are usually incomplete and difficult to
  interpret and the subject can be very helpful
  here. A good procedure is to review the protocol
  with the subject as soon as possible after the
  actual think aloud session. If representativeness
  is not verified, there is a serious threat to the
  studys internal validity.

20
Ensuring Validity

• In order to ensure that the data collected from
  participants who think aloud is valid it is
  advisable to include a small control group of
  participants who complete the same language tasks
  silently. The rationale behind having a control
  group is that if scores on post task assessments
  are statistically similar in the silent and
  think-aloud groups, it can be concluded that
  verbalization did not have reactive effects and,
  by extension, that thinking aloud did not
  substantially alter participants cognitive
  processes.

21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25

• Before the study
• Decide if think-alouds are appropriate to use
  with the type of language task you plan to use.
• Decide on the type and language of
  verbalization you want to elicit.
• Include an indication that voice samples will
  be recorded in the informed consent document.
• During the study
• In the think-aloud instructions provided to
  participants
• Include a rationale for having participants
  think aloud.
• Provide general instructions to participants
  about how to think aloud.
• Provide learners with a warm-up task to let
  them practice thinking aloud before they move on
  to the experimental task.
• Record think-alouds using appropriate equipment
  to ensure the necessary level of detail.
• To ensure validity
• Verify that participants in think-aloud groups
  continue thinking aloud throughout the task.
  Remind them of this as necessary.
• Include a small control group that performs the
  same tasks without thinking aloud as a check on
  validity.