Expert Appraisal

1
Expert Appraisal
2
Types of Expert Appraisal

• Some modifications of the instructional material
  are based on the developers hindsight
• Most should be based on feedback from others. . .

3

• This process is called formative evaluation. .
  .
• That is, evaluation undertaken for the
  improvement of the instruction material during
  its development

4

• Expert appraisal is one stage of formative
  evaluation.
• Based on this feedback the material is modified
  to improve its
• appropriateness,
• effectiveness,
• usability, and
• technical quality

5
(No Transcript)
6
(No Transcript)
7
Guidelines for conducting expert appraisals

• Use a varity of specialists
• Restrict each expert to his/her own field
• Present only relevant parts of the instructional
  material for evaluation by each expert
• Specify the purpose of the review by each expert
  by defining the context in which the material is
  to be used and provide a checklist or
  questionnaire

8
Technical review

• Language
• To check the appropriateness of the language for
  the target trainees and the correctness and
  clarity of it usage
• See example

9

• Media
• To check the technical quality of the finished
  product and the compatibility of the materials
  and equipment
• Reviewer should be knowledgeable in the specific
  medium used--and suggest emphasis upon ways of
  improving the technical quality.

10

• Format review
• Each instruction format has its own standards and
  criteria, for example
• Programmed instruction
• Step size, error rate, etc.
• Simulation games
• Fidelity, replicability, and play time/prep time
  ratio
• Given definition of the context and the purpose
  of the review--appraisal exper should be free to
  use own procedure for the evaluation

11
Instructional review

• Appropriatness
• The extent to which the objectives and the
  content of the instructional material are
  constant with the goals of the training
• See example

12

• Effectiveness
• The extent to which instructional objectives are
  attained by trainees
• See example

13

• Feasibility
• The extent to which the material is usable for
  training
• See example

14
From appraisal to revision

• The developers own hindsight on what might have
  done differently should not be discounted.
• Not all suggestions from all sources are worthy
  of being acted upon.
• Contradictory suggestion from different experts
  are often encounter. (more precise terminology
  vs too much jargon)

15

• Expert appraisal does not reduce the number of
  decisions necessary
• It merely presents different points of view
• Decisions on which suggestions to act on must be
  made and then translated into actual modification
  of the material.
• When a drastic change is proposed--it is a good
  idea to check back w/major appraisal experts
  involved in the original evaluation.