Strategic Planning for Training Evaluation

1
Strategic Planning for Training Evaluation

• Presented by
• Nita C. Lescher, M.S.W., L.I.C.S.W.
• American Humane Association

2
When to Evaluate?

• Before, During, and After
• Training

3
Why Evaluate?

• Before Training
• Identify needs or problems.
• Identify specific job competencies to close gaps.
• Identify the type of evaluation needed.

4
Why Evaluate?

• During Development of Training
• Determine which staff should be the targets of
  training.
• Provide feedback to curriculum designers on
  content, methods, and materials.
• Determine if course delivery is consistent and
  follows curriculum specifications.

5
Why Evaluate?

• After Training
• Note changes in participants attitudes and
  reactions to the course.
• Assess trainees mastery of knowledge.
• Assess trainees mastery of skills.
• Assess transfer of new knowledge and skills to
  the job.
• Measure whether or not performance gaps have
  begun to close.
• Measure relationship of training to agency goals
  and client outcomes.

6
Strategic Plan for Training Evaluation

• Why do it?
• Specify clear relationships among agency goals,
  resources, purposes of training evaluation and
  decisions about what to evaluate.

7
Factors to Consider

• Purpose of training
• Cost of evaluation
• Centrality of subject matter to competent job
  performance
• Consequences of poorly developed/delivered
  training
• Number of staff to be reached
• Length of course
• Legal mandates, public and political pressures
• Agency goals

8
AHA Training Evaluation Model
9
Apple Peel Levels

• Level 1 Course
• Evaluation of the training
• Content
• Structure
• Methods
• Materials
• Delivery

10
Apple Peel Levels

• Level 2 Satisfaction (Happiness Sheets)
• Assesses trainees feelings about
• Trainer
• Quality of material presented
• Methods of presentation
• Environment (e.g., room temperature)

11
Apple Peel Levels

• Level 3 Opinion
• Assesses trainees attitudes/perceptions about
• Utilization of training
• Own learning
• Trainings value

12
Apple Peel Levels

• Level 4 Knowledge Acquisition
• Assesses learning and recalling terms,
  definitions, and facts
• Level 5 Knowledge Comprehension
• Assesses understanding concepts and
  relationships, recognizing examples in practice,
  and problem solving

13
Apple Peel Levels

• Level 6 Skill Demonstration
• Assesses application of learned material to new
  and concrete situations within the training
  environment
• Level 7 Skill Transfer
• Assesses application of new knowledge and skills
  outside the classroom and on the job

14
Apple Peel Levels

• Level 8 Agency Impact
• Level 9 Client Outcomes
• Level 10 Community Impacts
• Assesses impact of training at agency, client,
  and community levels

15
Model for Successful Training Course Evaluation

• Step 1 Identify the purposes of the evaluation.
• Step 2 Determine the levels of evaluation
  needed.
• Step 3 Develop an evaluation design.
• Step 4 Select or develop instruments.
• Step 5 Develop a data collection plan.
• Step 6 Pilot and revise instruments and
  procedures.
• Step 7 Collect evaluation data.
• Step 8 Analyze evaluation data.
• Step 9 Report results.
• Step 10 Provide feedback/disseminate results.