Empirical Investigation of the Learner Centered Grading Debrief

1
Empirical Investigation of the Learner Centered
Grading Debrief

• Elizabeth L. Blickensderfer, Ph.D.
• Justin Jennison
• Embry-Riddle Aeronautical University
• Daytona Beach, FL

2
Graduate Assistants

• Mark Bartosiewicz
• Kern Jennison
• Nidia Doherty

3
Overview

• Background
• Define learner centered debriefing
• Discuss purpose of the study
• Method
• Results
• Discussion

4
Background

• Learner Centered Grading
• Part I Learner self-assessment
• To stimulate thought processes? behaviors
• To develop self-appraisal skills
• Part II Debrief
• Instructor facilitated
• Emphasis on self-critique process
• Component of the FAA/Industry Training Standards
  program (FITS)

5
Purpose

• Isolate the LCG debriefing approach and
  demonstrate empirically the value added.
• Examine the efficacy of using a behavioral
  checklist as the assessment tool.
• Focus Improve task management and single pilot
  resource management related skills low hour
  pilots.

6
Hypothesis

• When using the FITS approach, the individuals who
  are briefed using the LCG debrief strategy will
  demonstrate better performance on task management
  and single pilot resource management related
  skills than will individuals who are debriefed
  using the traditional debriefing strategy.

7
Method

• Participants
• 31 University students 27 male, 4 female
• VFR hours M 86.2, SD 33.4
• IFR hours M 10.2, SD 13. 3
• Apparatus
• Elite G501 iGATE flight simulation device
• Piper Arrow
• Traditional displays

8
Method

• Scenario I (Pre-test)
• Daytona Beach to Tampa family reunion
• Retracting landing gear after take-off
• Vacuum failure
• ATC re-route
• Temporary closing of destination airport (WX)
• Scenario II (Post-test)
• Daytona Beach to Jacksonville superbowl
• Pitot tube blockage
• Closure of destination airport
• Engine failure
• Problem with landing gear

9
Independent Variable

• LCG vs. Traditional debrief
• Traditional
• Instructor provides feedback
• Straightforward rights and wrongs
• LCG
• Participant self-assessment
• Instructor probed participant
• Ask P for his/her views
• Discuss discrepancies
• Focus on concrete examples
• Encourage P to think beyond the scenario to
  future flights
• Positive and negative examples behavior based
  feedback
• Reinforce P for participating in the
  discussion/debrief
• Use structure of scenario to guide the debrief
  chronological

10
Dependent Variables

• Performance metrics
• Behavioral checklists and rating scales
• 2 independent raters consensus meeting
• Categories
• Using passenger as a resource
• Notifying and debriefing the passenger
• Checklist use
• ATC use
• POH and Autopilot use
• Overall effectiveness

11
Manipulation Check

• Participant reactions
• Degree to which Ps perceived the instructor to be
  asking for the learners opinions, encouraging
  the learner to self-assess, encouraging learner
  to consider future flights, etc.
• 10 item, Likert survey combined score

12
Procedure

• Informed consent, demographics
• Practice flight familiarization with device
• Scenario I Preflight
• Scenario I Flight
• Self-Assessment 1 (LCG group only)
• Scenario I Debrief
• Reactions questionnaire
• Break
• Scenario II Preflight
• Scenario II Flight
• Self-Assessment 2 (LCG group only)
• Scenario II debrief
• Reactions questionnaire
• Experiment debrief

Total Time 3.25 3.75 hours
13
Results

• Significant findings
• Manipulation check
• Using passenger as a resource
• Notifying and briefing passenger
• Checklist use

14
Manipulation Check (Reactions survey)
15
Using Passenger as a Resource
16
Notifying and Briefing Passenger
17
Checklist Use
18
Overall Effectiveness
19

• Not significant
• ATC use, POH use, Autopilot use

20
Discussion

• Study examined the value of the LCG debrief.
• Results indicate that the LCG debrief was overall
  more effective than the traditional style of
  debrief
• Likely generalizable beyond DVs used in this
  study
• Why the lack of effect on certain DVs?
• Brevity of study
• Inadequate opportunities to assess
• Future research should examine why LCG helps
  (e.g., metacognition).
• In summary, an effective LCG debrief plays an
  important role in scenario based training.

21
Beth Blickensderfer, Ph.D. Assistant
Professor Human Factors and Systems
Dept. Embry-Riddle Aeronautical
University Daytona Beach, FL blick488_at_erau.edu