A Mixed Reality System for Enabling Collocated After Action Review

1
A Mixed Reality System for Enabling Collocated
After Action Review

• John Quarles
• Samsun Lampotang
• Ira Fischler
• Paul Fishwick
• Benjamin Lok

2
Mixed Reality for Training

• Training Mixed Reality(MR) Improved Training
• i.e Military x, industrialx, medical x
• MR is used during the training session
• After Action Review (AAR)
• Playback and review of ones training session
• i.e. AAR in the HPS
• Currently video reviews
• helpful for bringing trainees to competency
• AAR MR ?

3
After Action Review

• Students need
• Repetition, directed instruction, Feedback
• Educators need
• Assessment Tools
• Video Based Review offers
• Assessment Tools and Repetition
• video playback (FF,REW,etc)
• instruction and feedback
• Expert must be present during student review

4
Video Based AAR

• Widely used
• Limitations
• Fixed, exocentric viewpoint
• Minimal interaction (Play, Pause, etc)
• See only what the camera sees (i.e occlusions are
  problematic)
• Lost information

5
MR Based AAR

• Video review lost information
• Goal maximize information flow with MR
• Advantages of MR systems in general
• Egocentric viewpoint
• interactivity i.e. TUIs
• Collocated virtual information (i.e. gaze)
• Can we enhance AAR with MR?

6
Innovations

• The Augmented Anesthesia Machine Visualization
  and Interactive Debriefing System (AAMVID)
• MR based AAR system for Anesthesia education
• Educator version AAMVID-E
• Student version AAMVID-S
• User evaluation (19 students, 3 educators)
• General goal Evaluate MR for use in AAR

7
Previous Work

• AAR x,x,x
• Recording Virtual Experiences
• x,x,x
• AAR of Virtual Human Experiences
• IPSVizx

8
Anesthesia Education and MR

• The Augmented Anesthesia Machine x
• merges and abstract simulation of an anesthesia
  machine with a real machine (magic lens TUI)
• Improves learning

9
AAR in Anesthesia Education

• Currently Video Based
• Students perform a test (i.e. a fault test)
• Educators and students review and critique
  performance

10
AAR in Anesthesia Education

• MR system AAMVID
• Based on the AAM
• Students (AAMVID-S)
• View their own or an experts interaction in situ
• Follow a directed tutorial guided by the experts
  interaction visualization
• Educators (AAMVID-E)
• Can visualize and filter aggregate student data
• AAMVID addresses video review limitations

11
Student AAMVID-S System
playback controls
Gaze visualization
interaction visualization
12
Student AAMVID-S System

• 3 Visualization Modes
• User Review
• View what the student did
• Expert Review
• View what the expert did
• Expert Tutorial
• Collocates
• experts gaze and interaction event boxes
• Real time abstract simulation of the students
  machine
• Interaction with the real machine while mimicking
  expert
• Hands-on experience in correcting the fault

13
AAMVID-S Evaluation

• Usability and performance evaluation of AAMVID-S
• 19 psychology students

Day 2 (90 min) 1.Testing2. AAR of Tests with
AAMVID
Day 1 (90 min) Training with AAM
14
Day 2 Testing

• Three Fault Tests
• Experimenter caused a problem in the machine
• i.e. disabled a the flow of nitrous oxide
• Participant had to decide
• Whether or not a fault was present
• What the fault was (if present)
• How to correct the fault (if present)

15
Metrics

• Repeated Measures design
• Performance Measures
• Participants were asked questions (each scored
  0-4)
• Fault absent or present
• What was the fault?
• How does the fault affect the patient?
• How to correct the fault?
• Confidence Measures
• Self reported confidence ratings for above
  questions (0-4)

16
Results

• Insert graphs that show significant improvements
  in performance and confidence.

17
Discussion

• Main results
• AAMVID Improved machine fault understanding
• Performance question answers improved.
• AAMVID increased confidence in fault correction
  ability
• Confidence scores increased
• Users prefer Expert Tutorial Mode

18
Educator AAMVID-E System
Interactive Filtering(i.e. on spatial cog
data)
Aggregate Gaze Heat Mapping
19
Educator AAMVID-E System
Interactive Filtering(i.e. on spatial cog
data)
Interaction Graph(markov model of class
interaction)
20
Informal Evaluation

• 3 Experts in anesthesia education
• Dr. Samsun Lampotang
• David Lizdas
• Dr. Nicholas Gravenstein

21
Observations

• Magic lens vs desktop?
• What worked and what didnt work

22
Conclusions

• Presented AAMVID
• Improved student understanding and confidence
• Offered needed assessment abilities to educators
  that were not available before
• MR based AAR
• can offer visualizations and interactivity that
  video based AAR cannot