Immersive Virtual Characters for Educating Medical Communication Skills

1
Immersive Virtual Characters for Educating
Medical Communication Skills

• T. Bernard, C. Oxendine, D. S. Lind, P. Wagner
• Dept of Surgical Oncology (Medical College of
  Georgia)
• K. Johnsen, A. Raij, R. Dickerson, R. Wells, B.
  Lok
• Dept of Comp and Info Science and Eng (College of
  Eng)
• M. Cohen, A. Stevens
• Dept of Surgery (College of Med)
• J. Cendan, M. Duerson, R. Pauly
• Dept of Community Health and Family Med (College
  of Med) R. Ferdig

2
Previous Work Technology for Medical
Communication Skills

• Traditional approaches Bearman 2003
• Narrative
• Problem Solving
• Human Patient Simulator Meurs 1997
• Mechanical simulation
• Motors and actuators for I/O
• JUST VR Ponder 2002
• Immersive approach (Stereo Projection)
• Trains students to react to emergency situations
• Uses human instructor to control system
• Virtual Standardized Patient Hubal 2000
• Commercial Desktop application by RTI
• Speech Recognition
• Natural Language Engine

3
Project Description

• Simulate a standardized patient encounter
• Allow repeated interaction with an Immersive,
  interactive virtual patient in a constrained
  scenario
• Virtual patient, DIANA
• Virtual instructor, VIC
• To address issues with SPs
• Experience diversity
• Quality Control
• Feedback
• Communication skills
• No physical diagnosis
• Interpersonal Simulator

4

• Play Video
• Things to look for interaction modalities

5
System

• Low Cost
• lt 8,000(USD)
• COTSComponents
• Potential
• Every Hospital

6
Natural Interaction Input

• No Keyboard, No Mouse
• Speech Recognition
• Dragon Naturally Speaking 8 Pro
• Accuracy 90 with 10 minutes training
• 70 match to database
• Track Communication Cues
• Non-Verbal
• Track head gaze
• Track left hand
• Track body lean
• Verbal
• Inflection
• Jargon
• Gesture Recognition
• Pointing, handshake

7
Natural Interaction Output

• DIANA and VIC look at user
• Life-size characters
• Animation
• Hand gestures
• Head movement
• Perspective-Correct Rendering
• Why this works
• Does not rely on complete sentences
• Constrained scenario
• Students trained on specific questions

8
Eight Studies

• 2004
• April Project initiated
• August Prototype (n7) UF
• October Experts (n3) UF
• December Pilot Test (n10) UF
• 2005
• June Two Institutions (n16) UF/MCG
• July VP vs SP (n16, n8) UF/MCG
• October Cultural Bias (n16) MCG
• October Class Integration (n33) UF
• n 101
• Testing Centers
• Harrell Center at UF
• Medical College of Georgia
• Diana was in Exam Room 3 (video)

9
Eight Studies
10
VP ? SP

• How is experiencing an interpersonal scenario
  with a virtual person similar to and different
  from experiencing an interpersonal scenario
  with a real person?
• Clearly different
• But in what important ways?
• Patient-doctor interview provides a constrained
  scenario

• The study asks
• Are post-encounter impressions similar?
• Are empathy and other emotions and attitudes
  similarly expressed?
• Which social constructs are followed?
• These questions must be explored to
• Determine the extent to which interpersonal
  scenarios can be simulated with virtual humans
• Identify how component technologies need to
  improve to enable effective interpersonal virtual
  human systems

11
Overall Performance

• Similar Experiences
• Same of participants asked key questions
• Out of 12 6.3 1.7, 5.5 2.1 (a 0.37)
• Same of participants passed the scenario
• SP 50, VP 36
• But
• Some critical items were not asked at the same
  frequency
• Sexual activity (SP 0.88, VP 0.44)
• Nausea (SP0.88, VP0.25)
• Related to virtual patient expressiveness

12
Behavioral Measures

• Empathy
• Empathetic moment Im scared, can you help
  me?
• Expressed the same and same of times (SP
  2.2, VP 1.3)
• But
• Appears less genuine (very robotic)
• Conversation flow is rapid-fire
• Confirmatory phrases statistically different (SP
  20, VP 3.5)

13
Lessons Learned

• Overall experiences similar
• Questions asked
• Global measures
• Education goals met
• Students rated educational merits similarly
• Students rated difficulty similarly
• Global measures of realism do not work
• Battery of specific measures more accurate
• Practice tool in addition to SPs
• Diana rated a 6.36(.85) on a 1 to 10 scale
  (Average is a 7.47 for real SPs)
• VR works out of the lab

14
Papers

• Computer Science
• Raij,et al., Interpersonal Scenarios
  VirtualReal?, IEEE Virtual Reality 2006
• Johnsen, et al., Evolving an Immersive Medical
  Communication Skills Trainer, Journal on
  Presence Teleoperators and Virtual Environments
• Dickerson, et al., Virtual Patients Assessment
  of Synthesized Versus Recorded Speech, Medicine
  Meets Virtual Reality 14
• Johnsen, et al., Experiences in Using Immersive
  Virtual Characters to Educate Medical
  Communication Skills, IEEE Virtual Reality 2005
• Dickerson, et. al., Evaluating a Script-Based
  Approach to Simulating Patient-Doctor
  Interaction, SCS 2005 Intl Conf on HCI Adv for
  Modeling and Sim
• Medicine
• Stevens, et al., The Use of Virtual Patients to
  Teach Medical Students History Taking and
  Communication Skills, American Journal of
  Surgery
• Lind and Lok, The Role of Virtual Patients in
  Medical Education Teaching Tool Versus
  Technological Trend, FOCUS on Surgical Education
• Stevens, et al., Implementing a Virtual Patient
  into the Medical School Curriculum at the
  University of Florida, Southern Group on
  Education Affairs 2006 (pres.)
• Cohen, et al., Do Health Professions Students
  Respond Empathetically to a Virtual Patient?,
  Southern Group on Education Affairs 2006 (pres.)
• Bernard, et al. A Multiinstitutional Pilot Study
  to Evaluate the Use of Virtual Patients to Teach
  Health Professions Students History-Taking and
  Communication Skills, Society of Medical
  Simulation 2006
• Stevens, et al., The Use of Virtual Patients to
  Teach Medical Communication Skills, Assoc of
  Surg Educ 2005, Southern Group on Educational
  Affairs 2005 (pres.)

15
Current Work

• Incorporation into the classroom
• UF Essentials of Patient Care
• n34 students this year
• n135 next year
• MCG Patient Communication
• n200 each year
• Study
• Real Speech vs. Synethsized Speech
• Immersive vs. Non-immersive

16
Do people carry real world biases into the
virtual world?
17
Do people carry real world biases into the
virtual world?
18
The Use of a Virtual Scenario to Teach
Communication Skills for Geriatric Patients
19
Patient Diversity

• Measures
• Eye gaze
• Body Lean
• Vocal Inflection
• Time
• Interruptions

• Do students treat different virtual patients
  similarly?

20
Communication Skills

• Fundamental to clinical practice.
• Patient physician satisfaction.
• Patient understanding adherence.
• Healthcare outcomes.
• Malpractice litigation.
• Learned skill experiential learning alone is
  insufficient.

21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
Virtual Objective Structured Exam(VOSCE)
NBME
25
Curriculum integration
26
Virtual Patient Teams

• T. Bernard, C. Oxendine, D. S. Lind, P. Wagner
• Dept of Surgical Oncology (Medical College of
  Georgia)
• K. Johnsen, A. Raij, R. Dickerson, R. Wells, B.
  Lok
• Dept of Comp and Info Science and Eng (College of
  Eng)
• M. Cohen, A. Stevens
• Dept of Surgery (College of Med)
• J. Cendan, M. Duerson, R. Pauly
• Dept of Community Health and Family Med (College
  of Med) R. Ferdig