Portable Haptic Aids for Training and Rehabilitation

1
Portable Haptic Aids for Training and
Rehabilitation

• Li Jiang
• April 4th 2008

2
Presentation Outline

• Background
• Portable haptic aids for emergency personnel
  virtual reality (VR) training
• Portable haptic aids for stroke and multiple
  sclerosis
• Conclusion
• Future work

3
Portable vs Grounded
4
Haptic Devices
Portable
Grounded
Design Tradeoffs Performance Price Portability
..
?
5
Event-Cue vs Amplitude-Based Feedback
Event-Cue Feedback (ECF)
Amplitude-Based Feedback (ABF)
ECF symbolic information
ABF analog information
6
Portable Haptic Aid for Emergency Personnel
Virtual Reality (VR) Training
7
Haptic Devices for Emergency Personnel VR
Training
Immersive training systems withhaptic feedback
have been expensive and encumbering
8
Portable Haptic Aids for VR Emergency Personnel
Training
Unlike motor-skill related VR training, VR
emergency personnel training focuses on high
level abstract skill training such as team
procedures and collaborations in dangerous and
confusing environments.
9
Questions to Answer through First Set of
Experiments

• Can low-cost, portable vibrational feedback
  improve trainees training performance in
  emergency procedures?
• How does vibrational event-cue feedback
  compare with amplitude-based force feedback?

10
VR for Experiment I
Virtual environment generated by Half-Life
videogame engine
Half-life a science fiction first-person
shooter computer game.
11
Experiment I Scenario
The task given to users was to go through a dark,
cluttered and potentially hazardous corridor in
the aftermath of an explosion.
15 random-order obstacles in the corridor, A dim
red light indicates the direction of the exit.
12
Portable Haptic Feedback
13
Experiment Procedure and Metrics
Two Feedback Modes ECF (event cue) and
no-haptic feedback
Speed Trials Memory Trials
Requirement Go through the corridor as fast as possible, avoiding collisions One minute for negotiating the corridor (ample time to pass through the corridor).
Metric Time Number of obstacles recorded (i.e., remembered) Number of obstacles recorded correctly. Time
8 Subjects (5 male, 3 female) Experiment order
was randomized and counterbalanced Subjects
were told that time is not a metric in the memory
trials
14
Experiment Results Memory
Paired T test
ECF Moderecorded
No-Haptic Moderecorded
15
Experiment Results Memory
Paired T test
ECF Moderecordedcorrectly
No-Haptic ModerecordedCorrectly
16
Experiment Results Speed
17
Experiment II Event-Cue Vibration Feedback (ECF)
vs Amplitude-Based Force Feedback (ABF)
Commercial vibration, force joysticks Modified by
Immersion Studio.
VR generated by Half-Life Game enginea dark and
confusing building with rooms to navigate and
clear.
18
Experiment Metrics
While soldiers are clearing the building layout,
they should stop at each door before entering
each rooms to make sure there is no potential
danger in the rooms. Holifield, Leonard.,
Close-Quarter Combat A Soldier's Guide to
Hand-To-Hand Fighting, Paladin Press, May 1997.
Subjects were asked to stop at each door to mimic
this event. A contact between the subjects body
and the wall beside the door was counted as the
completion of the check. The number of times a
subject failed to execute this check is counted
as an error metric.
19
Building Layouts and Feedback Methods
Feedback Methods Amplitude-Based Force Feedback
(ABF) Event-Cue Vibration Feedback (ECF)
No Haptic Feedback (NHF)
Error metric Number of missed safety checks
Typical building layout (layouts were chosen
randomly from a variations on the one shown
above)
20
Experiment II Results
12 subjects, experiment order counterbalanced

• Bonferroni Corrected Paired T test
• F(1,11) 14.86 Plt0.01 (significant)

21
VR Emergency Personnel Training Experiment Results

• Can portable event-cue vibrational feedback
  improve trainees training performance?

Yes, Significantly !

• How does vibrational event-cue feedback compare
  with amplitude-based force feedback in VR
  emergency personnel training scenarios?

Event-cue vibrational feedback can improve
subjects performance as well as amplitude-based
force feedback.
22
Portable Haptic Aids for Stroke and Multiple
Sclerosis
23
Previous Haptic Feedback Devices
The ARCMIME system and MIME system
24
Haptic Feedback Strategy Framework
Force information collected by sensors on the
impaired hand are rendered to the healthy hand
through small vibrational tactors
25
Three Feedback Modes Provided

• NHF mode No-Haptic Feedback
• ABF mode Amplitude-Based Feedback
• ECF mode Event-Cue Feedback

26
Stroke and MS Experiment Goals

• Can vibrational feedback improve patients
  ability in manual force control?
• How does the event-cue feedback (ECF) strategy
  compare with the amplitude-based feedback (ABF)
  strategy?

vs.
27
Comparison of Impairments
Stroke Patients Multiple
Sclerosis Patients
Sensory loss
Sensory loss
Poor Hand Motor Skill
Better Hand Motor Skill Not able to open
impaired hand Able to open impaired hand
Conclusion First, we need a device to help
stroke patients to open their impaired hands
28
Hand-Opening Device Design
Design Goal A passive mechanical system that can
transfer power from healthy hand to the impaired
hand to help the impaired hand open.
Design Process Six design iterations and
prototypes (details in the thesis) Each prototype
was evaluated by stroke patients and therapists.
29
Hand Opening Device Demo
With the help from the healthy side
Without the help from the healthy side
30
Feedback Strategy for Stroke
Impaired hand grasps an instrumented box, with
help of vibration feedback applied to the back of
the opposite hand.
Vibration Tactor
Impaired Hand
Healthy Hand
31
Tactor A Low-Cost, Small, Wearable Vibrational
Feedback Device
Pager motor output (magnitude of vibratory force)
is a nonlinear function of applied voltage or
current. Best results are obtained by creating
a pulse train with varying frequency andduty
cycle.
32
Event-Cue Feedback Mode
Event-Cue Feedback (ECF)
33
Amplitude-Based Feedback Mode
Vibration pattern Period varies Reciprocal to
measured grasp force. Pulse width increases with
the increase of grasp force.
34
ABF mode Perception Test
Based on data of 6 subjects
R2 0.9909 Slope 13.7887
35
Force Sensing Experiment with Stroke Patients
Force Sensor
Instrumented object with force sensors to measure
the grasping forces that subjects provided with
their impaired hand.
36
Pilot Experiment with Stroke Patients

• Three Subjects (2 male, 1 female).
• TaskSubjects were asked to grasp an object and
  try to use minimum force to maintain a stable
  grasp without dropping the object for 10 seconds
  and then replace it.
• Two metrics
• Force Failure rate
• Three feedback modes
• NHF ABF
  ECF

37
ANOVA test p0.0005 Bonferroni corrected
paired T test NHF vs ABF plt0.005, NHF vs ECF
plt0.0002. ABF vs ECF plt0.905
Experiment Data Analysis
No significant results were found in Failure Rate
38
Masku Neurological Rehabilitation Center
39
Feedback Strategy for Multiple Sclerosis
Force Sensorattached to fingerpad
Vibrational tactorattached to fingernail
Impaired Hand
Healthy Hand
40
Force SensorsExperiment with Multiple Sclerosis
patients
41
Experiment Design
GoalDetermine whether portable haptic feedback
can improve patients finger force control
ability. TaskGrab an object and raise it up
from the desk. Subjects were asked to balance
the forces they applied on the index, middle and
ring finger. When they feel the force is
balanced, they should hold the object for 5
seconds and then put it back to the desk.
Feedback ModesNHF Mode, ABF Mode,
ECF Mode
ECF users alerted whenever one finger apply too
much or too less force. ABF vibration pattern
(intensity) is in proportion to the measured force
42
Experiment Procedure
24 Subjects (8 male, 16 female, age range 33 to
64 with a mean of 56.4). The recruited subjects
all have reduced sensation in one hand and good
sensation in the other hand.

• Every user completed the task under all three
  modes in one of the six possible orderings
  (NHF-ABF-ECF, NHF-ECF-ABF, ABF-ECF-NHF, etc).
• Ample time was provided for pre-test practice to
  minimize learning effects.
• Practice trials were provided every time the mode
  was switched.

43
Force Data Analysis
Analysis uses the last 5 seconds of force data
before subjects released the box. (Subjects were
asked to maintain force balance for 5 seconds
before replacing the box)

• Average force Fa (FiFmFr)/3Fi Index
  finger force Fm Middle finger force
  Fr Ring finger force
• The metricsum of the force differences with
  respect to the average forceForce_sum abs(Fi
  Fa)abs(Fm Fa)abs(Fr Fa)

44
Sum of Force Differences for 24 Subjects
Bonferroni corrected PairedT test NHF vs ECF,
p lt NHF vs ABF, p lt ABF vs ECF, p lt 0.27
45
Nine-Hole-Peg Test
Looking for evidence of correlation between
impairment level and most useful type of feedback
Task placing 9 dowels in 9 holes. Subjects
are scored on the amount of time it takes to
place and remove all 9 pegs. It is widely used
in MS clinical trials to quantitatively assess
upper extremity function.
46
Difference in Improvement for ABF vs ECF, with
Respect to NHF Baseline
Difference in Percent Improvement for ABF vs ECF
ABF better
ECF better
Impairment Level (IL)
more impaired
less impaired
47
Sum of Force Difference for More and Less
Impaired Groups
ABF vs ECF p lt 0.00001
ABF vs ECF p lt 0.00001
Less impaired group (10 subjects )
More impaired group (14 subjects)
48
Event-Cue Feedback vs Amplitude-Based Feedback
Mildly impaired patients performed better with
event-cue feedback (ECF), while severely impaired
patients performed better with amplitude-based
feedback (ABF).
49
Failure Rates for the Different Modes
Significance found NHF vs ECF, p lt 0.0006 NHF
vs ABF, p lt 710-8 ABF vs ECF, p lt 0.002
50
Summary of Stroke and MS Studies

• Can portable vibrational feedback (ECF or ABF)
  improve patients ability in grasping force
  control?

Yes! For stroke
15 For multiple sclerosis
60 They both result in reduced failures.

• How does the amplitude-based feedback strategy
  compare with the event-cue feedback strategy?

For mildly impaired patients ECF gt
ABFFor severely impaired patients
ABF gt ECF ABF results in less failures than
ECF overall.
51
Conclusions

• Event-cue, portable vibration feedback can
  significantly improve trainees performance in VR
  training procedures.
• It works as well as proportional force feedback
  for certain training procedures.
• Event-cue and amplitude-based vibrational
  feedback, applied to the opposite hand, can
  significantly improve the ability of stroke and
  multiple sclerosis patients to control their
  grasp forces.
• Mildly impaired MS patients performed better with
  event-cue vibration feedback while severely
  impaired patients performed better with
  amplitude-based feedback mode.
• This suggests that where a feedback channel is
  mostly intact, event cues are helpful to maintain
  performance (e.g. due to distraction of
  fatigue).Conversely, where a feedback channel is
  absent, it is best to provide one.

52
Other Contributions

• Designed a novel passive mechanical hand-opening
  device for stroke patients that can transfer
  power from the healthy hand to impaired hand.
• Developed a framework for feeding force
  information from the impaired hand to the healthy
  hand and proved its validity through experiments
  on stroke and MS patients. Part of the framework
  includes a method for mapping force magnitudes to
  the period and duty cycle of trains of vibration
  pulses.

53
Future Works

• Rehabilitation results have not been not shown in
  this research because of time limitations.
  Further studies could be done to determine
  whether any lasting rehabilitation results after
  long-term use of the devices.
• The concept of putting multiple, small tactors
  with patterns of pulses on the body could be
  extended to other training applications.

54
Acknowledgements
My great thanks to My advisor Professor Mark
Cutkosky Other mentors Professor Larry
Leifer Professor Machiel Van der Loos Professor
Joan Savall My labmates Karlin Bark, Jason
Wheeler, Yonglae Park, Sangbae Kim and all
other BDML labmates
55
Thank you very much!Questions?