Augmenting Speech Recognition Systems with Myoelectric Pattern Classification

1
Augmenting Speech Recognition Systems with
Myoelectric Pattern Classification
Dawn MacIsaac, PhD Candidate Institute of
Biomedical Engineering University of New Brunswick
2
Outline

• 20-20 speech V5 Continuous Speech Recognition
  System

• MES Pattern Classification system based on
  wavelet transform features, a PCA dimensionality
  block and a Linear discriminator

• Experimental protocol

- nonrandom and random signal acquisition
- Electrode Placement
- signal segmentation
- signal classification

• Results

- classification errors
- effects of Pre-triggering

• Conclusion

3
Department of National Defense, UK

• Attempting to enhance fighter pilot performance
  and safety by automating cockpit controls with
  speech

- currently, pilots of the Eurojet are in danger
of crashing because of the amount of time they
spend looking at their controls instead of their
flying environment
- using speech recognition systems like the
20-20 speech continuous speech recognition
system, pilots can use words to control some of
the jet instrumentation and displays.
- The problem is that systems like the 20-20
speech system use auditory inputs which can be
severely corrupted by the noise conditions inside
the cockpit of a jet.
- myoelectric signals dont have this problem so
they are an obvious alternative for assisting the
conventional speech recognition system.
4
Advantage of Myoelectric Signals in Speech
Recognition
- Not corrupted by audio noise
- there are similar sounding words with unique
mouth positions implying unique myoelectric
signals
Thus, if the myolectric signals of the mouth can
be sufficiently classified, they can be used to
augment the traditional speech classifier
Is there speech informatiuon in the myolectric
signal?
5
Experimental Protocol

• We collected two different types of data - random
  data and nonrandom data

- 7 subjects of non-random data repeating numbers
1-9 in blocks of 60 seconds with breaks between
- 2 subjects of random data repeating numbers
1-9 randomly in blocks of 60 seconds with breaks
between
- 5 myoelectric signals were collected along with
the acoustic signal

• levator Anguli Oris
• Zygomaticus Major
• Platysma
• Depressor Anguli Oris
• Anterior Belly of the Digastric

6
Electrode Placement in Mask
7
Data Segmentation

• Used the onset of sound as determined by the
  audio signal as the trigger
• full-wave rectification, followed by a moving
  average filter
• window length of 1024 ms
• Pre-triggered because the onset of myo activity
  precedes the onset of the auditory signal

8
Data Classification

• Wavelet transform feature set

9
Data Classification

• Principle Component analysis for Dimensionality
  Reduction

10
Data Classification

• Linear Discrimenant Analysis

11
Data Classification
12
Results
13
Conclusions and Future Work

• The myoelectric signal does contain speech
  information

• Myoelectric signal precedes the acoustic
  information
• Optimal pretrigger 400 500 ms

• In combination with auditory signal, better
  classification accuracy may be possible

14
Acknowledgements

• Supervisors

• Dr. Kevin Englehart

• Dr. Bernard Hudgins

• Dr. Dennis Lovely

• Defense Evaluation Research Agency

• National Sciences and Engineering Research
  Council of Canada