High Transfer Rate, Realtime BrainComputer Interface

1
High Transfer Rate, Real-time Brain-Computer
Interface

• Machine-based learning techniques towards a
  practical spelling device for the completely
  paralyzed

2
Agenda

• Brain Computer Interfaces brief intro.
• Our system
• Overview, technical details
• Machine learning Support Vector Machines
• Additional Bandwidth Word Prediction
• Results
• Future Improvements, QA
• Demonstration at Psychology Lab

3
BCIs the Need

• Locked-in patients
• Example J.D. Bauby, The Diving Bell and the
  Butterfly
• Persistence of life butterfly
• Extreme physical disability diving bell

4
BCIs the Need

• Amyotrophic Lateral Sclerosis (ALS), aka Lou
  Gherigs
• Degeneration of motor neurons, paralysis of
  voluntary muscles
• 120,000 diagnosed each year worldwide
• 2000 Canadians live with ALS right now
• Can leave patients locked-in
• Cognitive and sensory functions remain intact

5
BCI(1) Slow Cortical Potentials (SCPs)

• Extensive training 3 months using biofeedback
  mechanism
• Tested on ALS patients, learned to control SCPs

Ref N. Birbaumer et al., The thought
translation device (TTD) for completely paralyzed
patients, IEEE Trans. Rehab. Eng., Vol. 8, pp.
190-193,June 2000.
6
BCI(1) SCPs cont.

• Most successful subject artificially fed and
  respirated for 4 years
• After 3 months of training, wrote letter below
• Took 16 hours to write 2 letters/minute
• Expresses thanks, wants to have a party

7
BCI(2) Implants - Cyberkinetics Inc.

• BrainGate Neural Interface System Mkt. cap
  45mil.
• Control of cursor on PC using implant in motor
  cortex
• Undergoing limited clinical trials
• Limb movement possibilities

8
P300 Spelling Device the P300 Event Related
Potential

• Known as oddball or surprise paradigm
• Inherent

9
P300 Spelling Device the System

• Non-invasive
• Inherent Response

10
P300 Speller Terminology

• Epoch One flash of any row or column
• Trial 1 complete set of epochs - all rows and
  columns
• Symbol Alphanumeric characters or pictures

11
BCI Competition 2003

• Provided pre-collected data for competition
• P300 Spelling Paradigm
• Winners included Kaper et al.
• Used Support Vector Machines
• Achieved high transfer rate with real-time
  implementation possibilities

12
System Operation

• Steps
• Training (approximate 1hr)
• Provide visual stimuli (flashing of rows/columns)
• Record data with known classification label
• Run data through pattern recognition algorithm
  (SVM)
• Create customized models for each individual
• Spelling
• Load customized model for individual
• Provide visual stimuli (flashing of rows/columns)
• Record data with unknown classification label
• Run data through SVM classifier
• Sum up decision values
• Feedback most probable letter

13
Display

• Flexible matrix size
• Flexible matrix contents
• Alphanumeric Characters
• Words
• Symbols

14
Display cont

• Random and exhaustive flashing of all of the rows
  and columns on display
• Flashing cycle 300ms
• 100ms intensification period
• 200ms de-intensification period
• 10 second rest period at the end of each symbol

15
Data Collection

• Collect data from DAQ sampled at 240Hz
• 600ms after intensification
• Buffer overlap
• Flexible data collection delay
• Flexible data recording time

16
Data Collection cont.

• 10 channels collected simultaneously
• Data from each channel concatenated together
• Data stored into program memory
• Collected until end of a symbol
• Converted to array
• Memory cleared for next symbol
• System is timing critical

17
Timing Issue

• Purpose
• Process within 300ms window
• Bottleneck
• Online SVM processing
• Old design 340ms/Epoch
• New design 17.67ms/Epoch
• Requirement
• Pentium4 or equivalent is sufficient

18
Matlab Interface

• Why we use Matlab?
• VBMatlab interface using APIs
• Common functions
• Pass matrix array to Matlab workspace
• Get matrix array from Matlab workspace
• Execute command line or script

19
Support Vector Machines

• Pattern recognition Algorithm
• SVM used for
• Creating models for different individuals (train)
• Getting discriminant scores (spelling)
• Detailed information covered later

20
Score Matrix
21
Word Prediction

• Idea predict intended words based on previous
  spelling. Similar to cellular phone smart text
• Extract top ranked words
• SQL for fast searching
• Dynamic database
• Selection updated on
• the bottom of the display
• Words chosen same way

22
System Design

• Modular Design Approach

23
What is SVM?

• Developed by Vapnik in 1992 at Bell Labs
• Broad applications
• Based on concept of learn from examples
• Key concepts
• Linear Decision Boundary with Margin
• Nonlinear feature transformation

24
Basic Concept

• x1, ..., xn be our training data set
• yi Î 1,-1 be the class label of xi then,
• Find a decision boundary
• Make a decision on disjoint test data

25
Decision Boundary (linear)
Class 1

• Infinite possibility

Class -1
26
Bad Decision Boundary
Class 1
Class 1
Class -1
Class -1
27
Good Decision Boundary

• Want to maximize m
• Boundary found using constrained optimization
  problem

28
Optimization Problem

• Optimization Problem

29
After Training

• xis on the decision boundary are called SUPPORT
  VECTORS
• Support vectors and b defines the decision
  boundary

30
Geometrical Interpretation
31
Non-separable Samples

• Use of Soft Margin Separation
• Kernel Transformation

32
Soft Margin Separation
33
Soft Margin Separation

• Idea simultaneous maximization of margin and
  minimization of training error

34
Nonlinear Samples

• Some Samples are inherently nonlinear in input
  space
• No linear boundary is sufficiently accurate

35
Solution?
36
Kernel Transformation

• Idea map input space into feature space such
  that samples become linearly separable

37
Gaussian Kernel
38
SVM Implementation

• Matlab interface to libsvm
• Kernel RBF with ?? 6.6799e-4
• C parameter 20.007

39
SVM Implementation

• Average Method (61.538)
• Multi-Model Method (65.22)
• Concatenation Method (82.418)
• Weighted Concatenation Method
• (max. 86.264)

40
Possible Improvements

• Weighted concatenation method
• Customized Kernel Parameters

41
Measure of Performance

• Bit Rate
• N number of available symbols
• p prediction accuracy
• t number of seconds taken to choose one symbol
• Letters per minute

42
Cont

• Resulting Transfer Rates
• Without using dictionary
• With using dictionary

43
More Accurate Measure

• Resulting Transfer Rates
• Without using dictionary
• With using dictionary

44
Cont

• Mechanism
• Receives a chosen letter from control module
• Appends the letter to current letters in the word
• Searches SQL database
• Return list of most probable target words based
  on ranking

45
Result Analysis

• Accuracy across subjects
• Accuracy over time, same subject
• Accuracy over number of trials
• Accuracy versus model size

46
Accuracy Across Subjects
47
Accuracy Across Subjects
48
Accuracy Over Time, Same Subject

• Subject Jack

49
Accuracy Over Number of Trials

• Subject Jyh-Liang

50
Accuracy Versus Model Size

• Subject Jyh-Liang

51
Questions?