Some Mathematical Ideas for Attacking the Brain Computer Interface Problem : Some Mathematical Ideas for Attacking the Brain Computer Interface Problem Michael Kirby
Department of Mathematics
Department of Computer Science
Colorado State University
Overview : Overview The Brain Computer Interface (BCI) Challenge
Signal fraction analysis
Takens’ theorem and classification on manifolds
Nonlinear signal fraction analysis
Conclusions and future work
NSF BCI Group : NSF BCI Group Chuck Anderson (PI), Computer Science, Colorado State
Michael Kirby (Co-PI), Mathematics, Colorado State
James Knight, Ph.D. Student, Colorado State
Tim O’Connor, Ph.D. Student, Colorado State
Ellen Curran, Medical Ethics and Jurisprudence, Dept. of Law, Keele University, Staffordshire, UK
Doug Hundley, Consultant, Department of Mathematics, Whitman
Pattie Davies, Occupational Therapy Department, Colorado State
Bill Gavin, Dept. of Speech, Language and Hearing Sciences, University of Colorado “Geometric Pattern Analysis and Mental Task Design
for a Brain-Computer Interface”
SourceForge https://sourceforge.net/projects/csueeg/ : SourceForge https://sourceforge.net/projects/csueeg/ Development Status: 1 - Planning
Environment: Other Environment
Intended Audience: Science/Research
License: GNU General Public License (GPL)
Natural Language: English
Operating System: Linux, SunOS/Solaris
Topic: Artificial Intelligence, Human Machine Interfaces, Information Analysis, Mathematics, Medical Science Apps.
Chuck Anderson : Chuck Anderson
Pattie Davies : Pattie Davies
BCI Headlines in the News : BCI Headlines in the News
Computers obey brain waves of paralyzed, Associated Press, appearing in MSNBC News, April 6, 2005
Brainwaves Control Video Games, BBC March 2004
Brainwave cap controls computer, BBC December 2004
Brain Could Guide Artificial Limbs
Patients put on thinking caps, Wired News, January 2005
Monkey thoughts control computer, March 2002
Lou Gehrig’s Disease (ALS) : Lou Gehrig’s Disease (ALS) Amyotrophic Lateral Sclerosis (ALS) , or “Locked-In Syndrome”, is an extreme neurological disorder and many patients opt against life support.
Most commonly, the disease strikes people between the ages of 40 and 70, and as many as 30,000 Americans have the disease at any given time. (ALS Association website).
Genetic factors appear to only account for 10 percent of all ALS cases. ALS can strike anyone, anytime.
There are no effective treatments and no cure.
Brain activity appears to remain vigorous while muscle control atrophies degeneritively and completely.
Gulf War Veterans and ALS : Gulf War Veterans and ALS
The following information is from a news release sent out by the Department of Veteran Affairs on December 10, 2001. (ALS Association Web posting.)
“According to a news release on December 10, 2001 from the Department of Veteran Affairs, researchers conducting a large epidemiological study supported by both the Department of Veterans Affairs and the Department of Defense have found preliminary evidence that veterans who served in Desert Shield-Desert Storm are nearly twice as likely as their non-deploying counterparts to develop amyotrophic lateral sclerosis.”
The Brain Computer Interface (BCI) : The Brain Computer Interface (BCI) A means for communication between person and machine via measurements associated with cerebral activity, e.g., EEG, fMRI, MEG.
We assume that no muscle motion is employed such as eye twitching or finger movement.
Low-Cost EEG : Low-Cost EEG
History of EEG : History of EEG Duboi-Reymond (1848) reported the presence of electrical signals
Caton (1875) measured “feeble” currents on the scalp
Berger (1929) measured electrical signals with EEG
1930-50s EEG used in psychiatric and neurological sciences relying on visual inspection of EEG patterns
1960s-70s witness emergence of Quantitative EEG and confirmation of hemispheric specialization, e.g., left brain verbal and right brain spatial.
1980s+ observation of biofeedback
Characteristics of Brainwaves : Characteristics of Brainwaves Delta waves [0,4] Hz associated with sleep. Also empathy.
Theta waves [4, 7.5] associated with reverie, daydreaming, meditation, creative ideas
Alpha waves [7.5,12] prevalent when alert and eyes closed. Associated with relaxed positive feelings.
Beta waves 12Hz+ associated with active state, eyes open.
Reasons Why EEG Should Not Work for BCI : Reasons Why EEG Should Not Work for BCI Electrical activity generated by complex system of billions of neurons
Brain is a “gelatinous mass” suspended in a conducting fluid
Difficult to “register” electrode location
Artifacts from motion, eyeblinks, swallows, heartbeat, sweating…
Food, age, time of day, fatigue, motivation of subject
Why EEG Can Work for BCI : Why EEG Can Work for BCI Many EEG studies have reported reproducible changes in brain dynamics that are task dependent!
People are able to control their brainwaves via biofeedback!
Biofeedback : Biofeedback Patients may “correct” their waveforms to achieve a normal state.
Kamiya demonstrated the controllability of alpha waves in 1962.
Communication in morse code by turning alpha waves on and off.
Stress management and sleep therapy.
Move a pac-man by stimulating alpha and beta waves.
Note that artifacts are a serious problem for real-time biofeedback applications.
Motivation for Our Work : Motivation for Our Work Current biofeedback training requires 10 weeks to move a cursor.
Typing requires 5 minutes/letter with 90% accuracy.
Although there has been some mathematical work the field has been dominated by experiment and heuristics.
Suggestions by clinical EEG experts that understanding EEG problem will have a strong mathematical component.
Tremendous potential for results.
EEG Data Set: Mental Tasks : EEG Data Set: Mental Tasks Resting task
Imagined letter writing
Mental multiplication
Visualized counting
Geometric object rotation
Keirn and Aunon, “A new mode of communication between man and his surroundings”, IEEE Transactions on Biomedical Engineering, 37(12):1209-1214, December 1990
Lobes of the Brain : Lobes of the Brain Frontal Lobes
Personality, emotions, problem solving.
Parietal lobes
Cognition, spatial relationships and mathematical abilities, nonverbal memory.
Occipital lobes
Vision, color, shape and movement.
Temporal lobes
Speech and auditory processing, language comprehension, long-term memory.
Electrode Placement and Sample Data : Electrode Placement and Sample Data
Geometric Filtering of Noisy Time-Series : Geometric Filtering of Noisy Time-Series
Given a data set
The Q fraction of a basis vector is defined as where
Signal Fraction Optimization : Signal Fraction Optimization Determine such that D() is a maximum.
Solution via the GSVD equation
Slide24 : Original Signal SVD filter Signal fraction filter
Slide25 : SVD basis GSVD basis
Slide26 : SVD reconstruction GSVD reconstruction
Blind Signal Separation : Blind Signal Separation Unknown (tall) m £ n signal matrix S
Unknown mixing n £ n matrix A
Observed m £ n data matrix X
Task: recover A and S from X alone.
In general it is not possible to solve this problem.
Signal Fraction Analysis Separation : Signal Fraction Analysis Separation Theorem: The solution to the signal fraction analysis optimization problem solves the signal separation problem X = SA given
1) is observed
2)
3)
In particular,
Where is the solution to the GSVD problem for signal fraction analysis.
Slide29 : Original signals (unknown) Mixed signals (observed)
Slide30 : FastICA separation Signal fraction separation
Artifact Removal : Artifact Removal Given the separated signals = X we may filter the ith column of by setting
Where Id’ is the identity matrix with the ith row set to zero. The filtered version of the data is now
Where recall the original data is
Signal Fraction Filters : Signal Fraction Filters
Constructing Signal Fraction Filter : Constructing Signal Fraction Filter
Benefits of Signal Fraction Analysis : Benefits of Signal Fraction Analysis Can identity sources of noise such as respirators, eyeblinks, cranial heartbeat, line noise etc…
Filtering works over short periods of the signal, i.e., can remove artifacts from a time series of length 500.
Can use generalizations of the signal to noise ratio to separate quantities of interest.
Simple and fast to compute.
Classification on Manifolds : Classification on Manifolds Insert slide from Istec meeting manifold: H(x) = 0 dist(A,B) large but H(A)=H(B)=0
Dynamical Systems Perspective : Dynamical Systems Perspective Assume a system is described by the dynamical equations
and that the solutions reside on an attracting set, e.g., a manifold. What can be said about the full system if it is only possible to observe part of the system? In the extreme, imagine we can only observe a scalar value
Time Delay Embedding : Time Delay Embedding We may embed the scalar observable into a higher dimensional state space via the construction
So now it is clear that
Taken’s Theorem (simplified) : Taken’s Theorem (simplified) Given a continuous time dynamical system with solution on a compact invariant smooth manifold M of dimension d, a continuous measurement function h(x(t)) can be time-delay embedded in to dimension 2d+1 such that there is a diffeomorphism between the embedded attractor and the actual (unobserved) solution set.
The Lorenz Attractor : The Lorenz Attractor Given a data point (x,y,z) we know which lobe by the sgn of x. But what if we only observe the z value? The lobe can be classified using Taken’s theorem and Time delay embedding.
Do EEG data lie on an attractor? : Do EEG data lie on an attractor?
Elephants in the Clouds? : Elephants in the Clouds? Classification rate Random data
Super Resolution Skull Caps : Super Resolution Skull Caps How many electrodes are needed? 6, 16, 32, 128, 256, 512? We should be able to answer this question by means of evaluating an objective function.
Through attractor reconstruction, time delay embedding techniques may practically enhance the resolution of skull caps leading to significant savings in time and equipment.
Colleagues working on EEG studies in children are very enthusiastic about this!
Manifolds and Nonlinear Methods (work with Fatemeh Emdad) : Manifolds and Nonlinear Methods (work with Fatemeh Emdad) Veronese embeddings of the data:
Degree 1: (x,y)
Degree 2: (x2, xy, y2)
Degree 3: (x3, x2y, xy2, y3)
Degree 1: (x,y,z)
Degree 2: (x^2, xy, xz, y^2, yz, z^2)
Degree 3: (x^3, x^2y, x^2z, xy^2, xz^2, xyz, y^3, y^2z, yz^2, z^3)
Such embeddings are behind one variant of kernel SVD.
Kernel SVD versus Kernel SFA : Kernel SVD versus Kernel SFA Numerical Experiments:
KSVD (KPCA) degree = 1, 2, 3, 4
KSFA degree = 1, 2, 3, 4
Objective: compare mode classification rates using knn for k = 1,…, 10.
KSFA, KPCA degree 1 : KSFA, KPCA degree 1
KSFA, KPCA degree 2 : KSFA, KPCA degree 2
KSFA, KPCA degree 3 : KSFA, KPCA degree 3
KSFA, KPCA degree 4 : KSFA, KPCA degree 4
Relative Performance : Relative Performance
Conclusions and Future Work : Conclusions and Future Work Present a geometric subspace approach for signal separation, artifact removal and classification.
Provided evidence that brain dynamics might reside on an attractor and that time-delay embedding enhances classification rates.
Illustrated a nonlinear extension to signal fraction analysis and compared with similar extension to svd.
These ideas are presented in the context of EEG signals but are quite general and can be applied to images.