Single-trial EEG RSVP classification using convolutional neural networks

Jared Shamwell; Hyungtae Lee; Heesung Kwon; Amar R. Marathe; Vernon Lawhern; William Nothwang

doi:10.1117/12.2224172

Translator Disclaimer

25 May 2016 Single-trial EEG RSVP classification using convolutional neural networks

Jared Shamwell; Hyungtae Lee; Heesung Kwon; Amar R. Marathe; Vernon Lawhern; William Nothwang

Author Affiliations +

Proceedings Volume 9836, Micro- and Nanotechnology Sensors, Systems, and Applications VIII; 983622 (2016) https://doi.org/10.1117/12.2224172
Event: SPIE Defense + Security, 2016, Baltimore, Maryland, United States

Abstract

Traditionally, Brain-Computer Interfaces (BCI) have been explored as a means to return function to paralyzed or otherwise debilitated individuals. An emerging use for BCIs is in human-autonomy sensor fusion where physiological data from healthy subjects is combined with machine-generated information to enhance the capabilities of artificial systems. While human-autonomy fusion of physiological data and computer vision have been shown to improve classification during visual search tasks, to date these approaches have relied on separately trained classification models for each modality. We aim to improve human-autonomy classification performance by developing a single framework that builds codependent models of human electroencephalograph (EEG) and image data to generate fused target estimates. As a first step, we developed a novel convolutional neural network (CNN) architecture and applied it to EEG recordings of subjects classifying target and non-target image presentations during a rapid serial visual presentation (RSVP) image triage task. The low signal-to-noise ratio (SNR) of EEG inherently limits the accuracy of single-trial classification and when combined with the high dimensionality of EEG recordings, extremely large training sets are needed to prevent overfitting and achieve accurate classification from raw EEG data. This paper explores a new deep CNN architecture for generalized multi-class, single-trial EEG classification across subjects. We compare classification performance from the generalized CNN architecture trained across all subjects to the individualized XDAWN, HDCA, and CSP neural classifiers which are trained and tested on single subjects. Preliminary results show that our CNN meets and slightly exceeds the performance of the other classifiers despite being trained across subjects.

Citation Download Citation

Jared Shamwell, Hyungtae Lee, Heesung Kwon, Amar R. Marathe, Vernon Lawhern, and William Nothwang "Single-trial EEG RSVP classification using convolutional neural networks", Proc. SPIE 9836, Micro- and Nanotechnology Sensors, Systems, and Applications VIII, 983622 (25 May 2016); https://doi.org/10.1117/12.2224172