Paper
27 September 2024 Decoding an intuitive brain-computer interface combining speech and visual imagery using dual-branch network
Kunqing Wang, Ming Zhang, Hui Shen
Author Affiliations +
Proceedings Volume 13284, Third International Conference on Intelligent Mechanical and Human-Computer Interaction Technology (IHCIT 2024); 132841W (2024) https://doi.org/10.1117/12.3049210
Event: Third International Conference on Intelligent Mechanical and Human-Computer Interaction Technology (IHCIT 2024), 2024, Hangzhou, China
Abstract
As a new form of human-computer interaction, brain-computer interface (BCI) has been widely studied. However, most traditional BCI paradigms still face issues such as high cognitive load and easy fatigue. In recent years, intuitive BCI paradigms such as speech imagery (SI) and visual imagery (VI) have gradually become new research hotspots. Unfortunately, the classification performance of standalone SI and VI is still not ideal. In this paper, we designed an intuitive BCI paradigm combining SI and VI (SI+VI). Subjects are asked to simultaneously imagine images and Chinese characters associated with specific drone flight actions. Additionally, a dual-branch network using cross-attention mechanism to fuse time and frequency features (TFCA-DBNet) of electroencephalogram (EEG) signals is proposed. Experimental results show that the proposed TFCA-DBNet achieves a highest accuracy of 82.33% and an average accuracy of 67.33% in the four-class SI+VI task, which is improved by 15.80% and 6.90% compared to SI and VI, respectively. This indicates that the SI+VI paradigm has higher distinguishability than that of SI or VI alone. Compared with other methods, TFCA-DBNet also achieves better classification performance, which proves the effectiveness of the proposed method. Our study provides valuable reference and guidance for future applications of intuitive BCI paradigms.
(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.
Kunqing Wang, Ming Zhang, and Hui Shen "Decoding an intuitive brain-computer interface combining speech and visual imagery using dual-branch network", Proc. SPIE 13284, Third International Conference on Intelligent Mechanical and Human-Computer Interaction Technology (IHCIT 2024), 132841W (27 September 2024); https://doi.org/10.1117/12.3049210
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Electroencephalography

Brain-machine interfaces

Brain

Visualization

Electrodes

Convolution

Education and training

Back to Top