Research Progress and Application Prospects of Multimodal Brain-Computer Interface Technology
DOI:
https://doi.org/10.61173/3943a276Keywords:
brain-computer interface, multimodal signal fusion, deep learning, human-computer interaction, signal decoding, neuroscience, data processing techniquesAbstract
In the past decades, the rapid development of brain-computer interface (BCI) technology has provided new perspectives on human-computer interaction. Traditional single-modal BCI systems, although showing promising results in many applications, still face challenges in terms of decoding accuracy, real-time performance, and user adaptability due to limitations in signal acquisition. To address these issues, multimodal brain-computer interfaces (MMBCIs) have emerged, aiming to improve the overall system performance by integrating information from different physiological signal sources. This paper reviews the basic concepts, signal sources, signal fusion techniques, application examples, and current challenges and future perspectives of multimodal BCI. By analyzing the existing research literature, we hope to provide theoretical guidance and framework support for further research in this area.References
[1] Adhanom, I. B., MacNeilage, P., & Folmer, E. (2023). Eye Tracking in virtual reality: a broad review of applications and challenges. Virtual Reality, 1-25. Dean&Francis Xin Hong
[2] Belardinelli, A. (2023). Gaze-based intention estimation: principles, methodologies, and applications in HRI. arXiv preprint arXiv:2302.04530.
[3] Choi, SH., Shin, J., Park, C. et al. In vivo magnetogenetics for cell-type-specific targeting and modulation of brain circuits. Nat. Nanotechnol. 19 nat. nanotechnol. 19 , 1333-1343 (2024). https://doi.org/10.1038/s41565-024-01694-2
[4] Duchowski, A. T. (2018). Gaze-based interaction: a 30 year retrospective. Computers & Graphics, 73, 59-69.
[5] Davis, J., Hsieh, Y. H., & Lee, H. C. (2015). Humans perceive flicker artifacts at 500 Hz. scientific reports, 5(1), 7861.
[6] Fazli S, Mehnert J, Steinbrink J, et al. Enhanced performance by a hybrid NIRS-EEG brain computer interface[J]. Neuroimage, 2012, 59(1): 519-529.
[7] Lee, W. J., Kim, J. H., Shin, Y. U., Hwang, S., & Lim, H. W. (2019). Differences in eye movement range based on age and gaze direction.Eye, 33(7), 1145-1151.
[8] Ma T, Li H, Deng L, et al. The hybrid BCI system for movement control by combining motor imagery and moving onset visual evoked potential[J]. Journal of neural engineering, 2017, 14(2): 026015
[9] Ma Teng. Research on key technologies of multimodal braincomputer interface based on mVEP and MI [D]. University of Electronic Science and Technology, 2018.
[10] Melissa R. Warden1,2, Jessica A. Cardin5,6, and Karl Deisseroth2,3, Optical Neural Interfaces (2014) ANNUAL REVIEW OF BIOMEDICAL ENGINEERING Vol 16:103-129
[11] Silva, A.B., Littlejohn, K.T., Liu, J.R. et al. The speech neuroprosthesis. Nat. Rev. Neurosci. 25, 473-492 (2024). https:// doi.org/10.1038/s41583-024-00819-9
[12] Silva, A.B., Littlejohn, K.T., Liu, J.R. et al. The speech neuroprosthesis. Nat. Rev. Neurosci. 25, 473-492 (2024). https:// doi.org/10.1038/s41583-024-00819-9
[13] XIAO X, XIN F, MEI J, et al. A Review of Adaptive Brain- Computer Interface Research[J]. Journal of Electronics and Information, 2023, 45(7): 2386-2394.
[14] Yan HuangJiaxi XuXuehao ZhangEnyan Yu Research progress on vestibular dysfunction and visual-spatial cognition in patients with Alzheimer’s disease Article (Apr 2023) 35(19):15-
[15] Zou J, Chen H, Chen X, Lin Z, Yang Q, Tie C, Wang H, Niu L, Guo Y, Zheng H. Noninvasive closed-loop acoustic brain-computer interface for seizure control Theranostics 2024; 14(15):5965-5981. doi:10.7150/thno.99820. https://www.thno. org/v14p5965.htm
[16] Zhu Guangming. Research on feature extraction and classification method for multimodal brain-machine interface based on EEG and fNIRS[D]. Hangzhou: Hangzhou University of Electronic Science and Technology, 2017.
Downloads
Published
Issue
Section
License
Copyright (c) 2024 by the authors.
This work is licensed under a Creative Commons Attribution 4.0 International License.
