MBRSTCformer: a knowledge embedded local–global spatiotemporal transformer for emotion recognition

Abibullaev B, Keutayeva A, Zollanvari A(2023) Deep learning in EEG-based BCIS: a comprehensive review of transformer models, advantages, challenges, and applications. IEEE Access 11:127271–127301

Chen T, Sihang J, Yuan X, Elhoseny M, Ren F, Fan M, Chen Z (2018) Emotion recognition using empirical mode decomposition and approximation entropy. Comput Electr Eng 72:383–392

Article  Google Scholar 

Chen C, Li Z, Wan F, Leicai X, Bezerianos A, Wang H (2022a) Fusing frequency-domain features and brain connectivity features for cross-subject emotion recognition. IEEE Trans Instrum Meas 71:1–15

CAS  Google Scholar 

Chen C, Vong C-M, Wang S, Wang H, Pang M (2022b) Easy domain adaptation for cross-subject multi-view emotion recognition. Knowl Based Syst 239:107982

Article  Google Scholar 

Chen C, Li Z, Kou KI, Du J, Li C, Wang H, Vong CM (2024a) Comprehensive multisource learning network for cross-subject multimodal emotion recognition. IEEE Trans Emerg Top Comput Intell 9:365–380

Chen Y, Xu X, Bian X, Qin X (2024b) EEG emotion recognition based on ordinary differential equation graph convolutional networks and dynamic time wrapping. Appl Soft Comput 152:111181

Article  Google Scholar 

Cheng C, Liu W, Fan Z, Feng L, Jia Z (2024) A novel transformer autoencoder for multi-modal emotion recognition with incomplete data. Neural Netw 172:106111

Article  PubMed  Google Scholar 

Ding Y, Robinson N, Tong C, Zeng Q, Guan C (2023) LGGNET: learning from local–global–graph representations for brain–computer interface. IEEE Trans Neural Netw Learn Syst 35:9773–9786

Ding Y, Zhang S, Tang C, Guan C (2024) MASA-TCN: multi-anchor space-aware temporal convolutional neural networks for continuous and discrete EEG emotion recognition. IEEE J Biomed Health Inform 28:3953–3964

Du G, Su J, Zhang L, Su K, Wang X, Teng S, Liu PX (2022) A multi-dimensional graph convolution network for EEG emotion recognition. IEEE Trans Instrum Meas 71:1–11

Google Scholar 

Duan RN, Zhu JY, Lu BL (2013) Differential entropy feature for EEG-based emotion classification. In: 6th IEEE/EMBS international conference on neural engineering (NER), pp 81–84. IEEE

Fan C, Wang J, Huang W, Yang X, Pei G, Li T, Lv Z (2024) Light-weight residual convolution-based capsule network for EEG emotion recognition. Adv Eng Inform 61:102522

Article  Google Scholar 

Feng L, Cheng C, Zhao M, Deng H, Zhang Y (2022) EEG-based emotion recognition using spatial–temporal graph convolutional LSTM with attention mechanism. IEEE J Biomed Health Inform 26(11):5406–5417

Article  PubMed  Google Scholar 

Garg D, Verma GK, Singh AK (2024) EEG-based emotion recognition using mobilenet recurrent neural network with time-frequency features. Appl Soft Comput 154:111338

Article  Google Scholar 

Gong L, Li M, Zhang T, Chen W (2023) EEG emotion recognition using attention-based convolutional transformer neural network. Biomed Signal Process Control 84:104835

Article  Google Scholar 

Guo W, Wang Y (2024) Convolutional gated recurrent unit-driven multidimensional dynamic graph neural network for subject-independent emotion recognition. Expert Syst Appl 238:121889

Article  Google Scholar 

Guo W, Li Y, Liu M, Ma R, Wang Y (2024) Functional connectivity-enhanced feature-grouped attention network for cross-subject EEG emotion recognition. Knowl Based Syst 283:111199

Article  Google Scholar 

Hazmoune S, Bougamouza F (2024) Using transformers for multimodal emotion recognition: taxonomies and state of the art review. Eng Appl Artif Intell 133:108339

Article  Google Scholar 

Kanuboyina SNV, Penmetsa RRV (2023) Electroencephalograph based human emotion recognition using artificial neural network and principal component analysis. IETE J Res 69(3):1200–1209

Article  Google Scholar 

Lawhern VJ, Solon AJ, Waytowich NR, Gordon SM, Hung CP, Lance BJ (2018) EEGNET: a compact convolutional neural network for EEG-based brain–computer interfaces. J Neural Eng 15(5):056013

Article  PubMed  Google Scholar 

Li X, Song D, Zhang P, Yu G, Hou Y, Hu B (2016) Emotion recognition from multi-channel EEG data through convolutional recurrent neural network. In: IEEE international conference on bioinformatics and biomedicine (BIBM), pp 352–359. IEEE

Li X, Zhang Y, Tiwari P, Song D, Bin H, Yang M, Zhao Z, Kumar N, Marttinen P (2022) EEG based emotion recognition: a tutorial and review. ACM Comput Surv 55(4):1–57

Article  Google Scholar 

Li C, Li P, Zhang Y, Li N, Si Y, Li F, Cao Z, Chen H, Chen B, Yao D et al (2023a) Effective emotion recognition by learning discriminative graph topologies in EEG brain networks. IEEE Trans Neural Netw Learn Syst 35:10258–10272

Li D, Xie L, Wang Z, Yang H (2023b) Brain emotion perception inspired EEG emotion recognition with deep reinforcement learning. IEEE Trans Neural Netw Learn Syst 35:12979–12992

Li R, Ren C, Ge Y, Zhao Q, Yang Y, Shi Y, Zhang X, Bin H (2023c) MTLFUSENET: a novel emotion recognition model based on deep latent feature fusion of EEG signals and multi-task learning. Knowl Based Syst 276:110756

Article  Google Scholar 

Li C, Bian N, Zhao Z, Wang H, Schuller BW (2024a) Multi-view domain-adaptive representation learning for EEG-based emotion recognition. Inf Fusion 104:102156

Article  Google Scholar 

Li C, Wang F, Zhao Z, Wang H, Schuller BW (2024b) Attention-based temporal graph representation learning for EEG-based emotion recognition. IEEE J Biomed Health Inform 28:5755–5767

Lindquist KA, Wager TD, Kober H, Bliss-Moreau E, Barrett LF (2012) The brain basis of emotion: a meta-analytic review. Behav Brain Sci 35(3):121–143

Article  PubMed  PubMed Central  Google Scholar 

Liu S, Wang X, Zhao L, Li B, Hu W, Yu J, Zhang Y-D (2021) 3DCANN: a spatio-temporal convolution attention neural network for EEG emotion recognition. IEEE J Biomed Health Inform 26(11):5321–5331

Article  Google Scholar 

Liu B, Guo J, Chen C, Wu X, Zhang T (2023a) Fine-grained interpretability for EEG emotion recognition: Concat-aided grad-CAM and systematic brain functional network. IEEE Trans Affect Comput 15(2):671–684

Article  Google Scholar 

Liu S, Wang Z, An Y, Zhao J, Zhao Y, Zhang Y-D (2023b) EEG emotion recognition based on the attention mechanism and pre-trained convolution capsule network. Knowl Based Syst 265:110372

Article  Google Scholar 

Liu S, Zhao Y, An Y, Zhao J, Wang S-H, Yan J (2023c) GLFANet: a global to local feature aggregation network for EEG emotion recognition. Biomed Signal Process Control 85:104799

Article  Google Scholar 

Liu S, Wang Z, An Y, Li B, Wang X, Zhang Y (2024) DA-CapsNet: a multi-branch capsule network based on adversarial domain adaption for cross-subject EEG emotion recognition. Knowl Based Syst 283:111137

Article  Google Scholar 

Pang M, Wang H, Huang J, Vong CM, Zeng Z, Chen C (2024) Multi-scale masked autoencoders for cross-session emotion recognition. IEEE Trans Neural Syst Rehabil Eng 32:1637–1646

Pan B, Hirota K, Jia Z, Dai Y (2023) A review of multimodal emotion recognition from datasets, preprocessing, features, and fusion methods. Neurocomputing 561:126866

Peng Y, Jin F, Kong W, Nie F, Bao-Liang L, Cichocki A (2022) OGSSL: a semi-supervised classification model coupled with optimal graph learning for EEG emotion recognition. IEEE Trans Neural Syst Rehabil Eng 30:1288–1297

Article  PubMed  Google Scholar 

Peng G, Zhang H, Zhao K, Mengting H (2024) Spectrum-based channel attention cooperating with time continuity encoding in transformer for EEG emotion analysis. Biomed Signal Process Control 90:105863

Article  Google Scholar 

Pessoa L (2008) On the relationship between emotion and cognition. Nat Rev Neurosci 9(2):148–158

Article  CAS  PubMed  Google Scholar 

Poldrack RA, Farah MJ (2015) Progress and challenges in probing the human brain. Nature 526(7573):371–379

Article  CAS  PubMed  Google Scholar 

Prabowo DW, Nugroho HA, Setiawan NA, Debayle J (2023) A systematic literature review of emotion recognition using EEG signals. Cogn Syst Res 82:101152

Ramzan M, Dawn S (2023) Fused CNN–LSTM deep learning emotion recognition model using electroencephalography signals. Int J Neurosci 133(6):587–597

Article  PubMed  Google Scholar 

Rubinov M, Sporns O (2010) Complex network measures of brain connectivity: uses and interpretations. Neuroimage 52(3):1059–1069

Article  PubMed  Google Scholar 

Salankar N, Mishra P, Garg L (2021) Emotion recognition from EEG signals using empirical mode decomposition and second-order difference plot. Biomed Signal Process Control 65:102389

Article  Google Scholar 

Song T, Zheng W, Song P, Cui Z (2018) EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans Affect Comput 11(3):532–541

Article  Google Scholar 

Tarailis P, Koenig T, Michel CM, Griškova-Bulanova I (2024) The functional aspects of resting EEG microstates: a systematic review. Brain Topogr 37(2):181–217

Article  PubMed  Google Scholar 

Wang H, Linfeng X, Bezerianos A, Chen C, Zhang Z (2020) Linking attention-based multiscale CNN with dynamical GCN for driving fatigue detection. IEEE Trans Instrum Meas 70:1–11

Article  Google Scholar 

Wang Y, Qingfeng W, Wang S, Fang XQ, Ruan Q (2024) MI-EEG: generalized model based on mutual information for EEG emotion recognition without adversarial training. Expert Syst Appl 244:122777

Article  Google Scholar 

Wei L, Tan T-P, Ma H (2023) Bi-branch vision transformer network for EEG emotion recognition. IEEE Access 11:36233–36243

Article  Google Scholar 

Xu F, Pan D, Zheng H, Yu O, Jia Z, Zeng H (2024) EESCN: a novel spiking neural network method for EEG-based emotion recognition. Comput Methods Programs Biomed 243:107927

Article  PubMed  Google Scholar 

Yan H, Guo K, Xing X, Xu X (2024) Bridge graph attention based graph convolution network with multi-scale transformer for EEG emotion recognition. IEEE Trans Affect Comput 15:2042–2054

Yang Y, Wu Q, Qiu M, Wang Y, Chen X (2018) Emotion recognition from multi-channel EEG through parallel convolutional recurrent neural network. In: International joint conference on neural networks (IJCNN), pp 1–7. IEEE

Ying M, Shao X, Zhu J, Zhao Q, Li X, Bin H (2024) EDT: an EEG-based attention model for feature learning and depression recognition. Biomed Signal Process Control 93:106182

Article  Google Scholar 

Zhang X, Yao L (2021) Deep learning for EEG-based brain-computer interfaces: representations, algorithms and applications. World Scientific, Singapore

Book 

Comments (0)

No login
gif