Contrastive-Based Removal of Negative Information in Multimodal Emotion Analysis

Chen C, Li Z, Wan F, Xu L, Bezerianos A, Wang H. Fusing frequency-domain features and brain connectivity features for cross-subject emotion recognition. IEEE Trans Instrum Meas. 2022;71:1–15.

Google Scholar 

Zhu Z, Mao K. Knowledge-based BERT word embedding fine-tuning for emotion recognition. Neurocomputing. 2023;552: 126488. https://doi.org/10.1016/j.neucom.2023.01.067.

Article  Google Scholar 

Li B, Lima D. Facial expression recognition via ResNet-50. Int J Cogn Comput Eng. 2021;2:57–64.

Google Scholar 

Chen C, Li Z, Kou KI, et al. Comprehensive multisource learning network for cross-subject multimodal emotion recognition. IEEE Trans Emerg Top Comput Intell. 2024.

Amiriparian S, Christ L, Kathan A, Gerczuk M, Müller N, Klug S, et al. The MuSe 2024 multimodal sentiment analysis challenge: social perception and humor recognition. In: Proceedings of the 5th on Multimodal Sentiment Analysis Challenge and Workshop: Social Perception and Humor; 2024. p. 1-9.

Pang M, Wang H, Huang J, et al. Multi-scale masked autoencoders for cross-session emotion recognition. IEEE Trans Neural Syst Rehabil Eng. 2024.

El Alaoui D, Riffi J, Sabri A, Aghoutane B, Yahyaouy A, Tairi H. Comparative study of filtering methods for scientific research article recommendations. Big Data Cogn Comput. 2024;8(12):190.

Article  Google Scholar 

Chen C, Li Z, Kou KI, et al. Comprehensive multisource learning network for cross-subject multimodal emotion recognition. IEEE Trans Emerg Top Comput Intell. 2024.

Ragusa E, Gastaldo P, Zunino R, Cambria E. Learning with similarity functions: a tensor-based framework. Cognitive Computation. 2019 Feb;15(11):31–49.

Du K, Zhao Y, Mao R, Xing F, Cambria E. Natural language processing in finance: a survey. Inf Fusion. 2025;115. https://doi.org/10.1016/j.inffus.2024.12.010.

Sailunaz K, Dhaliwal M, Rokne J, Alhajj R. Emotion detection from text and speech: a survey. Soc Netw Anal Min. 2018;8(1):28. https://doi.org/10.1007/s13278-018-0565-0.

Article  Google Scholar 

Dutta A, Biswas S, Das AK. EmoComicNet: a multi-task model for comic emotion recognition. Pattern Recognition. 2024;150. https://doi.org/10.1016/j.patcog.2024.110261.

El Alaoui D, Riffi J, Aghoutane B, Sabri A, Yahyaouy A, Tairi H. Collaborative filtering: comparative study between matrix factorization and neural network method. In: Networked Systems: 8th International Conference, NETYS 2020, Marrakech, Morocco, June 3-5, 2020, Proceedings 8. Springer International Publishing; 2021. p. 361-367.

Pan J, Lu J, Wang S. A multi-stage visual perception approach for image emotion analysis. IEEE Transactions on Affective Computing 2024.

Zhang X, Li M, Lin S, Xu H, Xiao G. Transformer-based multimodal emotional perception for dynamic facial expression recognition in the wild. IEEE Trans Circuits Syst Video Technology 2023.

Srivignesh R, Anand A, et al. Facial expression recognition using convolutional neural network and HAAR classifier. In: Proceedings of the 2023 International Conference on Artificial Intelligence and Knowledge Discovery in Concurrent Engineering (ICECONF); 2023 Jun 12-14; [Location]. New York: IEEE; 2023. p. 1-5.

Mao J, Xu R, Yin X, Chang Y, Nie B, Huang A, Wang Y. Poster++: a simpler and stronger facial expression recognition network. Pattern Recognition 2024;110951. https://doi.org/10.1016/j.patcog.2024.110951.

Issa D, Demirci MF, Yazici A. Speech emotion recognition with deep convolutional neural networks. Biomedical Signal Processing and Control. 2020;59: 101894. https://doi.org/10.1016/j.bspc.2020.101894.

Article  Google Scholar 

Liu Q, Han S, Cambria E, Li Y, Kwok K. PrimeNet: a framework for commonsense knowledge representation and reasoning based on conceptual primitives. Cognitive Computation. 2024;16(6):3429–56.

Article  Google Scholar 

Mocanu B, Tapu R. Speech emotion recognition using GhostVLAD and sentiment metric learning. In: Proceedings of the 12th International Symposium on Image and Signal Processing and Analysis (ISPA); 2021 Sep 1-3; Novi Sad, Serbia. New York: IEEE; 2021. p. 126-130.

Wang R, Zhu J, Wang S, Wang T, Huang J, Zhu X. Multi-modal emotion recognition using tensor decomposition fusion and self-supervised multi-tasking. Int J Multimed Inf Retr. 2024;13(4):39.

Article  Google Scholar 

El Alaoui D, Riffi J, Aghoutane B, Sabri A, Yahyaouy A, Tairi H. Overview of the main recommendation approaches for the scientific articles. In: International Conference on Business Intelligence. Springer International Publishing: May; Cham; 2021. p. 107–18.

Chapter  Google Scholar 

Zadeh AB, Liang PP, Poria S, Cambria E, Morency L-P. Multimodal language analysis in the wild: CMU-MOSEI dataset and interpretable dynamic fusion graph. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); 2018 Jul 15-20; Melbourne, Australia. Stroudsburg (PA): Association for Computational Linguistics; 2018. p. 2236-2246.

Zhu X, Guo C, Feng H, et al. A review of key technologies for emotion analysis using multimodal information. Cogn Comput. 2024;16(4):1504–30.

Article  Google Scholar 

Yu W, Xu H, Meng F, Zhu Y, Ma Y, Wu J, et al. CH-SIMS: a Chinese multimodal sentiment analysis dataset with fine-grained annotation of modality. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics; 2020 Jul 5-10; Online. Stroudsburg (PA): Association for Computational Linguistics; 2020. p. 3718-3727.

Kim J. Multimodal parametric fusion for emotion recognition. Int J Adv Smart Converg. 2020;9(1):193–201.

MathSciNet  Google Scholar 

Tsai Y-HH, Bai S, Liang PP, Kolter JZ, Morency LP, Salakhutdinov R, et al. Multimodal transformer for unaligned multimodal language sequences. In: Proceedings of the Conference. Association for Computational Linguistics. Meeting; 2019 Jul 28-Aug 2; Florence, Italy. Stroudsburg (PA): Association for Computational Linguistics; 2019. p. 6558.

Yang D, Liu Y, Huang C, Li M, Zhao X, Wang Y, et al. Target and source modality co-reinforcement for emotion understanding from asynchronous multimodal sequences. Knowl Based Syst. 2023;265: 110370.

Article  Google Scholar 

El Alaoui D, Riffi J, Sabri A, Aghoutane B, Yahyaouy A, Tairi H. Social recommendation system based on heterogeneous graph attention networks. Int J Data Sci Anal. 2024;1-17.

El Alaoui, D., Riffi, J., Sabri, A., Aghoutane, B., Yahyaouy, A., Tairi, H. (2025). A Novel session-based recommendation system using capsule graph neural network. Neural Networks, 107176.

Hazmoune S, Bougamouza F. Using transformers for multimodal emotion recognition: taxonomies and state of the art review. Eng Appl Artif Intell. 2024;133.

Yadav A, Vishwakarma DK. A deep multi-level attentive network for multimodal sentiment analysis. ACM Trans Multimed Comput Commun Appl. 2023;19(1):1–19.

Article  Google Scholar 

Dashtipour K, Gogate M, Cambria E, Hussain A. A novel context-aware multimodal framework for Persian sentiment analysis. Neurocomputing. 2021;457:377–88.

Article  Google Scholar 

Diwali A, Saeedi K, Dashtipour K, Gogate M, Cambria E, Hussain A, et al. Sentiment analysis meets explainable artificial intelligence: a survey on explainable sentiment analysis. IEEE Trans Affect Comput. 2023.

El Alaoui, D., Riffi, J., Sabri, A., Aghoutane, B., Yahyaouy, A., Tairi, H. (2024). Contextual recommendations: dynamic graph attention networks with edge adaptation. IEEE Access.

Degottex G, Kane J, Drugman T, Raitio T, Scherer S. COVAREP-A collaborative voice analysis repository for speech technologies. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 2014 May 4-9; Florence, Italy. New York: IEEE; 2014. p. 960-4.

Koroteev MV. BERT: a review of applications in natural language processing and understanding. arXiv preprint arXiv:2103.11943; 2021.

Baltrusaitis T, Zadeh A, Lim Y, Morency LP. Openface 2.0: facial behavior analysis toolkit. In: Proceedings of the 13th IEEE International Conference on Automatic Face & Gesture Recognition; 2018.

Wang H, Li X, Ren Z, Wang M, Ma C. Multimodal sentiment analysis representations learning via contrastive learning with condense attention fusion. Sensors. 2023;23(5):2679.

Article  Google Scholar 

Trueman TE, Jayaraman AK, Cambria E, Ananthakrishnan G, Mitra S. An n-gram-based BERT model for sentiment classification using movie reviews. In: Proceedings of the 2022 International Conference on Artificial Intelligence and Data Engineering (AIDE); 2022 Nov 4-6; Tokyo, Japan. IEEE; 2022. p. 41-46.

Zhou R, Li X, Bing L, Cambria E, Miao C. Improving self-training for cross-lingual named entity recognition with contrastive and prototype learning. arXiv preprint arXiv:2305.13628; 2023.

Yu W, Li C, Hu X, Zhu W, Cambria E, Jiang D. Dialogue emotion model based on local-global context encoder and commonsense knowledge fusion attention. Int J Mach Learn Cybern. 2024;1-15. Springer.

Zadeh A, Chen M, Poria S, Cambria E, Morency L-P. Tensor fusion network for multimodal sentiment analysis. arXiv preprint arXiv:1707.07250; 2017.

Liu Z, Shen Y, Lakshminarasimhan VB, Liang PP, Zadeh A, Morency L-P. Efficient low-rank multimodal fusion with modality-specific factors. arXiv preprint arXiv:1806.00064; 2018.

Tsai Y-HH, Liang PP, Zadeh A, Morency L-P, Salakhutdinov R. Learning factorized multimodal representations. arXiv preprint arXiv:1806.06176; 2018.

Tsai Y-HH, Bai S, Liang PP, Kolter JZ, Morency L-P, Salakhutdinov R. Multimodal transformer for unaligned multimodal language sequences. In: Proceedings of the conference. Association for Computational Linguistics. Meeting; 2019; 6558. NIH Public Access.

Sun Z, Sarma P, Sethares W, Liang Y. Learning relationships between text, audio, and video via deep canonical correlation for multimodal language analysis. In: Proceedings of the AAAI conference on artificial intelligence. 2020;34(05):8992-9.

Hazarika D, Zimmermann R, Poria S. Misa: Modality-invariant and-specific representations for multimodal sentiment analysis. In: Proceedings of the 28th ACM international conference on multimedia; 2020. p. 1122-31.

Rahman W, Hasan MK, Lee S, Zadeh A, Mao C, Morency L-P, Hoque E. Integrating multimodal information in large pretrained transformers. In: Proceedings of the conference. Association for Computational Linguistics. Meeting; 2020. p. 2359. NIH Public Access.

Yu W, Xu H, Yuan Z, Wu J. Learning modality-specific representations with self-supervised multi-task learning for multimodal sentiment analysis. In: Proceedings of the AAAI conference on artificial intelligence. 2021;35(12):10790-7.

Han W, Chen H, Poria S. Improving multimodal fusion with hierarchical mutual information maximization for multimodal sentiment analysis. arXiv preprint arXiv:2109.00412; 2021.

Yang J, Yu Y, Niu D, Guo W, Xu Y. Confede: Contrastive feature decomposition for multimodal sentiment analysis. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); 2023. p. 7617-30.

Hazarika D, Li Y, Cheng B, Zhao S, Zimmermann R, Poria S. Analyzing modality robustness in multimodal sentiment analysis. arXiv preprint arXiv:2205.15465. 2022.

Hinton G, Van Der Maaten L. Visualizing data using t-SNE. J Mach Learn Res. 2008;9:2579–605.

Google Scholar 

El Alaoui D, Riffi J, Sabri A, Aghoutane B, Yahyaouy A, Tairi H. Deep GraphSAGE-based recommendation system: jumping knowledge connections with ordinal aggregation network. Neural Comput Appl. 2022;34(14):11679–90.

Article  Google Scholar 

Comments (0)

No login
gif