A Novel Multi-modality Deepfake Detection Using Cross-Model Fusion–Based Hybrid Harris Whale Depth-Wise Separable Dense Convolutional Bi-GRU Deepfake Model

Rana MS, Nobi MN, Murali B, Sung AH. Deepfake detection: a systematic literature review. IEEE Access. 2022;10:25494–513.

Article  Google Scholar 

Das S, Seferbekov S, Datta A, Islam MS, Amin MR. Towards solving the deepfake problem: an analysis on improving deepfake detection using dynamic face augmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision; 2021. pp. 3776–3785.

Kumar M, Sharma HK. A GAN-based model of deepfake detection in social media. Procedia Computer Science. 2023;218:2153–62.

Article  Google Scholar 

Wang T, Chow KP. Noise based deepfake detection via multi-head relative-interaction. In: Proceedings of the AAAI Conference on Artificial Intelligence 2023 Jun 26 (Vol. 37, No. 12, pp. 14548–14556).

Dong S, Wang J, Ji R, Liang J, Fan H, Ge Z. Implicit identity leakage: the stumbling block to improving deepfake detection generalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2023. pp. 3994–4004.

Xu Y, Raja K, Verdoliva L, Pedersen M. Learning pairwise interaction for generalizable deepfake detection. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision; 2023. pp. 672–682.

Yang W, Zhou X, Chen Z, Guo B, Ba Z, Xia Z, Cao X, Ren K. Avoid-df: Audio-visual joint learning for detecting deepfake. IEEE Trans Inf Forensics Secur. 2023;18:2015–29.

Article  Google Scholar 

Cozzolino D, Pianese A, Nießner M, Verdoliva L. Audio-visual person-of-interest deepfake detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; 2023. pp. 943–952.

Tian Y, Chen Y, Tang Y, Fu B. Deepfake algorithm recognition through multi-model fusion based on manifold measure. In: Proceedings of IJCAI 2023 Workshop on Deepfake Audio Detection and Analysis. 2023.

Elpeltagy M, Ismail A, Zaki MS, Eldahshan K. A novel smart deepfake video detection system. Int J Adv Comput Sci Appl. 2023;14(1):407–419.

Zhang Y, Lin W, Xu J. Joint audio-visual attention with contrastive learning for more general deepfake detection. ACM Trans Multimed Comput Commun Appl. 2024;20(5):1–23.

Lewis JK, Toubal IE, Chen H, Sandesera V, Lomnitz M, Hampel-Arias Z, Prasad C, Palaniappan K. Deepfake video detection based on spatial, spectral, and temporal inconsistencies using multi-modal deep learning. In: 2020 IEEE Applied Imagery Pattern Recognition Workshop (AIPR); IEEE; 2020. pp. 1–9.

Raza MA, Malik KM. Multimodaltrace: deepfake detection using audio-visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023. pp. 993–1000.

Mittal T, Bhattacharya U, Chandra R, Bera A, Manocha D. Emotions don’t lie: an audio-visual deepfake detection method using affective cues. In: Proceedings of the 28th ACM International Conference on Multimedia. 2020. pp. 2823–2832.

Khalid H, Kim M, Tariq S, Woo SS. Evaluation of an audio-video multi-modal deepfake dataset using unimodal and multi-modal detectors. In: Proceedings of the 1st workshop on synthetic multimedia-audiovisual deepfake generation and detection. 2021; 7–15.

Ilyas H, Javed A, Malik KM. AVFakeNet: A unified end-to-end Dense Swin Transformer deep learning model for audio–visual deepfakes detection. Appl Soft Comput. 2023;136: 110124.

Article  Google Scholar 

Wang R, Ye D, Tang L, Zhang Y, Deng J. AVT2-DWF: improving deepfake detection with audio-visual fusion and dynamic weighting strategies. arXiv preprint arXiv:2403.14974. 2024.

Gu Y, Zhao X, Gong C, Yi X. Deepfake video detection using audio-visual consistency. InDigital Forensics and Watermarking: 19th International Workshop, IWDW 2020, Melbourne, VIC, Australia, November 25–27, 2020, Revised Selected Papers 19 2021 (pp. 168–180). Springer International Publishing.

Awotunde JB, Jimoh RG, Imoize AL, Abdulrazaq AT, Li CT, Lee CC. An enhanced deep learning-based deepfake video detection and classification system. Electronics. 2022;12(1): 87.

Article  Google Scholar 

Suratkar S, Kazi F. Deep fake video detection using transfer learning approach. Arab J Sci Eng. 2023;48(8):9727–37.

Article  Google Scholar 

Bos H, Muir D. Sub-mW Neuromorphic SNN audio processing applications with Rockpool and Xylo. In: Embedded artificial intelligence. River Publishers; 2023. pp. 69–78.

Andono PN, Shidik GF, Prabowo DP, Yanuarsari DH, Sari Y, Pramunendar RA. Feature selection on gammatone cepstral coefficients for bird voice classification using particle swarm optimization. Int J Intell Eng Syst. 2023;16(1).

Ni D, Jia Z, Yang J, Kasabov N. Online low-light sand-dust video enhancement using adaptive dynamic brightness correction and a rolling guidance filter. IEEE Trans Multimed. 2023;26:2192-2206

Zhang J, Wang X, Wan Y, Wang L, Wang J, Philip SY. SOR-TC: Self-attentive octave ResNet with temporal consistency for compressed video action recognition. Neurocomputing. 2023;533:191–205.

Article  Google Scholar 

Garbouge H, Rasti P, Rousseau D. Enhancing the tracking of seedling growth using RGB-Depth fusion and deep learning. Sensors. 2021;21(24): 8425.

Article  Google Scholar 

Pandeya YR, Lee J. Deep learning-based late fusion of multi-modal information for emotion classification of music video. Multimedia Tools and Applications. 2021;80(2):2887–905.

Article  Google Scholar 

Liu F, Xu H, Qi M, Liu D, Wang J, Kong J. Depth-wise separable convolution attention module for garbage image classification. Sustainability. 2022;14(5): 3099.

Article  Google Scholar 

Qararyah F, Azhar MW, Maleki MA, Trancoso P. Fusing depthwise and pointwise convolutions for efficient inference on GPUs. arXiv preprint arXiv:2404.19331. 2024.

Song Y, Luktarhan N, Shi Z, Wu H. TGA: a novel network intrusion detection method based on TCN, BiGRU and attention mechanism. Electronics. 2023;12(13): 2849.

Article  Google Scholar 

Singh H, Rai V, Kumar N, Dadheech P, Kotecha K, Selvachandran G, Abraham A. An enhanced whale optimization algorithm for clustering. Multimedia tools and applications. 2023;82(3):4599–618.

Article  Google Scholar 

https://www.kaggle.com/datasets/crazyt/vidtimit-audiovideo-dataset

https://www.kaggle.com/datasets/mahafinalyearproject/fakeavceleb

Kaggle. 2019. https://kaggle.com/competitions/deepfake-detection-challenge. Accessed 12 June 2020.

Karras T, Laine S, Aila T. A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019. pp. 4401–4410.

Karras T, Aila T, Laine S, Lehtinen J. Progressive growing of gans for improved quality, stability, and variation. https://doi.org/10.48550/arXiv.1710.10196.

Choi Y, Choi M, Kim M, Ha JW, Kim S, Choo J. Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In:Proceedings of the IEEE conference on computer vision and pattern recognition. 2018; 8789–8797.

Rössler A, Cozzolino D, Verdoliva L, Riess C, Thies J, Nießner M. Faceforensics: a large-scale video dataset for forgery detection in human faces. arXiv preprint arXiv:1803.09179. 2018.

Hou Y, Guo Q, Huang Y, Xie X, Ma L, Zhao J. Evading deepfake detectors via adversarial statistical consistency. In:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023; 12271–12280.

Muppalla S, Jia S, Lyu S. Integrating audio-visual features for multi-modal deepfake detection. In: 2023 IEEE MIT Undergraduate Research Technology Conference (URTC), IEEE. 2023; 1–5.

Cai J, Li Y, Liu B, Wu Z, Zhu S, Chen Q, Lei Q, Hou H, Guo Z, Jiang H, Guo S, Wang F, Huang S, Zhu S, Fan X, Tao S. Developing deep LSTMs with later temporal attention for predicting covid-19 severity, clinical outcome, and antibody level by screening serological indicators over time. IEEE J Biomed Health Inform. 2024;28(7):4204–54215. https://doi.org/10.1109/JBHI.2024.3384333.

Cai J, Chen T, Qi Y, Liu S, Chen R. Fibrosis and inflammatory activity diagnosis of chronic hepatitis C based on extreme learning machine. Sci Rep. 2025;15(1):11.

Comments (0)

No login
gif