Dual-Branch Pre-activation Bottleneck Transformer for Face Forgery Detection

DeepFakes. www.github.com/deepfakes/faceswap. Accessed 1 May 2024.

Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. Advances in neural information processing systems. 2014;27.

Kong C, Wang S, Li H. Digital and physical face attacks: reviewing and one step further. APSIPA Transactions on Signal and Information Processing. 2022;12(1):e25.

Google Scholar 

Han W, Tian Z, Zhu C, Huang Z, Jia Y, Guizani M. A topic representation model for online social networks based on hybrid human–artificial intelligence. IEEE Trans Comput Social Syst. 2019;8(1):191–200.

Article  Google Scholar 

Li S, Jiang L, Wu X, Han W, Zhao D, Wang Z. A weighted network community detection algorithm based on deep learning. Appl Math Comput. 2021;15(401):126012.

MathSciNet  Google Scholar 

Wang Z, Hu Z, Li F, Ho SB, Cambria E. Learning-based stock trending prediction by incorporating technical indicators and social media sentiment. Cogn Comput. 2023;15(3):1092–102.

Article  Google Scholar 

Verdoliva L. Media forensics and deepfakes: an overview. IEEE J Selected Topics Signal Process. 2020;14(5):910–32.

Article  Google Scholar 

Heidari A, Navimipour NJ, Dag H, Talebi S, Unal M. A novel blockchain-based deepfake detection method using federated and deep learning models. Cogn Comput. 2024;26:1–9.

Google Scholar 

Khowaja, S.A., Khuwaja, P., Dev, K. et al. ChatGPT needs SPADE (sustainability, privacy, digital divide, and ethics) evaluation: a review. Cognitive Computation. 2024. https://doi.org/10.1007/s12559-024-10285-1

Fan Z, De Queiroz RL. Identification of bitmap compression history: JPEG detection and quantizer estimation. IEEE Trans Image Process. 2003;12(2):230–5.

Article  Google Scholar 

Popescu AC, Farid H. Exposing digital forgeries in color filter array interpolated images. IEEE Trans Signal Process. 2005;53(10):3948–59.

Article  MathSciNet  Google Scholar 

Bianchi T, Piva A. Image forgery localization via block-grained analysis of JPEG artifacts. IEEE Trans Inf Forensics Secur. 2012;7(3):1003–17.

Article  Google Scholar 

Lyu S, Pan X, Zhang X. Exposing region splicing forgeries with blind local noise estimation. Int J Comput Vision. 2014;110:202–21.

Article  Google Scholar 

Lin K, Zhao H, Lv J, Li C, Liu X, Chen R, Zhao R. Face detection and segmentation based on improved mask R-CNN. Discret Dyn Nat Soc. 2020;1(2020):1–1.

Google Scholar 

Thies J, Zollhofer M, Stamminger M, Theobalt C, Nießner M. Face2face: real-time face capture and reenactment of rgb videos. In: Proceedings of the IEEE conference on computer vision and pattern recognition 2016 (pp. 2387–2395).

FaceSwap. www.github.com/MarekKowalski/FaceSwap. Accessed 1 May 2024.

Thies J, Zollhöfer M, Nießner M. Deferred neural rendering: image synthesis using neural textures. Acm Trans Graphics (TOG). 2019;38(4):1–2.

Article  Google Scholar 

Chen Z, Lin J, Zhou T, Wu F. Sequential gating ensemble network for noise robust multiscale face restoration. IEEE Trans Cybernetics. 2019;51(1):451–61.

Article  Google Scholar 

Zhou P, Han X, Morariu VI, Davis LS. Learning rich features for image manipulation detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition 2018 (pp. 1053–1061).

Fridrich J, Kodovsky J. Rich models for steganalysis of digital images. IEEE Trans Inf Forensics Secur. 2012;7(3):868–82.

Article  Google Scholar 

Wang J, Wu Z, Ouyang W, Han X, Chen J, Jiang YG, Li SN. M2tr: multi-modal multi-scale transformers for deepfake detection. In: Proceedings of the 2022 international conference on multimedia retrieval 2022(pp. 615–623).

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. Advances in neural information processing systems. 2017;30.

Srinivas A, Lin TY, Parmar N, Shlens J, Abbeel P, Vaswani A. Bottleneck transformers for visual recognition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2021 (pp. 16519–16529).

Tolosana R, Vera-Rodriguez R, Fierrez J, Morales A, Ortega-Garcia J. Deepfakes and beyond: a survey of face manipulation and fake detection. Information Fusion. 2020;1(64):131–48.

Article  Google Scholar 

Wang Q, Qin Z, Nie F, Li X. C2DNDA: a deep framework for nonlinear dimensionality reduction. IEEE Trans Industr Electron. 2020;68(2):1684–94.

Article  Google Scholar 

Korshunov P, Marcel S. Deepfakes: a new threat to face recognition? Assessment and detection. arXiv preprint arXiv:1812.08685. 2018.

Wang Z, Chen Z, Wu F. Thermal to visible facial image translation using generative adversarial networks. IEEE Signal Process Lett. 2018;25(8):1161–5.

Article  Google Scholar 

Graves A, Graves A. Long short-term memory. Supervised sequence labelling with recurrent neural networks. 2012:37–45.

Afchar D, Nozick V, Yamagishi J, Echizen I. Mesonet: a compact facial video forgery detection network. In: 2018 IEEE international workshop on information forensics and security (WIFS) 2018 (pp. 1–7). IEEE.

Zhou P, Han X, Morariu VI, Davis LS. Two-stream neural networks for tampered face detection. In: 2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW) 2017 (pp. 1831–1839). IEEE.

Güera D, Delp EJ. Deepfake video detection using recurrent neural networks. In: 2018 15th IEEE international conference on advanced video and signal based surveillance (AVSS) 2018(pp. 1–6). IEEE.

Rossler A, Cozzolino D, Verdoliva L, Riess C, Thies J, Nießner M. Faceforensics++: learning to detect manipulated facial images. In: Proceedings of the IEEE/CVF international conference on computer vision 2019 (pp. 1–11).

Nguyen HH, Yamagishi J, Echizen I. Use of a capsule network to detect fake images and videos. arXiv preprint arXiv:1910.12467. 2019 Oct 28.

Nguyen HH, Yamagishi J, Echizen I. Capsule-forensics: using capsule networks to detect forged images and videos. In: ICASSP 2019–2019 IEEE international conference on acoustics, speech and signal processing (ICASSP) 2019 (pp. 2307–2311). IEEE.

Sabour S, Frosst N, Hinton GE. Dynamic routing between capsules. Advances in neural information processing systems. 2017;30.

Lin K, Han W, Li S, Gu Z, Zhao H, Ren J, Zhu L, Lv J. IR-capsule: two-stream network for face forgery detection. Cogn Comput. 2023;15(1):13–22.

Article  Google Scholar 

Szegedy C, Ioffe S, Vanhoucke V, Alemi A. Inception-v4, inception-resnet and the impact of residual connections on learning. In: Proceedings of the AAAI conference on artificial intelligence 2017 (Vol. 31, No. 1).

Kong C, Chen B, Li H, Wang S, Rocha A, Kwong S. Detect and locate: exposing face manipulation by semantic-and noise-level telltales. IEEE Trans Inf Forensics Secur. 2022;28(17):1741–56.

Article  Google Scholar 

Li L, Bao J, Zhang T, Yang H, Chen D, Wen F, Guo B. Face x-ray for more general face forgery detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2020 (pp. 5001–5010).

Lin K, Han W, Li S, Gu Z, Zhao H, Mei Y. Detecting deepfake videos using spatiotemporal trident network. ACM Trans Multimed Comput Commun Appl. 2024;20(11):1–20.

Article  Google Scholar 

Luo A, Kong C, Huang J, Hu Y, Kang X, Kot AC. Beyond the prior forgery knowledge: mining critical clues for general face forgery detection. IEEE Trans Inf Forensics Secur. 2023;13(19):1168–82.

Google Scholar 

Yu P, Fei J, Xia Z, Zhou Z, Weng J. Improving generalization by commonality learning in face forgery detection. IEEE Trans Inf Forensics Secur. 2022;27(17):547–58.

Article  Google Scholar 

Cao J, Ma C, Yao T, Chen S, Ding S, Yang X. End-to-end reconstruction-classification learning for face forgery detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2022 (pp. 4113–4122).

Li Y, Chang MC, Lyu S. In ictu oculi: exposing ai created fake videos by detecting eye blinking. In: 2018 IEEE International workshop on information forensics and security (WIFS) 2018 (pp. 1–7). IEEE.

Fernandes S, Raj S, Ortiz E, Vintila I, Salter M, Urosevic G, Jha S. Predicting heart rate variations of deepfake videos using neural ode. In: Proceedings of the IEEE/CVF international conference on computer vision workshops 2019 (pp. 0–0).

Ciftci UA, Demir I, Yin L. Fakecatcher: detection of synthetic portrait videos using biological signals. IEEE transactions on pattern analysis and machine intelligence. 2020.

Nirkin Y, Wolf L, Keller Y, Hassner T. Deepfake detection based on discrepancies between faces and their context. IEEE Trans Pattern Anal Mach Intell. 2021;44(10):6111–21.

Article  Google Scholar 

Wodajo D, Atnafu S. Deepfake video detection using convolutional vision transformer. arXiv preprint arXiv:2102.11126. 2021 Feb 22.

Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J. An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929. 2020 Oct 22.

Heo YJ, Choi YJ, Lee YW, Kim BG. Deepfake detection scheme based on vision transformer and distillation. arXiv preprint arXiv:2104.01353. 2021 Apr 3.

Coccomini DA, Messina N, Gennaro C, Falchi F. Combining efficientnet and vision transformers for video deepfake detection. In: International conference on image analysis and processing 2022 (pp. 219–229). Cham: Springer International Publishing

Kong C, Li H, Wang S. Enhancing general face forgery detection via vision transformer with low-rank adaptation. In: 2023 IEEE 6th International Conference on Multimedia Information Processing and Retrieval (MIPR) 2023 (pp. 102–107). IEEE.

Kong C, Luo A, Bao P, Yu Y, Li H, Zheng Z, Wang S, Kot AC. Moe-ffd: mixture of experts for generalized and parameter-efficient face forgery detection. arXiv preprint arXiv:2404.08452. 2024 Apr 12.

Khan SA, Dai H. Video transformer for deepfake detection with incremental learning. In: Proceedings of the 29th ACM international conference on multimedia 2021 (pp. 1821–1828).

Sun Y, Zhang Z, Qiu C, Wang L, Sun L, Wang Z. Faketransformer: exposing face forgery from spatial-temporal representation modeled by facial pixel variations. In: 2022 7th international conference on intelligent computing and signal processing (ICSP) 2022 (pp. 705–713). IEEE.

Zhang K, Zhang Z, Li Z, Qiao Y. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process Lett. 2016;23(10):1499–503.

Article  Google Scholar 

He K, Zhang X, Ren S, Sun J. Identity mappings in deep residual networks. In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14 2016 (pp. 630–645). Springer International Publishing.

Gao Y, Beijbom O, Zhang N, Darrell T. Compact bilinear pooling. In: Proceedings of the IEEE conference on computer vision and pattern recognition 2016 (pp. 317–326).

Dogonadze N, Obernosterer J, Hou J. Deep face forgery detection. arXiv preprint arXiv:2004.11804. 2020 Apr 6.

Feng H, Hong Z, Yue H, Chen Y, Wang K, Han J, Liu J, Ding E. Learning generalized spoof cues for face anti-spoofing. arXiv preprint arXiv:2005.03922. 2020 May 8.

Li Y, Yang X, Sun P, Qi H, Lyu S. Celeb-df: a large-scale challenging dataset for deepfake forensics. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2020 (pp. 3207–3216).

Yang J, Li A, Xiao S, Lu W, Gao X. MTD-Net: learning to detect deepfakes images by multi-scale texture difference. IEEE Trans Inf Forensics Secur. 2021;3(16):4234–45.

Article  Google Scholar 

Cozzolino D, Poggi G, Verdoliva L. Recasting residual-based local descriptors as convolutional neural networks: an application to image forgery detection. In: Proceedings of the 5th ACM workshop on information hiding and multimedia security 2017 (pp. 159–164).

Zi B, Chang M, Chen J, Ma X, Jiang YG. Wilddeepfake: a challenging real-world dataset for deepfake detection. In: Proceedings of the 28th ACM international conference on multimedia 2020 (pp. 2382–2390).

Yang X, Li Y, Lyu S. Exposing deep fakes using inconsistent head poses. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2019 (pp. 8261–8265). IEEE.

Rahmouni N, Nozick V, Yamagishi J, Echizen I. Distinguishing computer graphics from natural images using convolution neural networks. In: 2017 IEEE workshop on information forensics and security (WIFS) 2017 (pp. 1–6). IEEE.

Kohli A, Gupta A. Detecting deepfake, faceswap and face2face facial forgeries using frequency CNN. Multimed Tools Appl. 2021;80(12):18461–78.

Article  Google Scholar 

Bayar B, Stamm MC. A deep learning approach to universal image manipulation detection using a new convolutional layer. In: Proceedings of the 4th ACM workshop on information hiding and multimedia security 2016 (pp. 5–10).

Liu H, Simonyan K, Yang Y. Darts: differentiable architecture search. arXiv preprint arXiv:1806.09055. 2018 Jun 24.

Baek JY, Yoo YS, Bae SH. Generative adversarial ensemble learning for face forensics. Ieee Access. 2020;4(8):45421–31.

Article  Google Scholar 

Comments (0)

No login
gif