TAR: Generalized Forensic Framework to Detect Deepfakes Using Weakly Supervised Learning - Inria - Institut national de recherche en sciences et technologies du numérique
Conference Papers Year : 2021

TAR: Generalized Forensic Framework to Detect Deepfakes Using Weakly Supervised Learning

Sangyup Lee
  • Function : Author
  • PersonId : 1154432
Shahroz Tariq
  • Function : Author
  • PersonId : 1154433
Junyaup Kim
  • Function : Author
  • PersonId : 1154434
Simon S. Woo
  • Function : Author
  • PersonId : 1117621

Abstract

Deepfakes have become a critical social problem, and detecting them is of utmost importance. Also, deepfake generation methods are advancing, and it is becoming harder to detect. While many deepfake detection models can detect different types of deepfakes separately, they perform poorly on generalizing the detection performance over multiple types of deepfake. This motivates us to develop a generalized model to detect different types of deepfakes. Therefore, in this work, we introduce a practical digital forensic tool to detect different types of deepfakes simultaneously and propose Transfer learning-based Autoencoder with Residuals (TAR). The ultimate goal of our work is to develop a unified model to detect various types of deepfake videos with high accuracy, with only a small number of training samples that can work well in real-world settings. We develop an autoencoder-based detection model with Residual blocks and sequentially perform transfer learning to detect different types of deepfakes simultaneously. Our approach achieves a much higher generalized detection performance than the state-of-the-art methods on the FaceForensics++ dataset. In addition, we evaluate our model on 200 real-world Deepfake-in-the-Wild (DW) videos of 50 celebrities available on the Internet and achieve 89.49% zero-shot accuracy, which is significantly higher than the best baseline model (gaining 10.77%), demonstrating and validating the practicability of our approach.
Fichier principal
Vignette du fichier
512098_1_En_23_Chapter.pdf (1.34 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03746055 , version 1 (04-08-2022)

Licence

Identifiers

Cite

Sangyup Lee, Shahroz Tariq, Junyaup Kim, Simon S. Woo. TAR: Generalized Forensic Framework to Detect Deepfakes Using Weakly Supervised Learning. 36th IFIP International Conference on ICT Systems Security and Privacy Protection (SEC), Jun 2021, Oslo, Norway. pp.351-366, ⟨10.1007/978-3-030-78120-0_23⟩. ⟨hal-03746055⟩
50 View
14 Download

Altmetric

Share

More