Learning a deep dual-level network for robust DeepFake detection

作者:

Highlights:

• We alleviate the adverse impact of an imbalanced dataset by introducing the AUC loss to directly maximize the AUC score and minimize the adverse impact of an imbalanced dataset.

• We combine the features of both frame-level and video-level methods into an integrated network that detects not only each individual forged frame but also takes the temporal relations of frames into consideration to provide an overall prediction for a whole video.

• By sampling the Celeb-DF and DFDC datasets, we generate five subsets to simulate real-world imbalanced data conditions.

• We conduct an exhaustive analysis on the data imbalance problem to highlight the efficacy of our proposed joint loss function and the robustness of our model when facing data imbalance problems.

摘要

•We alleviate the adverse impact of an imbalanced dataset by introducing the AUC loss to directly maximize the AUC score and minimize the adverse impact of an imbalanced dataset.•We combine the features of both frame-level and video-level methods into an integrated network that detects not only each individual forged frame but also takes the temporal relations of frames into consideration to provide an overall prediction for a whole video.•By sampling the Celeb-DF and DFDC datasets, we generate five subsets to simulate real-world imbalanced data conditions.•We conduct an exhaustive analysis on the data imbalance problem to highlight the efficacy of our proposed joint loss function and the robustness of our model when facing data imbalance problems.

论文关键词:DeepFake detection,Multitask learning,Imbalanced learning,AUC optimization

论文评审过程:Received 24 August 2021, Revised 29 March 2022, Accepted 3 June 2022, Available online 3 June 2022, Version of Record 13 June 2022.

论文官网地址:https://doi.org/10.1016/j.patcog.2022.108832