Shuo Liu
ABSTRACT
Recently, due to the increasing harm caused by facial deepfake videos, the detection technology of facial deepfakes has attracted wide interest, and designing a scientific evaluation mechanism is conducive to the application of the technology. However, the complex perturbations in the actual scene reduce the detection performance of facial deepfake detection, and the existing evaluation indicators can not fully measure the robustness of the detection technology. To this end, this paper proposes a new robustness evaluation index system, the AUC decrease rate, and AUC stability range under multiple perturbations. Through the verification of multiple detection models with multiple datasets, the experimental results show that the proposed index can simply and intuitively evaluate the robustness of the detection technology, which is valuable for the construction of future evaluation standards.
- Mirsky Yisroel, and Wenke Lee. 2021. The creation and detection of deepfakes: A survey. ACM Computing Surveys (CSUR), 54(1): 1-41. https://dl.acm.org/doi/abs/10.1145/3425780Google Scholar
- Bingquan Zhu, Hao Fang, Yanan Sui, and Luming Li.2020. Deepfakes for Medical Video De-Identification: Privacy Protection and Diagnostic Information Preservation.National Conference on Artificial Intelligence. https://dl.acm.org/doi/abs/10.1145/3375627.3375849Google Scholar
- Asad Malik, Minoru Kuribayashi, Sani M. Abdullahi, and Ahmad Neyaz Khan.2022 .DeepFake Detection for Human Face Images and Videos: A Survey.IEEE Access, vol. 10, pp. 18757-18775. https://ieeexplore.ieee.org/abstract/document/9712265Google ScholarCross Ref
- Cairong Zhao, Chutian Wang, Guosheng Hu, Haonan Chen, Chun Liu, and Jinhui Tang.2023. ISTVT: Interpretable Spatial-Temporal Video Transformer for Deepfake Detection. IEEE Transactions on Information Forensics and Security, vol. 18, pp. 1335-1348. https://ieeexplore.ieee.org/abstract/document/10024806Google ScholarCross Ref
- Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu.2019. Celeb-DF: A large-scale challenging dataset for DeepFake forensics arXiv:1909.12962.https://openaccess.thecvf.com/content_CVPR_2020/papers/Li_Celeb-DF_A_Large-Scale_Challenging_Dataset_for_DeepFake_Forensics_CVPR_2020_paper.pdfGoogle Scholar
- Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Niessner.2019. Faceforensics++: Learning to detect manipulated facial images[C]//Proceedings of the IEEE/CVF international conference on computer vision.1-11. https://openaccess.thecvf.com/content_ICCV_2019/papers/Rossler_FaceForensics_Learning_to_Detect_Manipulated_Facial_Images_ICCV_2019_paper.pdfGoogle Scholar
- Liming Jiang, Ren Li, Wayne Wu, Chen Qian, and Chen Change Loy.2020. DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection.2020 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE. https://openaccess.thecvf.com/content_CVPR_2020/papers/Jiang_DeeperForensics-1.0_A_Large-Scale_Dataset_for_Real-World_Face_Forgery_Detection_CVPR_2020_paper.pdfGoogle Scholar
- Bojia Zi, Minghao Chang, Jingjing Chen, Xingjun Ma, and Yu-Gang Jiang.2020. WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection. The 28th ACM International Conference on Multimedia. ACM. https://dl.acm.org/doi/abs/10.1145/3394171.3413769Google Scholar
- Falko Matern, Christian Riess, and Marc Stamminger.2019. Exploiting visual artifacts to expose deepfakes and face manipulations.2019 IEEE Winter Applications of Computer Vision Workshops (WACVW). IEEE: 83-92. https://ieeexplore.ieee.org/abstract/document/8638330Google Scholar
- Darius Afchar, Vincent Nozick, Junichi Yamagishi, and Isao Echizen.2018. Mesonet: a compact facial video forgery detection network. IEEE international workshop on information forensics and security (WIFS). IEEE: 1-7. https://ieeexplore.ieee.org/abstract/document/8630761Google Scholar
- Ning Yu, Larry S. Davis, and Mario Fritz. 2019.Attributing fake images to gans: Learning and analyzing gan fingerprints.Proceedings of the IEEE/CVF international conference on computer vision: 7556-7566. https://openaccess.thecvf.com/content_ICCV_2019/papers/Yu_Attributing_Fake_Images_to_GANs_Learning_and_Analyzing_GAN_Fingerprints_ICCV_2019_paper.pdfGoogle Scholar
- Zhihao Gu, Yang Chen, Taiping Yao, Shouhong Ding, Jilin Li, Feiyue Huang, and Lizhuang Ma. 2021. Spatiotemporal inconsistency learning for DeepFake video detection. Proc. 29th ACM Int. Conf. Multimedia, Oct. 2021, pp. 3473–3481. https://dl.acm.org/doi/abs/10.1145/3474085.3475508Google ScholarDigital Library
- Peipeng Yu, Jianwei Fei, Zhihua Xia, Zhili Zhou, and Jian Weng.2022.Improving generalization by commonality learning in face forgery detection.IEEE Trans. Inf. Forensics Security, vol. 17, pp. 547–558. https://ieeexplore.ieee.org/abstract/document/9694644Google ScholarCross Ref
- Yuezun Li, Ming-Ching Chang, and Siwei Lyu.2018. In ictu oculi: Exposing ai created fake videos by detecting eye blinking.2018 IEEE international workshop on information forensics and security (WIFS). IEEE: 1-7. https://arxiv.org/pdf/1806.02877.pdfGoogle Scholar
- Xiaoyi Dong, Jianmin Bao, Dongdong Chen, Ting Zhang, Weiming Zhang, Nenghai Yu, Dong Chen, Fang Wen, and Baining Guo.2022. Protecting Celebrities from DeepFake with Identity Consistency Transformer.2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, pp. 9458-9468. https://openaccess.thecvf.com/content/CVPR2022/papers/Dong_Protecting_Celebrities_From_DeepFake_With_Identity_Consistency_Transformer_CVPR_2022_paper.pdfGoogle Scholar
- IIan J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014.Generative adversarial nets."NIPS" 14, " Proceedings of the 27th Inter- national Conference on Neural Information Processing Sys- tems, vol. 2, pp. 2672–2680. https://doi.org/10.48550/arXiv.1406.2661Google ScholarCross Ref
- Tero Karras, Samuli Laine, and Timo Aila. 2019.A style-based generator architecture for generative adversarial networks.IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 4401–4410. https://doi.org/10.48550/arXiv.1812.04948Deepfakes Github. [Online]. Available: https://github.com/deepfakes/faceswapGoogle ScholarCross Ref
- Deepfakes 2020.Github. Retrieved [ Apr 16, 2023] from https://github.com/deepfakes/faceswapGoogle Scholar
- Deepfacelab 2020. Github. Retrieved [Apr 15, 2023] from https://github.com/iperov/DeepFaceLabGoogle Scholar
- Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies and Matthias Nießner. 2018. Faceforensics: A large-scale video dataset for forgery detection in human faces. arXiv preprint arXiv:1803.09179. https://doi.org/10.48550/arXiv.1803.09179Google ScholarCross Ref
- Pavel Korshunov and Sebastien Marcel. 2018 Deepfakes: A new threat to face recognition? Assessment and detection. arXiv preprint arXiv:1812.08685. https://doi.org/10.48550/arXiv.1812.08685Google ScholarCross Ref
- Brian Dolhansky, Russ Howes, Ben Pflaum, Nicole Baram and Cristian Canton Ferrer.2019. The deepfake detection challenge (DFDC) preview dataset. arXiv:1910.08854. https://doi.org/10.48550/arXiv.1910.08854Google ScholarCross Ref
- Loc Trinh and Yan Liu.1972-1982. Interpretable and Trustworthy Deepfake Detection via Dynamic Prototypes. arXiv:2006.15473. https://doi.org/10.48550/arXiv.2006.15473Google ScholarCross Ref
- Rafael C. Gonzalez, Richard E. Woods. 2011. Digital Image Processing, Third Edition. Beijing, China.Google Scholar
- Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, Yu Qiao. 2016. Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks. IEEE Signal Processing Letters 23, 10 (April 2016), 1499-1503. https://doi.org/10.48550/arXiv.1604.02878Google ScholarCross Ref
- MTCNN 2019. Github. Retrieved [(January,1, 2017)] from https://github.com/ipazc/mtcnnGoogle Scholar
- Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, Baining Guo. 2021. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). IEEE/CVF, Montreal, QC, Canada, 9992-10002. https://doi.org/10.48550/arXiv.2103.14030Google ScholarCross Ref
- Mingxing Tan, Quoc V. Le. 2019. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. International Conference on Machine Learning, 2019 (September 2019). https://doi.org/10.48550/arXiv.1905.11946Google ScholarCross Ref
Index Terms
- Research on robustness evaluation and verification for facial deepfake detection
Index terms have been assigned to the content through auto-classification.
Recommendations
Facial Landmark Detection and Tracking for Facial Behavior Analysis
ICMR '16: Proceedings of the 2016 ACM on International Conference on Multimedia RetrievalThe face is the most dominant and distinct communication tool of human beings. Automatic analysis of facial behavior allows machines to understand and interpret a human's states and needs for natural interactions. This research focuses on developing ...
CMMR: A Composite Multidimensional Models Robustness Evaluation Framework for Deep Learning
Algorithms and Architectures for Parallel ProcessingAbstractAccurately evaluating the defense models against adversarial examples has been proven to be a challenging task. We have recognized the limitations of mainstream evaluation standards, which fail to account for the discrepancies in evaluation ...
Comments