文献类型：期刊文献

英文题名：Local attention and long-distance interaction of rPPG for deepfake detection

作者：Wu, Jiahui[1];Zhu, Yu[1,2];Jiang, Xiaoben[1];Liu, Yatong[1];Lin, Jiajun[1]

机构：[1]East China Univ Sci & Technol, Sch Informat Sci & Engn, Shanghai 200237, Peoples R China;[2]Shanghai Engn Res Ctr Internet Things Resp Med, Shanghai 200032, Peoples R China

年份：2024

卷号：40

期号：2

起止页码：1083

外文期刊名：VISUAL COMPUTER

收录：;EI(收录号：20231413842425);WOS:【SCI-EXPANDED(收录号:WOS:000967291000001)】；

基金：The authors greatly appreciate the financial supports of Natural Science Foundation of Shanghai under Grant 22ZR1444700, National Natural Science Foundation of China under Grant 82170110, Science and Technology Commission of Shanghai Municipality under Grant 20DZ2261200.

语种：英文

外文关键词：Digital video forensics; Deepfake; PPG; CNN

摘要：With the development of generative models, abused Deepfakes have aroused public concerns. As a defense mechanism, face forgery detection methods have been intensively studied. Remote photoplethysmography (rPPG) technology extract heartbeat signal from recorded videos by examining the subtle changes in skin color caused by cardiac activity. Since the face forgery process inevitably disrupts the periodic changes in facial color, rPPG signal proves to be a powerful biological indicator for Deepfake detection. Motivated by the key observation that rPPG signals produce unique rhythmic patterns in terms of different manipulation methods, we regard Deepfake detection also as a source detection task. The Multi-scale Spatial-Temporal PPG map is adopted to further exploit heartbeat signal from multiple facial regions. Moreover, to capture both spatial and temporal inconsistencies, we propose a two-stage network consisting of a Mask-Guided Local Attention module (MLA) to capture unique local patterns of PPG maps, and a Temporal Transformer to interact features of adjacent PPG maps in long distance. Abundant experiments on FaceForensics + + and Celeb-DF datasets prove the superiority of our method over all other rPPG-based approaches. Visualization also demonstrates the effectiveness of the proposed method.