文献类型：期刊文献

英文题名：Uncertainty-Aware Graph Contrastive Fusion Network for multimodal physiological signal emotion recognition

作者：Li, Guangqiang[1]; Chen, Ning[1]; Zhu, Hongqing[1]; Li, Jing[1]; Xu, Zhangyong[1]; Zhu, Zhiying[1]

机构：[1] School of Information Science and Engineering, East China University of Science and Technology, Shanghai, 200237, China

年份：2025

卷号：187

外文期刊名：Neural Networks

收录：EI(收录号：20251218063194)

语种：英文

外文关键词：Convolutional neural networks - Graph neural networks - Physiological models - Semantics - Signal modulation

摘要：Graph Neural Networks (GNNs) have been widely adopted to mine topological patterns contained in physiological signals for emotion recognition. However, since physiological signals are non-stationary and susceptible to various noises, there exists inter-sensor connectivity uncertainty in each modality. Such intra-modal connectivity uncertainty may further lead to inter-modal semantic gap uncertainty, which will cause the unimodal bias problem and greatly affect the fusion effectiveness. While, such issue has never been fully considered in existing multimodal fusion models. To this end, we proposed an Uncertainty-Aware Graph Contrastive Fusion Network (UAGCFNet) to fuse multimodal physiological signals effectively for emotion recognition. Firstly, a probabilistic model-based Uncertainty-Aware Graph Convolutional Network (UAGCN), which can estimate and quantify the inter-sensor connectivity uncertainty, is constructed for each modality to extract its uncertainty-aware graph representation. Secondly, a Transitive Contrastive Fusion (TCF) module, which combines the Criss-Cross Attention (CCA)-based fusion mechanism and Transitive Contrastive Learning (TCL)-based calibration strategy organically, is designed to achieve effective fusion of multimodal graph representations by eliminating the unimodal bias problem resulting from the inter-modal semantic gap uncertainty. Extensive experimental results on DEAP, DREAMER, and MPED datasets under both subject-dependent and subject-independent scenarios demonstrate that (i) the proposed model outperforms State-Of-The-Art (SOTA) multimodal fusion models with fewer parameters and lower computational complexity; (ii) each key module and loss function contributes significantly to the performance enhancement of the proposed model; (iii) the proposed model can eliminate the unimodal bias problem effectively. ? 2025 Elsevier Ltd