文献类型：期刊文献

英文题名：Length Uncertainty-Aware Graph Contrastive Fusion Network for multimodal physiological signal emotion recognition

作者：Li, Guangqiang[1];Chen, Ning[1];Zhu, Hongqing[1];Li, Jing[1];Xu, Zhangyong[1];Zhu, Zhiying[1]

机构：[1]East China Univ Sci & Technol, Sch Informat Sci & Engn, Shanghai 200237, Peoples R China

年份：2025

卷号：187

外文期刊名：NEURAL NETWORKS

收录：;WOS:【SCI-EXPANDED(收录号:WOS:001449937500001)】；

基金：This work was supported by the National Natural Science Founda-tion of China [grant number 61771196, 61872143] . We would like to thank the authors of RHPRNet (Tang et al., 2024) for sharing the DEAP data features they adopted with us.

语种：英文

外文关键词：Multimodal emotion recognition; Physiological signal uncertainty; Unimodal bias; Graph contrastive fusion; Uncertainty-aware graph; Transitive contrastive learning

摘要：Graph Neural Networks (GNNs) have been widely adopted to mine topological patterns contained in physiological signals for emotion recognition. However, since physiological signals are non-stationary and susceptible to various noises, there exists inter-sensor connectivity uncertainty in each modality. Such intra-modal connectivity uncertainty may further lead to inter-modal semantic gap uncertainty, which will cause the unimodal bias problem and greatly affect the fusion effectiveness. While, such issue has never been fully considered in existing multimodal fusion models. To this end, we proposed an Uncertainty-Aware Graph Contrastive Fusion Network (UAGCFNet) to fuse multimodal physiological signals effectively for emotion recognition. Firstly, a probabilistic model-based Uncertainty-Aware Graph Convolutional Network (UAGCN), which can estimate and quantify the inter-sensor connectivity uncertainty, is constructed for each modality to extract its uncertainty-aware graph representation. Secondly, a Transitive Contrastive Fusion (TCF) module, which combines the Criss-Cross Attention (CCA)-based fusion mechanism and Transitive Contrastive Learning (TCL)-based calibration strategy organically, is designed to achieve effective fusion of multimodal graph representations by eliminating the unimodal bias problem resulting from the inter-modal semantic gap uncertainty. Extensive experimental results on DEAP, DREAMER, and MPED datasets under both subject-dependent and subject-independent scenarios demonstrate that (i) the proposed model outperforms State-Of-The-Art (SOTA) multimodal fusion models with fewer parameters and lower computational complexity; (ii) each key module and loss function contributes significantly to the performance enhancement of the proposed model; (iii) the proposed model can eliminate the unimodal bias problem effectively.