文献类型：期刊文献

中文题名：基于区域上下文感知增强网络的图像情感迁移

英文题名：Regional Context Perception Enhanced Network for Image Emotion Transfer

作者：朱继祥[1];张静[1]

机构：[1]华东理工大学信息科学与工程学院,上海200237

年份：2025

卷号：51

期号：5

起止页码：681

中文期刊名：华东理工大学学报(自然科学版)

外文期刊名：Journal of East China University of Science and Technology

收录：;北大核心:【北大核心2023】；

基金：上海市自然科学基金(22ZR1418400)。

语种：中文

中文关键词：图像情感迁移;区域上下文感知增强;情感建模关系;情感迁移综合度量;自注意机制

外文关键词：image emotion transfer;regional context perception enhanced;emotion modeling relationship;emotion transfer comprehensive metrics;self attention mechanism

摘要：为实现准确有效的图像情感迁移,本文提出基于情感建模关系引导的区域上下文感知增强网络。该网络在外部情感知识的指导下联合多种损失函数以实现准确的图像诱发情感迁移;在该网络中,提出一个新颖的区域上下文感知块,通过多种自注意提取图像中不同感受野的上下文特征,并借助交叉注意力将这些特征进行自适应融合,更全面地整合图像信息;在此基础上通过残差连接恢复深度特征融合中损失的信息,更准确地保留图像的内容。同时,提出一种新颖的情感轮引导模块,该模块基于情感轮中的情感分布,使用联合损失引导模型准确地迁移图像情感。为了准确有效地评估模型迁移图像情感的能力,提出情感迁移综合度量,综合情感类别、情感极性以及情感在情感轮上的位置,多角度地评估图像情感迁移的效果,并基于4个风格不同且广泛使用的情感数据集,构建一个新的数据集FATE。在FATE上进行的大量实验充分验证了提出方法的有效性,并在情感迁移效果和图像质量上优于其他对比的方法。
To achieve accurate and effective image emotion transfer,a region context aware enhancement network guided by emotion modeling relationships is proposed.Under the guidance of external emotion knowledge,multiple losses are combined to achieve accurate image induced emotion transfer.In this network,a novel regional context perception block(RCPB)is introduced to enhance the Twins-SVT encoder.It extracts context features of different receptive fields in images through various Locally-grouped Self Attention,Axis-wise Self Attention,and Neighbor Self Attention,and adaptively fuses them through cross attention to comprehensively integrate image information.On this basis,the lost information in the fusion of deep features are restored through residual connections to more accurately preserve the image content.Additionally,a novel emotion-wheel guided module(EGM)is proposed,which uses the emotion distribution in the emotion wheel to guide the model in accurately transferring image emotions.To accurately and effectively evaluate the ability of the model to transfer image emotion,an innovative emotion transfer comprehensive metric(ETCM)is proposed,which evaluates the effect of image emotion transfer from multiple perspectives,including emotion categories,emotion polarities,and the position of emotions in the emotion wheel.To effectively evaluate the model effectiveness of the model,a new emotion transfer dataset FATE is constructed based on four widely used emotion datasets with different styles:FI,Twitter-LDL,Emotion6,and Artphoto.The extensive experiments on FATE have fully validated the effectiveness of the proposed method and demonstrated its superiority over other compared methods.