文献类型：期刊文献

英文题名：Siamese infrared and visible light fusion network for RGB-T tracking

作者：Peng, Jingchao[1];Zhao, Haitao[1];Hu, Zhengwei[1];Zhuang, Yi[1];Wang, Bofan[1]

机构：[1]East China Univ Sci & Technol, Sch Informat Sci & Engn, Meilong Rd 130, Shanghai 200237, Peoples R China

年份：2023

卷号：14

期号：9

起止页码：3281

外文期刊名：INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS

收录：;EI(收录号：20231714009516);WOS:【SCI-EXPANDED(收录号:WOS:000975611600001)】；

基金：AcknowledgementsThis research is sponsored by National Natural Science Foundation of China (62173143 and 61973122).

语种：英文

外文关键词：Object tracking; Deep learning; Fusion tracking; Siamese network

摘要：Due to the different photosensitive properties of infrared and visible light, infrared and visible light images have individual features. However, since the registered RGB-T image pairs shot in the same scene, they also contain common features. This paper proposes a Siamese infrared and visible light fusion Network (SiamIVFN) for RBG-T image-based tracking. SiamIVFN contains two main subnetworks: a complementary-feature-fusion network (CFFN) and a contribution-aggregation network (CAN). CFFN utilizes a two-stream multilayer convolutional structure that separately extracts individual features, and filters in each layer are partially coupled to extract common features. CFFN is a feature-level fusion network, which can cope with the misalignment of the RGB-T image pairs. Through adaptively calculating the contributions of infrared and visible light features obtained from CFFN, CAN makes the tracker robust under various light conditions. Experiments show that compared to state-of-the-art techniques, SiamIVFN improves the PR/SR score with 1.5%/8.8% on RGBT234 and 2.1%/6.9% on GTOT. The tracking speed of SiamIVFN is 147.6FPS, the current fastest RGB-T fusion tracker. The source codes are available at https://github.com/PengJingchao/SiamIVFN.