文献类型：期刊文献

英文题名：IVFuseNet: Fusion of infrared and visible light images for depth prediction

作者：Li, Yuqi[1];Zhao, Haitao[1];Hu, Zhengwei[1];Wang, Qianqian[1];Chen, Yuru[1]

机构：[1]East China Univ Sci & Technol, 130 Meilong Rd, Shanghai 200237, Peoples R China

年份：2020

卷号：58

起止页码：1

外文期刊名：INFORMATION FUSION

收录：;EI(收录号：20200107978888);WOS:【SCI-EXPANDED(收录号:WOS:000516799200001)】；

基金：This research is sponsored by National Natural Science Foundation of China (61375007 and 61375012) and Basic Research Programs of Science and Technology Commission Foundation of Shanghai (15JC1400600).

语种：英文

外文关键词：Depth prediction; Partially coupled filter; Adaptive weighted fusion; Visible image; Infrared image

摘要：Depth prediction is an essential component in the research of unmanned driving. Most existing research works predict depth only based on visible light images or infrared images. However, both visible light images and infrared images have their own advantages and disadvantages, and these two kinds of images contain complementary information when the images are filmed from the same scence. In order to fuse the complementary information and predict depth under various conditions, this paper proposes a convolutional-neural-network-based architecture, called infrared and visible light images fusion network (IVFuseNet), for depth prediction. Specifically, we construct common-feature-fusion subnetwork, full-feature-fusion subnetwork, and high-resolution reconstruction subnetwork, aiming to leverage the complementarity of these two kinds of images. The common-feature-fusion subnetwork adopts a two-stream multilayer convolutional structure whose filters for each layer are partially coupled to fuse the common features extracted from infrared images and visible light images respectively. The full-feature-fusion subnetwork fuses the two-stream features generated from the common-feature-fusion subnetwork by adaptive fusion weights instead of prefixed fusion weights. Additional, residual dense convolution that can accurately map the fused low-resolution features to the corresponding high-resolution features is adopted in the high-resolution reconstruction subnetwork to enhance the reconstruction of the details for depth prediction. All three subnetworks collaborate together to conduct the depth prediction task. Our NUST-SR dataset is composed of the actual road scenes captured while unmanned vehicle driving. The proposed IVFuseNet obtains the best performances on this dataset. IVFuseNet decreases the root mean squared error to 3.4513 and the mean relative error to 0.1651 respectively and outperforms other methods.