文献类型：期刊文献

英文题名：LRB-T: local reasoning back-projection transformer for the removal of bad weather effects in images

作者：Wang, Pengyu[1]; Zhu, Hongqing[1]; Zhang, Huaqi[2]; Yang, Suyi[3]

机构：[1] School of Information Science and Engineering, East China University of Science and Technology, No. 130 Meilong Road, Shanghai, 200237, China; [2] School of Computer and Information Technology, Beijing Jiaotong University, No. 3 Shangyuancun, Beijing, 100044, China; [3] Department of Mathematics, Natural, Mathematical and Engineering Sciences, King’s College London, Strand, London, England, WC2R 2LS, United Kingdom

年份：2024

卷号：36

期号：2

起止页码：773

外文期刊名：Neural Computing and Applications

收录：EI(收录号：20234214918166)

语种：英文

外文关键词：Demulsification - Drops - Image enhancement - Iterative methods - Rain

摘要：In computer vision, transformers have shown increasing effectiveness for high-level vision tasks. To further cope with low-level vision tasks, we propose a general framework, namely local reasoning back-projection transformer (LRB-T) for removing multiple types of bad weather (rain, haze, rain fog, raindrop, et al.) affecting images. Specifically, this paper first integrates the back-projection mechanism into the transformer architecture, where iterative up- and down-projection modules effectively feed and correct the feature reconstruction errors for spatial information preservation, and reduce computational costs since nearly half of the feature maps participate in the back-projection using half resolution. Besides, the proposed adaptive local reasoning block captures important neighborhood information through multiple local reasoning schemes. It can aggregate adjacent tokens to produce spatial-specific involution kernels, attention weights and dynamic positional encodings for local structure updating, and provide implicit spatial feature transform to achieve spatial-wise feature modulation. A pyramid scale guidance module is also established to enable arbitrary size generation consistent with the input, and generate scale-dependent trainable parameters to enhance skip connections. Extensive experiments on four types of well-known bad weather datasets show that the proposed LRB-T improves the image deraining, dehazing, de-rain fog and de-raindrop performance in terms of PSNR and SSIM effectively and outperforms state-of-the-art task-specific bad weather removal methods. ? 2023, The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature.