文献类型：期刊文献

英文题名：CapHDR2IR: Caption-Driven Transfer from Visible Light to Infrared Domain

作者：Peng, Jingchao[1,2]; Bashford-Rogers, Thomas[1]; Shao, Zhuang[3]; Zhao, Haitao[2]; Singh, Aru Ranjan[1]; Goswami, Abhishek[1]; Debattista, Kurt[1]

机构：[1] WMG, University of Warwick, United Kingdom; [2] School of Information Science and Technology, East China University of Science and Technology, China; [3] School of Engineering, Newcastle University, United Kingdom

年份：2024

外文期刊名：arXiv

收录：EI(收录号：20240502144)

语种：英文

外文关键词：Infrared detectors - Thermography (imaging) - Visual languages

摘要：Infrared (IR) imaging offers advantages in several fields due to its unique ability of capturing content in extreme light conditions. However, the demanding hardware requirements of high-resolution IR sensors limit its widespread application. As an alternative, visible light can be used to synthesize IR images but this causes a loss of fidelity in image details and introduces inconsistencies due to lack of contextual awareness of the scene. This stems from a combination of using visible light with a standard dynamic range, especially under extreme lighting, and a lack of contextual awareness can result in pseudo-thermal-crossover artifacts. This occurs when multiple objects with similar temperatures appear indistinguishable in the training data, further exacerbating the loss of fidelity. To solve this challenge, this paper proposes CapHDR2IR, a novel framework incorporating vision-language models using high dynamic range (HDR) images as inputs to generate IR images. HDR images capture a wider range of luminance variations, ensuring reliable IR image generation in different light conditions. Additionally, a dense caption branch integrates semantic understanding, resulting in more meaningful and discernible IR outputs. Extensive experiments on the HDRT dataset show that the proposed CapHDR2IR achieves state-of-the-art performance compared with existing general domain transfer methods and those tailored for visible-to-infrared image translation. Copyright ? 2024, The Authors. All rights reserved.