文献类型：期刊文献

英文题名：Semantic Completion: Enhancing Image-Text Retrieval withInformation Extraction andCompression

作者：Chen, Xue[1]; Guo, Yi[1,2,3]

机构：[1] East China University of Science and Technology, Shanghai, China; [2] Business Intelligence and Visualization Research Center, National Engineering Laboratory for Big Data Distribution and Exchange Technologies, Shanghai, China; [3] Shanghai Engineering Research Center of Big Data and Internet Audience, Shanghai, China

年份：2024

卷号：14648 LNAI

起止页码：59

外文期刊名：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

收录：EI(收录号：20242116127850)

语种：英文

外文关键词：Image compression - Image enhancement - Image reconstruction - Information retrieval

摘要：Image-text retrieval is an essential branch in the field of information retrieval, facing the serious challenge of the cross-modal semantic gap. Although significant progress has been made in recent years, most research has ignored an essential problem: text as image description is incomplete, so the problem of semantic loss between image and text still exists. In this paper, we propose a novel information extraction and compression based image-text retrieval method to alleviate the above problem. The method aims to bridge the semantic gap between the two modalities by generating rich and high-quality semantic descriptions from a set of related sentences via an information extraction and compression module. To validate the effectiveness of the method, we conducted extensive experiments on the Flickr30K and MSCOCO datasets. The experimental results show that our method achieves significant performance improvement in image text retrieval with an appropriate fusion ratio. When the amount of pre-trained images is 4M, the evaluation metrics of our method improve by at least 4.22% and 3.69% compared to the baseline method. This further confirms the advantages and potential of our method in solving the semantic loss problem in image-text retrieval. ? The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.