文献类型：期刊文献

英文题名：Beyond Entities: A Large-Scale Multi-Modal Knowledge Graph with Triplet Fact Grounding

作者：Liu, Jingping[1]; Zhang, Mingchuan[2]; Li, Weichen[2]; Wang, Chao[3]; Li, Shuang[2]; Jiang, Haiyun[4]; Jiang, Sihang[2]; Xiao, Yanghua[2]; Chen, Yunwen[5]

机构：[1] School of Information Science and Engineering, East China University of Science and Technology, Shanghai, China; [2] Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University, Shanghai, China; [3] School of Future Technology, Shanghai University, Shanghai, China; [4] Tencent AI Lab., Shenzhen, China; [5] DataGrand Inc., Shanghai, China

年份：2024

卷号：38

期号：17

起止页码：18653

外文期刊名：Proceedings of the AAAI Conference on Artificial Intelligence

收录：EI(收录号：20241515874392)

语种：英文

外文关键词：Image enhancement

摘要：Much effort has been devoted to building multi-modal knowledge graphs by visualizing entities on images, but ignoring the multi-modal information of the relation between entities. Hence, in this paper, we aim to construct a new large-scale multi-modal knowledge graph with triplet facts grounded on images that reflect not only entities but also their relations. To achieve this purpose, we propose a novel pipeline method, including triplet fact filtering, image retrieving, entity-based image filtering, relation-based image filtering, and image clustering. In this way, a multi-modal knowledge graph named ImgFact is constructed, which contains 247,732 triplet facts and 3,730,805 images. In experiments, the manual and automatic evaluations prove the reliable quality of our ImgFact. We further use the obtained images to enhance model performance on two tasks. In particular, the model optimized by our ImgFact achieves an impressive 8.38% and 9.87% improvement over the solutions enhanced by an existing multimodal knowledge graph and VisualChatGPT on F1 of relation classification. We release ImgFact and its instructions at https://github.com/kleinercubs/ImgFact. ? 2024, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.