文献类型：期刊文献

英文题名：Unexpected Phenomenon: LLMs' Spurious Associations in Information Extraction

作者：Zhang, Weiyan[1]; Lu, Wanpeng[1]; Wang, Jiacheng[1]; Wang, Yating[1]; Chen, Lihan[2]; Jiang, Haiyun[3]; Liu, Jingping[1]; Ruan, Tong[1]

机构：[1] School of Information Science and Engineering, East China University of Science and Technology, Shanghai, China; [2] Beijing Institute of Control Engineering, Beijing, China; [3] Tencent AI Lab, Shenzhen, China

年份：2024

起止页码：9176

外文期刊名：Proceedings of the Annual Meeting of the Association for Computational Linguistics

收录：EI(收录号：20244017142408)

语种：英文

外文关键词：Computational linguistics - Information retrieval - Natural language processing systems - Nonbibliographic retrieval systems - Semantics

摘要：Information extraction plays a critical role in natural language processing. When applying large language models (LLMs) to this domain, we discover an unexpected phenomenon: LLMs' spurious associations. In tasks such as relation extraction, LLMs can accurately identify entity pairs, even if the given relation (label) is semantically unrelated to the pre-defined original one. To find these labels, we design two strategies in this study, including forward label extension and backward label validation. We also leverage the extended labels to improve model performance. Our comprehensive experiments show that spurious associations occur consistently in both Chinese and English datasets across various LLM sizes. Moreover, the use of extended labels significantly enhances LLM performance in information extraction tasks. Remarkably, there is a performance increase of 9.55%, 11.42%, and 21.27% in F1 scores on the SciERC, ACE05, and DuEE datasets, respectively. ? 2024 Association for Computational Linguistics.