文献类型：期刊文献

英文题名：QUERT: Continual Pre-training of Language Model for Query Understanding in Travel Domain Search

作者：Xie, Jian[1]; Liang, Yidan[2]; Liu, Jingping[3]; Xiao, Yanghua[1]; Wu, Baohua[2]; Ni, Shenghua[2]

机构：[1] Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University, Shanghai, China; [2] Alibaba Group, Hangzhou, China; [3] School of Information Science and Engineering, East China University of Science and Technology, Shanghai, China

年份：2023

起止页码：5282

外文期刊名：Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

收录：EI(收录号：20230221040)

语种：英文

外文关键词：Behavioral research - Computational linguistics - HTTP - Information retrieval

摘要：In light of the success of the pre-trained language models (PLMs), continual pre-training of generic PLMs has been the paradigm of domain adaption. In this paper, we propose QUERT, A Continual Pre-trained Language Model for QUERy Understanding in Travel Domain Search. QUERT is jointly trained on four tailored pre-training tasks to the characteristics of query in travel domain search: Geography-aware Mask Prediction, Geohash Code Prediction, User Click Behavior Learning, and Phrase and Token Order Prediction. Performance improvement of downstream tasks and ablation experiment demonstrate the effectiveness of our proposed pre-training tasks. To be specific, the average performance of downstream tasks increases by 2.02% and 30.93% in supervised and unsupervised settings, respectively. To check on the improvement of QUERT to online business, we deploy QUERT and perform A/B testing on Fliggy APP. The feedback results show that QUERT increases the Unique Click-Through Rate and Page Click-Through Rate by 0.89% and 1.03% when applying QUERT as the encoder. Resources are available at https://github.com/hsaest/QUERT ? 2023 ACM.