文献类型：期刊文献

英文题名：Hierarchical Prompt Tuning for Few-Shot Multi-Task Learning

作者：Liu, Jingping[1]; Chen, Tao[2]; Liang, Zujie[3]; Jiang, Haiyun[2]; Xiao, Yanghua[2]; Wei, Feng[3]; Qian, Yuxi[3]; Hao, Zhenghong[3]; Han, Bing[3]

机构：[1] School of Information Science and Engineering, East China University of Science and Technology, Shanghai, China; [2] Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University, Shanghai, China; [3] Ant Group, Shanghai, China

年份：2023

起止页码：1556

外文期刊名：International Conference on Information and Knowledge Management, Proceedings

收录：EI(收录号：20234815122624)

语种：英文

外文关键词：Knowledge management - Semantics

摘要：Prompt tuning has enhanced the performance of Pre-trained Language Models for multi-task learning in few-shot scenarios. However, existing studies fail to consider that the prompts among different layers in Transformer are different due to the diverse information learned at each layer. In general, the bottom layers in the model tend to capture low-level semantic or structural information, while the upper layers primarily acquire task-specific knowledge. Hence, we propose a novel hierarchical prompt tuning model for few-shot multi-task learning to capture this regularity. The designed model mainly consists of three types of prompts: shared prompts, auto-adaptive prompts, and task-specific prompts. Shared prompts facilitate the sharing of general information across all tasks. Auto-adaptive prompts dynamically select and integrate relevant prompt information from all tasks into the current task. Task-specific prompts concentrate on learning task-specific knowledge. To enhance the model's adaptability to diverse inputs, we introduce deep instance-aware language prompts as the foundation for constructing the above prompts. To evaluate the effectiveness of our proposed method, we conduct extensive experiments on multiple widely-used datasets. The experimental results demonstrate that the proposed method achieves state-of-the-art performance for multi-task learning in few-shot settings and outperforms ChatGPT in the full-data setting. ? 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.