文献类型：期刊文献

英文题名：Large Language Models guided Generative Prompt for Dialogue Generation

作者：Liu, Sijie[1]; Fang, Yiquan[1]; Cheng, Hua[1]; Pan, Yiming[1]; Liu, Yufei[1]; Gao, Caiting[1]

机构：[1] East China University of Science and Technology, School of Information Science and Engineering, Shanghai, China

年份：2023

起止页码：10

外文期刊名：Proceedings - 2023 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, CyberC 2023

收录：EI(收录号：20241015709032)

语种：英文

外文关键词：Computational linguistics - Computer vision

摘要：The applications of large language models (LLMs) such as ChatGPT exhibit impressive comprehension and generative capabilities in dialogue task. LLMs require massive high-quality data and computational cost, which limits their application to low-resource tasks. Dialogue generation when using smaller language models like GPT-2 encounters difficulties in maintaining context consistency. To address the problem of dialogue generation under resource constraints, we propose an LLM-guided Generative Prompt method (LGP). LGP enhances the relevance and coherence of generated dialogues through a smaller model GPT-2 and generative prompt (GP). GP is produced by the proposed Prompt Network, which leverages prompt encoder to learn dialogue history features and utilizes LSTM to extract contextual temporal features. Therefore, GP shown as the simple fixed-length learnable embeddings can replace the original complex and redundant context in GPT-2. The few-shot training of GP is guided by the LLM's responses, which facilitates GPT-2 in generating more contextually consistent and comprehensive responses. Experiments on the DailyDialog and MultiWOZ datasets show that LGP achieves high improvements in BLEU, NIST, METEOR and ROUGE-L metrics. Remarkably, LGP achieves these results with approximately 18% of the training data, surpassing other full-data-finetuning methods in automatic evaluation metrics. ? 2023 IEEE.