文献类型：期刊文献

英文题名：Few Adjustable Parameters Prediction Model Based on Lightweight Prefix-Tuning: Learning Session Dropout Prediction Model Based on Parameter-Efficient Prefix-Tuning

作者：Lu, Yuantong[1];Wang, Zhanquan[1]

机构：[1]East China Univ Sci & Technol, Sch Informat Sci & Engn, Shanghai 200237, Peoples R China

年份：2024

卷号：14

期号：23

外文期刊名：APPLIED SCIENCES-BASEL

收录：;EI(收录号：20245117528342);WOS:【SCI-EXPANDED(收录号:WOS:001376312900001)】；

基金：This research was funded by [Smart Education Platform Solution Key Technology Consulting Services, Shanghai Xinyan Information Technology Co., Ltd.] grant number [H300-72308] and [Relying on Enterprises to Implement Professional Integration: Construction and Practice of the Artificial Intelligence Course Dell-Ministry of Education Industry-Academia Collaboration Project] grant number [H300-42211].

语种：英文

外文关键词：prefix-tuning; learning session dropout prediction; datamining; smart education; limited data scenario

摘要：Featured Application This work can be applied to an online education platform to predict students' current learning states.Abstract In response to the challenge of low predictive accuracy in scenarios with limited data, we propose a few adjustable parameters prediction model based on lightweight prefix-tuning (FAP-Prefix). Prefix-tuning is an efficient fine-tuning method that only adjusts prefix vectors while keeping the model's original parameters frozen. In each transformer layer, the prefix vectors are connected with the internal key-value pair of the transformer structure. By training on the synthesized sequence of the prefix and original input with masked learning, the transformer model learns the features of individual learning behaviors. In addition, it can also discover hidden connections of continuous learning behaviors. During fine-tuning, all parameters of the pre-trained model are frozen, and downstream task learning is accomplished by adjusting the prefix parameters. Continuous trainable prefix vectors can influence subsequent vector representations, leading to the generation of session dropout prediction results. The experiments show that FAP-Prefix significantly outperforms traditional methods in data-limited settings, with AUC improvements of +4.58%, +3.53%, and +8.49% under 30%, 10%, and 1% data conditions, respectively. It also surpasses state-of-the-art models in prediction performance (AUC +5.42%, ACC +5.3%, F1 score +5.68%).