文献类型：期刊文献

英文题名：Federated probability memory recall for federated continual learning

作者：Wang, Zhe[1,2];Zhang, Yu[1,2];Xu, Xinlei[1,2];Fu, Zhiling[1,2];Yang, Hai[2];Du, Wenli[1]

机构：[1]East China Univ Sci & Technol, Key Lab Smart Mfg Energy Chem Proc, Minist Educ, Shanghai 200237, Peoples R China;[2]East China Univ Sci & Technol, Dept Comp Sci & Engn, Shanghai 200237, Peoples R China

年份：2023

卷号：629

起止页码：551

外文期刊名：INFORMATION SCIENCES

收录：;EI(收录号：20230713597782);WOS:【SCI-EXPANDED(收录号:WOS:000953442700001)】；

基金：This work is supported by Shanghai Science and Technology Program "Federated based cross-domain and cross-task incremental learning" under Grant No. 21511100800, Shanghai Science and Technology Program "Distributed and generative few-shot algorithm and theory research" under Grant No. 20511100600, Natural Science Foundation of China under Grant No. 62076094. Chinese Defense Program of Science and Technology under Grant No. 2021-JCJQ-JJ-0041, China Aerospace Science and Technology Corporation Industry-University-Research Cooperation Foundation of the Eighth Research Institute under Grant No. SAST2021-007.

语种：英文

外文关键词：Catastrophic forgetting; Federated continual learning; Knowledge distillation

摘要：Federated Continual Learning (FCL) approaches exist two major problems of the probability bias and the imbalance in parameter variations. These two problems lead to catastrophic forgetting of the network in the FCL process. Therefore, this paper proposes a novel FCL framework, Federated Probability Memory Recall (FedPMR), to mitigate the probability bias problem and the imbalance in parameter variations. Firstly, for the probability bias problem, this paper designs the Probability Distribution Alignment (PDA) module, which consolidates the memory of old probability experience. Specifically, PDA maintains a replay buffer and uses the probability memory stored in the buffer to correct the offset probabilities of the previous tasks during the two-stage training. Secondly, to alleviate the imbalance in parameter variations, this paper designs the Parameter Consistency Constraint (PCC) module, which constrains the magnitude of neural weight changes for previous tasks. Concretely, PCC applies a set of adaptive weights to subsets of the regularization term that constrains parameter changes, forcing the current model to be sufficiently close to the past model in parameter space distance. Experiments with various levels of task similitude across clients demonstrate that our technique establishes the new state-of-the-art performance when compared to previous FCL approaches.