文献类型：期刊文献

英文题名：Multi-attention mutual information distributed framework for few-shot learning

作者：Wang, Zhe[1,2];Ma, Pingchuan[1,2];Chi, Ziqiu[1,2];Li, Dongdong[2];Yang, Hai[2];Du, Wenli[1]

机构：[1]East China Univ Sci & Technol, Lab Smart Mfg Energy Chem Proc, Minist Educ, Shanghai 200237, Peoples R China;[2]East China Univ Sci & Technol, Dept Comp Sci & Engn, Shanghai 200237, Peoples R China

年份：2022

卷号：202

外文期刊名：EXPERT SYSTEMS WITH APPLICATIONS

收录：;EI(收录号：20221812057204);WOS:【SCI-EXPANDED(收录号:WOS:000804926500010)】；

基金：This work is supported by Shanghai Science and Technology Program "Distributed and generative few-shot algorithm and theory research", PR China under Grant No. 20511100600, Shanghai Science and Technology Program "Federated based cross-domain and cross-task incremental learning, PR China under Grant No. 21511100800, Natural Science Foundation of China under Grant No. 62076094.

语种：英文

外文关键词：Few-shot learning; Attention mechanism; Mutual learning; Distributed learning

摘要：The purpose of few-shot learning is to learn a classifier, even if only a limited number of samples are used, a good generalization effect can be achieved. Recently, many methods based on meta-learning learn a large number of multi-classification tasks to train a general classifier to solve this problem. Methods based on metric learning use the distance relationship between labeled samples and unlabeled samples for classification. These methods all have good performance. However, these methods rarely pay attention to the problems of insufficient feature extraction and low training efficiency. To this end, we propose multi-attention mutual information distributed framework for few-shot learning (MAMD). Specifically, we use the attention mechanism to help the feature embedding module extract more representative features. We use multiple attention mechanisms because different attention mechanisms can focus on different features. After multiple attention modules extract features, we use the mutual learning to aggregate the extracted features. The mutual learning method is similar to knowledge distillation. The difference between knowledge distillation and mutual learning is that mutual learning does not require a large network to guide a small network, but two networks learn from each other and progress together. In addition, we use distributed learning to improve the training speed and shorten the consumption of time. We combine distributed learning with few-shot learning for the first time and propose the concept of distributed few-shot learning. It provides a new direction for few-shot learning. We evaluate MAMD on two few-shot learning benchmark datasets and achieve the expected results.