文献类型：期刊文献

英文题名：SECL: Sampling enhanced contrastive learning

作者：Tang, Yixin[1];Cheng, Hua[1];Fang, Yiquan[1];Cheng, Tao[1]

机构：[1]East China Univ Sci & Technol, Sch Informat Sci & Engn, Shanghai 200237, Peoples R China

年份：2023

卷号：36

期号：1

起止页码：1

外文期刊名：AI COMMUNICATIONS

收录：;WOS:【SCI-EXPANDED(收录号:WOS:000935004900001)】；

语种：英文

外文关键词：Contrastive learning; sampling enhancement; InfoNCE loss; model collapse

摘要：Instance-level contrastive learning such as SimCLR has been successful as a powerful method for representation learning. However, SimCLR suffers from problems of sampling bias, feature bias and model collapse. A set-level based Sampling Enhanced Contrastive Learning based on SimCLR (SECL) is proposed in this paper. We use the proposed super-sampling method to expand the augmented samples into a contrastive-positive set, which can learn class features of the target sample to reduce the bias. The contrastive-positive set includes Augmentations (the original augmented samples) and Neighbors (the supersampled samples).We also introduce a samples-correlation strategy to prevent model collapse, where a positive correlation loss or a negative correlation loss is computed to adjust the balance of model's Alignment and Uniformity. SECL reaches 94.14% classification precision on SST-2 dataset and 89.25% on ARSC dataset. For the multi-class classification task, SECL achieves 90.99% on AGNews dataset. They are all about 1% higher than the precision of SimCLR. Experiments show that the training convergence of SECL is faster, and SECL reduces the risk of bias and model collapse.