文献类型：期刊文献

英文题名：In-Batch Negatives' Enhanced Self-Supervised Learning

作者：Tang, Yixin[1]; Cheng, Hua[1]; Fang, Yiquan[1]; Pan, Yiming[1]

机构：[1] School of Information Science and Engineering, East China University of Science and Technology, Shanghai, China

年份：2022

卷号：2022-October

起止页码：161

外文期刊名：Proceedings - International Conference on Tools with Artificial Intelligence, ICTAI

收录：EI(收录号：20231914055215)

语种：英文

外文关键词：Classification (of information) - Text processing - Vector spaces

摘要：In contrastive self-supervised learning, such as Sim-CLR and MOCO, negative samples play an important role on the model's robustness. Instead of increasing the negatives' number, we promote the in-batch negatives' quality by adjusting their embedding. In-Batch Negatives' Enhanced Self-Supervised Learning (IBN-SSL) focus on negatives' quality by an importance-weighted algorithm and an online boundary. The importance-weighted negatives' quality algorithm decreases the negatives' relative locations from positive in projection space, which pushes the model to learn to distinguish positives from harder negatives. And the online boundary compresses negatives' vector space and keep negatives farther from positive. A better representation model can be leaned by IBN-SSL, and experiments show that it outperforms both RoBERTa and SimCLR in text classification task and similar sentence pair task. ? 2022 IEEE.