文献类型：期刊文献

中文题名：基于RoBERTa和对抗训练的中文医疗命名实体识别

英文题名：Chinese Medical Named Entity Recognition Based on RoBERTa and Adversarial Training

作者：郭瑞[1];张欢欢[1]

机构：[1]华东理工大学信息科学与工程学院,上海200237

年份：2023

卷号：49

期号：1

起止页码：144

中文期刊名：华东理工大学学报（自然科学版）

外文期刊名：Journal of East China University of Science and Technology

收录：Scopus;北大核心:【北大核心2020】；CSCD:【CSCD_E2023_2024】；

语种：中文

中文关键词：医疗命名实体识别;RoBERTa-wwm-ext-large;对抗训练;双向长短期记忆网络;条件随机场

外文关键词：medical named entity recognition;RoBERTa-wwm-ext-large;adversarial training;bidirectional long short-term memory network;conditional random field

摘要：BERT(Bidirectional Encoder Representations from Transformers)和神经网络模型相结合的方法目前已被广泛应用于中文医疗命名实体识别领域。但BERT在中文中是以字为粒度切分的,没有考虑到中文分词。而神经网络模型往往局部不稳定,即使微小的扰动也可能误导它们,导致模型的鲁棒性差。为了解决这两个问题,提出了一种基于RoBERTa(A Robustly OptimizedBERTPre-trainingApproach)和对抗训练的中文医疗命名实体识别模型(ATRBC)。首先,使用RoBERTa-wwm-ext-large(ARobustlyOptimizedBERTPre-training Approach-whole word masking-extended data-large)预训练模型得到输入文本的初始向量表示;其次,在初始向量表示上添加一些扰动来生成对抗样本;最后,将初始向量表示和对抗样本一同依次输入双向长短期记忆网络和条件随机场中,得到最终的预测结果。在CCKS 2019数据集上的实验结果表明,AT-RBC模型的F1值达到了88.96%;在Resume数据集上的实验结果表明,AT-RBC模型的F1值也达到了97.14%,证明了该模型的有效性。
Recently, the method of combining BERT(Bidirectional Encoder Representations from Transformers)and neural network model has been widely used in the field of Chinese medical named entity recognition. However,BERT is word segmentation in Chinese, without considering Chinese word segmentation. Neural network models are often locally unstable, and even small disturbances may mislead them and result in poor model robustness. In order to solve these two problems, this paper proposes a Chinese medical named entity recognition model based on RoBERTa(A Robustly Optimized BERT Pre-training Approach) and adversarial training, namely AT-RBC(Adversarial Training with RoBERTa-wwm-ext-large+BiLSTM+CRF). Firstly, RoBERTa-wwm-ext-large(A Robustly Optimized BERT Pre-training Approach-whole word masking-extended data-large) pre-trained model is utilized to obtain the initial vector representation of input text. Secondly, some perturbations are added to the initial vector representation for generating adversarial samples. Finally, the initial vector representation and adversarial samples are sequentially inputted into bidirectional long short-term memory network and conditional random field to obtain the final prediction.Finally, it is shown via the experiments that the F1 value of AT-RBC model on the CCKS 2019 data set can reach 88.96%, and this value on the resume dataset reaches 97.14%.