文献类型：期刊文献

中文题名：基于异构低秩多模态融合网络的后囊膜混浊预测

英文题名：Prediction of posterior capsular opacification based on heterogeneous low-rank multimodal fusion network

作者：陈志刚[1];万永菁[1];王于蓝[2];蒋翠玲[1];陈霞[2]

机构：[1]华东理工大学信息科学与工程学院,上海200237;[2]上海市眼病防治中心,上海200041

年份：2021

卷号：55

期号：11

起止页码：2045

中文期刊名：浙江大学学报（工学版）

外文期刊名：Journal of Zhejiang University：Engineering Science

收录：CSTPCD;;EI(收录号：20214811246789);Scopus;北大核心:【北大核心2020】；CSCD:【CSCD2021_2022】；

基金：国家自然科学基金资助项目(61872143);上海市申康医院发展中心临床科技创新资助项目(SHDC2018X16);上海市卫生健康委员会卫生行业临床研究专项课题资助项目(20204Y0218).

语种：中文

中文关键词：异构;低秩分解;多模态融合;后照影像;后囊膜混浊;计算机辅助诊断

外文关键词：heterogeneity;low-rank decomposition;multimodal fusion;retro-illumination image;posterior capsular opacification;computer-aided diagnosis

摘要：针对后囊膜混浊并发症发病周期长、筛查范围广的问题,提出利用多模态机器学习预测后囊膜混浊并发症的计算机辅助诊断方法.对后照影像进行感兴趣区域(ROI)提取和白色反光区域填充,所构建的异构低秩多模态融合网络(HLMF)能同时输入后照影像和视觉质量参数进行特征提取与融合,HLMF模型基于通道积融合多模态信息;采用卷积核参数低秩分解解决过拟合问题;选用Focal Loss损失函数解决类别不均衡的问题;在训练过程中还采用预训练和模态腐蚀的训练方法,使模型更好地提取单一模态的特征并进行融合.该算法在后囊膜混浊数据集上的十折交叉验证准确率为95.63%,F1分数为96.72%.实验结果表明,所提算法能较好地提取单模态特征并进行特征融合,相比于其他多模态融合模型有更好的性能.
A computer-aided diagnosis method for posterior capsular opacification using multimodal machine learning was proposed for the long incidence cycle and wide screening range of the complication of posterior capsular opacification.The region of interest(ROI)was extracted and the white reflective region was filled on the retro-illumination image.Retro-illumination image and visual quality data can be input into constructed heterogeneous low-rank multimodal fusion network(HLMF)simultaneously for performing feature extraction and fusion,and multimodal information was fused based on channel product.Overfitting problem was solved by low-rank decomposition of convolution kernel parameters,and the Focal Loss was chosen to solve the problem of uneven category.The pre-training and corrupted augmentation methods were used in the training process to better extract and fuse the features of single modality.The accuracy and F1 score of 10-fold cross-validation of the algorithm on the posterior capsule opacification dataset were 95.63%and 96.72%,respectively.Experimental results show that the proposed algorithm can extract single modality features and perform feature fusion well,and has better performance compared with other multimodal fusion models.