文献类型：期刊文献

英文题名：Transformer-based cross-modal multi-contrast network for ophthalmic diseases diagnosis

作者：Yu, Yang[1];Zhu, Hongqing[1,2]

机构：[1]East China Univ Sci & Technol, Sch Informat Sci & Engn, Shanghai 200237, Peoples R China;[2]East China Univ Sci & Technol, Sch Informat Sci & Engn, 130 Mei Long Rd, Shanghai 200237, Peoples R China

年份：2023

卷号：43

期号：3

起止页码：507

外文期刊名：BIOCYBERNETICS AND BIOMEDICAL ENGINEERING

收录：;WOS:【SCI-EXPANDED(收录号:WOS:001033991600001)】；

基金：This work was supported by the National Nature Science Foundation of China under Grant 61872143.

语种：英文

外文关键词：Ophthalmic disease diagnosis; Vision transformer; Cross -modal; Contrastive learning; Feature fusion

摘要：Automatic diagnosis of various ophthalmic diseases from ocular medical images is vital to support clinical decisions. Most current methods employ a single imaging modality, espe-cially 2D fundus images. Considering that the diagnosis of ophthalmic diseases can greatly benefit from multiple imaging modalities, this paper further improves the accuracy of diag-nosis by effectively utilizing cross-modal data. In this paper, we propose Transformer -based cross-modal multi-contrast network for efficiently fusing color fundus photograph (CFP) and optical coherence tomography (OCT) modality to diagnose ophthalmic diseases. We design multi-contrast learning strategy to extract discriminate features from cross -modal data for diagnosis. Then channel fusion head captures the semantically shared information across different modalities and the similarity features between patients of the same category. Meanwhile, we use a class-balanced training strategy to cope with the situation that medical datasets are usually class-imbalanced. Our method is evaluated on public benchmark datasets for cross-modal ophthalmic disease diagnosis. The experi-mental results demonstrate that our method outperforms other approaches. The codes and models are available at https://github.com/ecustyy/tcmn. & COPY; 2023 Nalecz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences. Published by Elsevier B.V. All rights reserved.