文献类型：会议论文

英文题名：Stroke Outcome Prediction via Multi-level Feature and Multi-modal Fusion Network

作者：Xiao, Ting[1];Shi, Lei[1];Wang, Hao[1];Wang, Zhe[1];Lin, Yi[2]

机构：[1]East China Univ Sci & Technol, Dept Informat Sci & Engn, Shanghai 200237, Peoples R China;[2]Harbin Med Univ, Sch Interdisciplinary Med & Engn, Harbin 150076, Peoples R China

会议论文集：2024 International Conference on Bioinformatics and Biomedicine

会议日期：DEC 03-06, 2024

会议地点：Lisbon, PORTUGAL

语种：英文

外文关键词：Disease Prediction; Multi-modal Fusion; Ischemic Stroke; Multi-level Feature Fusion

摘要：Deep learning has been widely applied in medical fields, including medical image analysis, disease prediction, drug development, and clinical decision-making. In disease prediction, researchers have developed advanced models to enhance accuracy. However, medical data comes in various forms, such as tables and images, and few models effectively leverage both modalities for prediction, particularly in stroke outcomes. This paper focuses on predicting outcomes for ischemic stroke and proposes a deep learning model that utilizes multi-level image features and multi-modal feature fusion for disease prediction. The model is designed as a multi-task learning framework that simultaneously addresses image segmentation and disease prediction tasks. The segmentation task employs a U-shaped network, with an encoder capable of extracting multi-level image features from various encoder blocks. These multi-level features are fused through a multi-level feature fusion (MFF) module, providing more direct information about lesions for subsequent multi-modal feature fusion. For the prediction task, clinical features are extracted from tabular data and then integrated with image features via a multi-modal fusion (MMF) module. The MMF module employs an attention mechanism to comprehensively integrate medical image features with each variable in the clinical data. The framework is trained with segmentation and classification losses for joint supervision, enabling the model to segment lesion locations while simultaneously performing disease prediction. Extensive experimental results demonstrate that our framework can achieve the best prediction performance.