文献类型：期刊文献

英文题名：Stroke Outcome Prediction via Multi-level Feature and Multi-modal Fusion Network

作者：Xiao, Ting[1]; Shi, Lei[1]; Wang, Hao[1]; Wang, Zhe[1]; Lin, Yi[2]

机构：[1] East China University of Science and Technology, Department of Information Science and Engineering, Shanghai, 200237, China; [2] Harbin Medical University, School of Interdisciplinary Medicine and Engineering, Harbin, 150076, China

年份：2024

起止页码：6732

外文期刊名：Proceedings - 2024 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2024

收录：EI(收录号：20250717854458)

语种：英文

外文关键词：Deep learning - Image coding - Medical image processing - Multi-task learning - Network coding - Prediction models

摘要：Deep learning has been widely applied in medical fields, including medical image analysis, disease prediction, drug development, and clinical decision-making. In disease prediction, researchers have developed advanced models to enhance accuracy. However, medical data comes in various forms, such as tables and images, and few models effectively leverage both modalities for prediction, particularly in stroke outcomes. This paper focuses on predicting outcomes for ischemic stroke and proposes a deep learning model that utilizes multi-level image features and multi-modal feature fusion for disease prediction. The model is designed as a multi-task learning framework that simultaneously addresses image segmentation and disease prediction tasks. The segmentation task employs a U-shaped network, with an encoder capable of extracting multi-level image features from various encoder blocks. These multi-level features are fused through a multi-level feature fusion (MFF) module, providing more direct information about lesions for subsequent multi-modal feature fusion. For the prediction task, clinical features are extracted from tabular data and then integrated with image features via a multi-modal fusion (MMF) module. The MMF module employs an attention mechanism to comprehensively integrate medical image features with each variable in the clinical data. The framework is trained with segmentation and classification losses for joint supervision, enabling the model to segment lesion locations while simultaneously performing disease prediction. Extensive experimental results demonstrate that our framework can achieve the best prediction performance. ? 2024 IEEE.