文献类型：期刊文献

英文题名：A Deep Learning Multimodal Fusion-Based Method for Cell and Nucleus Segmentation

作者：Shen, Bin[1,2];Gu, Zhen[1];Zhou, Jiale[1];Yan, Bingyong[1];Fang, Yiquan[2];Wang, Huifeng[1]

机构：[1]East China Univ Sci & Technol, Key Lab Smart Mfg Energy Chem Proc, Minist Educ, Shanghai 200237, Peoples R China;[2]East China Univ Sci & Technol, Informatizat & Data Management Ctr, Shanghai 200237, Peoples R China

年份：2026

卷号：45

期号：1

起止页码：112

外文期刊名：IEEE TRANSACTIONS ON MEDICAL IMAGING

收录：;EI(收录号：20253318972334);WOS:【SCI-EXPANDED(收录号:WOS:001683430000029)】；

基金：This work was supported in part by the National Key Research and Development Program of China under Grant 2023YFA1802002 and Grant 2021YFC2101101 and in part by the National Natural Science Foundation of China under Grant 62103148.

语种：英文

外文关键词：Image segmentation; Computer architecture; Microprocessors; Training; Deep learning; Weak supervision; Visualization; Computational modeling; Transfer learning; Linguistics; Cellular image; deep learning; cell segmentation; nucleus segmentation; multimodal

摘要：In recent years, deep learning has been widely utilized in the fields of biomedical image segmentation and cellular image analysis. Supervised deep neural networks trained on annotated data have demonstrated good performance in tasks related to cell and nucleus segmentation. However, the use of supervised models necessitates carefully constructed training data and a substantial amount of ground truth information. Unfortunately, high-quality annotated data for cellular images are scarce. To address the issue of limited datasets, we propose a cell and nucleus segmentation method based on deep learning multimodal fusion. The proposed method includes three modules: a segmentation fundamental module, a multimodal prompter module, and an object output module. This comprehensive approach enables cell and nucleus segmentation tasks to be performed without the need for retraining on new data. The segmentation fundamental module is the core of the framework, as it provides essential segmentation capabilities. By leveraging preexisting models trained on natural imagery, this module effectively performs cell segmentation by incorporating prior knowledge. The multimodal prompter module, a pretrained model, aids in combining image and textual information. It employs a data fusion technique for multiple modalities to deliver prompts that steer the network's output, thereby avoiding the constraints inherent to single-modality approaches. The object output module combines the inputs from the preceding modules to generate the final segmentation output. The experimental validation confirms the superiority of the proposed method, which outperforms comparative methods in cell and nucleus segmentation tasks and has promise for future applications in cell tracking.